Regularization Application in the Proximal Adagrad Optimizer in Tensorflow

Question

I have been trying to implement l1-regularization in Tensorflow using the l1_regularization_strength parameter in the ProximalAdagradOptimizer function from Tensorflow. (I am using this optimizer specifically to get a sparse solution.) I have two questions regarding the regularization.

1. Does the l1-regularization used in the optimizer apply to forward and backward propagation for a neural network or only the back propagation?
2. Is there a way to break down the optimizer so the regularization only applies to specific layers in the network?

Answer 1

1. Regularization applies neither to forward or backpropagation but to the weight updates.

2. You can use different optimizers for different layers by explicitly passing the variables to minimize to each optimizer.