How to update a value of a shared variable in Theano efficiently?

I try to implement non-negative matrix factorization in Theano. In more detail, I try to find two matrices L and R such that their product L x R represents a give matrix M as accurate as possible.

For finding L and R matrices I use back propagation. At some point I have noticed that values in L and R can be negative (of course nothing prevents back prop from doing that). I have tried to correct this behavior by adding the following lines after the back propagation step:

self.L.set_value(T.abs_(self.L).eval())
self.R.set_value(T.abs_(self.R).eval())

After that my program became much more slower.

Am I doing something wrong? Do I update the values of the tensors in a wrong way? Is there a way to do it faster?

ADDED

As requested in the comments, I provide more code. This is how I define the function in the __init__.

self.L = theano.shared(value=np.random.rand(n_rows, n_hids), name='L', borrow=True)
self.R = theano.shared(value=np.random.rand(n_hids, n_cols), name='R', borrow=True)
Y = theano.dot(self.L, self.R)

diff = X - Y
D = T.pow(diff, 2)
E = T.sum(D)
gr_L = T.grad(cost=E, wrt=self.L)
gr_R = T.grad(cost=E, wrt=self.R)

self.l_rate = theano.shared(value=0.000001)
L_ups = self.L - self.l_rate*gr_L
R_ups = self.R - self.l_rate*gr_R

updates = [(self.L, L_ups), (self.R, R_ups)]
self.backprop = theano.function([X], E, updates=updates)

Then in my train function I had this code:

for i in range(self.n_iter):
    costs = self.backprop(X, F)

    self.L.set_value(T.abs_(self.L).eval())
    self.R.set_value(T.abs_(self.R).eval())

A minor remark, I use the abs_ function, but it would make actually more sense to use a function that replace negative values by zero.

Solution

You can force the symbolic update values for L and R to always be positive like this:

self.l_rate = theano.shared(value=0.000001)
L_ups = self.L - self.l_rate*gr_L
R_ups = self.R - self.l_rate*gr_R

# This force R and L to always be updated to a positive value
L_ups_abs = T.abs_(L_ups)
R_ups_abs = T.abs_(R_ups)

# Use the update L_ups_abs instead of L_ups (same with R_ups)
updates = [(self.L, L_ups_abs), (self.R, R_ups_abs)]
self.backprop = theano.function([X], E, updates=updates)

and remove the lines

self.L.set_value(T.abs_(self.L).eval())
self.R.set_value(T.abs_(self.R).eval())

from your training loop