I've been experimenting with very basic recurrent networks and have seen really strange behaviors. I've spent quite a bit of time trying to narrow down where it goes wrong and I ended up finding that the gradients computed by theano and by finite differentiation are radically different when using a recurrent layer. What is going on here?
Here is the kind of problem I have:
I have n_seq sequences of n_steps feature vectors of dimension n_feat, along with their labels among n_class classes. The labels are per time step, not per sequence (so I have n_seq*n_steps labels). My goal is to train a model to correctly classify the feature vectors.
Here's my minimal example:
(In reality, there would be some sequential information in the data, hence recurrent networks should do better, but in this minimal example I generate purely random data, which is good enough to expose the bug.)
I create 2 minimal networks:
1) A regular feed-forward (not recurrent), with just the input layer and an output layer with softmax (no hidden layer). I discard the sequential information by considering a "batch" of n_seq*n_steps "independent" feature vectors.
2) An identical network but where the output layer is recurrent. The batch is now of size n_seq and each input is a full sequence of n_steps feature vectors. Finally I reshape the output back to a "batch" of size n_seq*n_steps.
If the recurrent weights are set to 0, the 2 networks should be equivalent. Indeed I do see that the initial loss for both networks are the same in this case, no matter what random initialization for the feed-forward weights I have. If I implement a finite differentiation, I also get that the (initial) gradients for the feed-forward weights are the same (as they should). However the gradients obtained from theano are radically different (but only for the recurrent network).
Here is my code with sample result:
NOTE: when run for the first time, I get this warning, I don't know what triggers it but I bet it is relevant to my problem. Warning: In the strict mode, all neccessary shared variables must be passed as a part of non_sequences 'must be passed as a part of non_sequences', Warning)
Any insight would be much appreciated!
CODE:
import numpy as np
import theano
import theano.tensor as T
import lasagne
# GENERATE RANDOM DATA
n_steps = 10**4
n_seq = 10
n_feat = 2
n_class = 2
data_X = lasagne.utils.floatX(np.random.randn(n_seq, n_steps, n_feat))
data_y = np.random.randint(n_class, size=(n_seq, n_steps))
# INITIALIZE WEIGHTS
# feed-forward weights (random)
W = theano.shared(lasagne.utils.floatX(np.random.randn(n_feat,n_class)), name="W")
# recurrent weights (set to 0)
W_rec = theano.shared(lasagne.utils.floatX(np.zeros((n_class,n_class))), name="Wrec")
# bias (set to 0)
b = theano.shared(lasagne.utils.floatX(np.zeros((n_class,))), name="b")
def create_functions(model, X, y, givens):
"""Helper for building a network."""
loss = lasagne.objectives.categorical_crossentropy(lasagne.layers.get_output(model, X), y).mean()
get_loss = theano.function(
[], loss,
givens=givens
)
all_params = lasagne.layers.get_all_params(model)
get_theano_grad = [
theano.function(
[], g,
givens=givens
)
for g in theano.grad(loss, all_params)
]
return get_loss, get_theano_grad
def feedforward():
"""Creates a minimal feed-forward network."""
l_in = lasagne.layers.InputLayer(
shape=(n_seq*n_steps, n_feat),
)
l_out = lasagne.layers.DenseLayer(
l_in,
num_units=n_class,
nonlinearity=lasagne.nonlinearities.softmax,
W=W,
b=b
)
model = l_out
X = T.matrix('X')
y = T.ivector('y')
givens={
X: theano.shared(data_X.reshape((n_seq*n_steps, n_feat))),
y: T.cast(theano.shared(data_y.reshape((n_seq*n_steps,))), 'int32'),
}
return (model,) + create_functions(model, X, y, givens)
def recurrent():
"""Creates a minimal recurrent network."""
l_in = lasagne.layers.InputLayer(
shape=(n_seq, n_steps, n_feat),
)
l_out = lasagne.layers.RecurrentLayer(
l_in,
num_units=n_class,
nonlinearity=lasagne.nonlinearities.softmax,
gradient_steps=1,
W_in_to_hid=W,
W_hid_to_hid=W_rec,
b=b,
)
l_reshape = lasagne.layers.ReshapeLayer(l_out, (n_seq*n_steps, n_class))
model = l_reshape
X = T.tensor3('X')
y = T.ivector('y')
givens={
X: theano.shared(data_X),
y: T.cast(theano.shared(data_y.reshape((n_seq*n_steps,))), 'int32'),
}
return (model,) + create_functions(model, X, y, givens)
def finite_diff(param, loss_func, epsilon):
"""Computes a finitie differentation gradient of loss_func wrt param."""
loss = loss_func()
P = param.get_value()
grad = np.zeros_like(P)
it = np.nditer(P , flags=['multi_index'])
while not it.finished:
ind = it.multi_index
dP = P.copy()
dP[ind] += epsilon
param.set_value(dP)
grad[ind] = (loss_func()-loss)/epsilon
it.iternext()
param.set_value(P)
return grad
def theano_diff(net, get_theano_grad):
for p,g in zip(lasagne.layers.get_all_params(net), get_theano_grad):
if p.name == "W":
gW = np.array(g())
if p.name == "b":
gb = np.array(g())
return gW, gb
def compare_ff_rec():
eps = 1e-3 # for finite differentiation
ff, get_loss_ff, get_theano_grad_ff = feedforward()
rec, get_loss_rec, get_theano_grad_rec = recurrent()
gW_ff_finite = finite_diff(W, get_loss_ff, eps)
gb_ff_finite = finite_diff(b, get_loss_ff, eps)
gW_rec_finite = finite_diff(W, get_loss_rec, eps)
gb_rec_finite = finite_diff(b, get_loss_rec, eps)
gW_ff_theano, gb_ff_theano = theano_diff(ff, get_theano_grad_ff)
gW_rec_theano, gb_rec_theano = theano_diff(rec, get_theano_grad_rec)
print "\nloss:"
print "FF:\t", get_loss_ff()
print "REC:\t", get_loss_rec()
print "\ngradients:"
print "W"
print "FF finite:\n", gW_ff_finite.ravel()
print "FF theano:\n", gW_ff_theano.ravel()
print "REC finite:\n", gW_rec_finite.ravel()
print "REC theano:\n", gW_rec_theano.ravel()
print "b"
print "FF finite:\n", gb_ff_finite.ravel()
print "FF theano:\n", gb_ff_theano.ravel()
print "REC finite:\n", gb_rec_finite.ravel()
print "REC theano:\n", gb_rec_theano.ravel()
compare_ff_rec()
RESULTS:
loss:
FF: 0.968060314655
REC: 0.968060314655
gradients:
W
FF finite:
[ 0.23925304 -0.23907423 0.14013052 -0.14001131]
FF theano:
[ 0.23917811 -0.23917811 0.14011626 -0.14011627]
REC finite:
[ 0.23931265 -0.23907423 0.14024973 -0.14001131]
REC theano:
[ 1.77408110e-05 -1.77408110e-05 1.21677476e-05 -1.21677458e-05]
b
FF finite:
[ 0.00065565 -0.00047684]
FF theano:
[ 0.00058145 -0.00058144]
REC finite:
[ 0.00071526 -0.00047684]
REC theano:
[ 7.53380482e-06 -7.53380482e-06]
The problem was coming from non-intuitive (maybe) effect of gradient_steps clipping in BPTT as explained here: https://groups.google.com/forum/#!topic/theano-users/QNge6fC6C4s