Search code examples
deep-learningchainer

What is the type of xs in NStepBiLSTM in Chainer?


The manual on the NStepBiLSTM says its forward function expects xs to come in the format of a list of Variable-wrapped sequences. However I get an error that implies xs is supposed to be a np array. What am I missing?

I use this function to turn an input array into a list of variable(array) with array shape (n,1).

def cut_data(data, batchsize):
    q = data.shape[0] // batchsize
    data = data[:q*batchsize]
    data = data.reshape((batchsize, q))
    xs = []
    for i in range(q):
        a = data[:,i].reshape(batchsize,1)
        xs.append(Variable(a))
    return xs

But when I call my predictor with such an xs I get this error:

<ipython-input-27-cb554613ad71> in __call__(self, xs, ts)
     10         batchlen = len(xs)
     11         loss = F.sum(F.mean_squared_error(
---> 12         self.predictor(xs), ts, reduce='no')) / batch
     13 
     14         chainer.report({'loss': loss}, self)

<ipython-input-28-ce9434e91153> in __call__(self, x)
     12     def __call__(self, x):
     13         self.h, self.c, y = self.lstm(self.h,self.c,x)
---> 14         output = self.out(y)
     15         return output
     16 

~\Anaconda2\lib\site-packages\chainer\links\connection\linear.py in __call__(self, x)
    127             in_size = functools.reduce(operator.mul, x.shape[1:], 1)
    128             self._initialize_params(in_size)
--> 129         return linear.linear(x, self.W, self.b)

~\Anaconda2\lib\site-packages\chainer\functions\connection\linear.py in linear(x, W, b)
    165 
    166     """
--> 167     if x.ndim > 2:
    168         x = x.reshape(len(x), -1)
    169 

AttributeError: 'list' object has no attribute 'ndim'

This is my simple network:

class LSTM_RNN(Chain):

    def __init__(self, n_hidden, n_input=1, n_out=1):
        super(LSTM_RNN, self).__init__()
        with self.init_scope():
            self.lstm = L.NStepBiLSTM(n_layers=n_hidden, in_size=n_input, out_size=n_out, dropout=0.5)
            self.out = L.Linear(n_hidden, n_out)
            self.h = None
            self.c = None

    def __call__(self, x):
        self.h, self.c, y = self.lstm(self.h,self.c,x)
        output = self.out(y)
        return output

    def reset_state(self):
        self.h = None
        self.c = None

Solution

  • The error message

    ---> 14         output = self.out(y)
    

    indicates that your error is triggered by self.out() method, not self.lstm()

    According to the official API reference, L.NStepBiLSTM.run() returns a tuple of hy, cy, and ys, which ys is a list.

    Your code

        def __call__(self, x):
            self.h, self.c, y = self.lstm(self.h,self.c,x)
            output = self.out(y)
            return output
    

    indicates that y (which is referred as ys in the official document) is directly passed to self.out, which is L.Linear.__call__. This causes type-mismatch.

    In general, the shape of y in ys differs each other, because x in xs can be a sequence of different lengths.

    If you need more help, please feel free to ask questions!