I have searched and found similar problems, but none that seem to be the same issue as what I'm facing. I'm trying to implement a neural network with Keras using Theano backend (both up to date), which involves a Lambda layer that takes the 1-dimensional output of a layer, and converts it into an n-dimensional vector with the 1-d output repeated n times.
The problem that I seem to be running into is that at the Lambda layer Keras seems to be expecting that the input has the same dimension as the output shape I'm specifying:
x=Input(shape=(2,))
V1=Dense(1)(x)
V2=Lambda(lambda B : B[0,0]*K.ones((3,)),output_shape=(3,))(V1)
model=Model(inputs=x,outputs=V2)
rms = RMSprop()
model.compile(loss='mse', optimizer=rms)
model.predict(np.array([1,2]).reshape((1,2)))
which gives this error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-7-40a7e91d5963> in <module>()
----> 1 model.predict(np.array([1,2]).reshape((1,2)))
/Users/user/anaconda/envs/py35/lib/python3.5/site-packages/keras/engine /training.py in predict(self, x, batch_size, verbose)
1504 f = self.predict_function
1505 return self._predict_loop(f, ins,
-> 1506 batch_size=batch_size, verbose=verbose)
1507
1508 def train_on_batch(self, x, y,
/Users/user/anaconda/envs/py35/lib/python3.5/site-packages/keras/engine/training.py in _predict_loop(self, f, ins, batch_size, verbose)
1137
1138 for i, batch_out in enumerate(batch_outs):
-> 1139 outs[i][batch_start:batch_end] = batch_out
1140 if verbose == 1:
1141 progbar.update(batch_end)
ValueError: could not broadcast input array from shape (3) into shape (1)
I know there are other ways to try to do this (K.repeat_elements) but this has also given me error messages about broadcasting. Note, the problem persists even if I remove the B[0,0]* (so that the Lambda layer doesn't depend on B at all). If I change the (3,) in the K.ones and output_shape to (1,) then it seems to work.
From what I understand, the Lambda layers should be able to handle input/output pairs of differing dimension, is that correct?
In output_shape, you don't consider the batch size. So this is correct: (3,)
But in tensors, the batch size is not ignored. You need at least two dimensions in the result of your expression: (Batch_size,3).
Also, don't use elements of the tensors, use the entire tensors. I haven't found a case where it would be important or useful to use separate elements (since you're supposed to do exactly the same operation to the entire batch)
I suggest you use K.repeat_elements(B, rep=3, axis=-1)