Using Keras fit_generator gives an error of wrong shape

Question

I am getting an error on fit_generator. My generator returns the following:

yield(row.values, label)

For example, using it:

myg = generate_array()
for i in myg:
    print((i[0].shape))
    print(i)
    break

(9008,)
(array([0.116516, 0.22419 , 0.03373 , ..., 0.      , 0.      , 0.      ]), 0)

But the following throws an exception:

model = Sequential()
model.add(Dense(84, activation='relu', input_dim=9008))

ValueError: Error when checking input: expected dense_1_input to have shape 
(9008,) but got array with shape (1,)

Any idea?

Answer 1

As suggested by Kota Mori: data generator needs to give a batch of data, not a single sample. See e.g.: https://stanford.edu/~shervine/blog/keras-how-to-generate-data-on-the-fly

Since I want a stochastic gradient descent (batch size is one) the following code fixed the problem:

def generate_array():
   while True:
    X = np.empty((1, 9008))
    y = np.empty((1), dtype=int)
    # Some processing
    X[0] = row
    y[0] = label
    yield(X,y)