Use part of the layers of VGG19 in Keras with TimeDistributed layer

I want to use the first 9 layers of the trained VGG19 model combined with TimeDistributed layers. But I get an InvalidArgumentError.

def build_vgg(in_shape):
    vgg = VGG19(weights="imagenet")
    vgg.outputs = [vgg.layers[9].output]
    img = keras.Input(in_shape)
    img_features = vgg(img)
    return keras.Model(img, img_features)

vggmodel = build_vgg((50,50,3))
input_layer = keras.Input(batch_shape=(10,10,50,50,3))
h2 = keras.layers.wrappers.TimeDistributed(vggmodel)(input_layer)
model = keras.Model(input_layer,h2)
model.summary()

I'm getting this error:

InvalidArgumentError                      Traceback (most recent call last)
~/.conda/envs/py3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
   1566   try:
-> 1567     c_op = c_api.TF_FinishOperation(op_desc)
   1568   except errors.InvalidArgumentError as e:

InvalidArgumentError: Dimensions must be equal, but are 512 and 25088 for 'time_distributed_1/vgg19/fc1/MatMul' (op: 'MatMul') with input shapes: [10,512], [25088,4096].

Solution

First, your model should not use an additional input in build_vgg. You should only take the tensors you want.

Second, you should use a compatible input shape.

Third, you can't include top if you're changing the input shape AND loading imagenet weights:

def build_vgg(in_shape):
    vgg = VGG19(weights="imagenet", input_shape= in_shape, include_top = False)
    outputs = vgg.layers[9].output

    return keras.Model(vgg.input, outputs)

Then the rest

vggmodel = build_vgg((50,50,3))
#vggmodel.summary()
input_layer = keras.Input(batch_shape=(10,10,50,50,3))
h2 = keras.layers.wrappers.TimeDistributed(vggmodel)(input_layer)
model = keras.Model(input_layer,h2)
model.summary()

model.predict(np.ones((10,10,50,50,3)))