I have two different problems occurs at the same time.
I am having dimensionality problems with MaxPooling2d and having same dimensionality problem with DQNAgent.
The thing is, I can fix them seperately but cannot at the same time.
First Problem
I am trying to build a CNN network with several layers. After I build my model, when I try to run it, it gives me an error.
!pip install PyOpenGL==3.1.* PyOpenGL-accelerate==3.1.*
!pip install tensorflow gym keras-rl2 gym[atari] keras pyvirtualdisplay
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten, Convolution2D, MaxPooling2D, Activation
from keras_visualizer import visualizer
from tensorflow.keras.optimizers import Adam
env = gym.make('Boxing-v0')
height, width, channels = env.observation_space.shape
actions = env.action_space.n
input_shape = (3, 210, 160, 3) ## input_shape = (batch_size, height, width, channels)
def build_model(height, width, channels, actions):
model = Sequential()
model.add(Convolution2D(32, (8,8), strides=(4,4), activation="relu", input_shape=input_shape, data_format="channels_last"))
model.add(MaxPooling2D(pool_size=(2, 2), data_format="channels_last"))
model.add(Convolution2D(64, (4,4), strides=(1,1), activation="relu"))
model.add(MaxPooling2D(pool_size=(2, 2), data_format="channels_last"))
model.add(Convolution2D(64, (3,3), activation="relu"))
model.add(Flatten())
model.add(Dense(512, activation="relu"))
model.add(Dense(256, activation="relu"))
model.add(Dense(actions, activation="linear"))
return model
model = build_model(height, width, channels, actions)
It gives below error:
ValueError: Input 0 of layer "max_pooling2d_12" is incompatible with the layer: expected ndim=4, found ndim=5. Full shape received: (None, 3, 51, 39, 32)
Second Problem
My input_shape is (3, 210, 160, 3). I am using the first 3 on purpose due to I have to specify the batch_size before. If I do not specify it before and pass it as (210, 160, 3) to the build_model function, below build_agent function gives me an another error:
def build_agent(model, actions):
policy = LinearAnnealedPolicy(EpsGreedyQPolicy(), attr="eps", value_max=1., value_min=.1, value_test=.2, nb_steps=10000)
memory = SequentialMemory(limit=1000, window_length=3)
dqn = DQNAgent(model=model, memory=memory, policy=policy,
enable_dueling_network=True, dueling_type="avg",
nb_actions=actions, nb_steps_warmup=1000)
return dqn
dqn = build_agent(model, actions)
dqn.compile(Adam(learning_rate=1e-4))
dqn.fit(env, nb_steps=10000, visualize=False, verbose=1)
ValueError: Error when checking input: expected conv2d_11_input to have 4 dimensions, but got array with shape (1, 3, 210, 160, 3)
Deleting batch size number in the model construction phase, removes the MaxPooling2D incompatibility error but throws DQNAgent dimensionality error. Adding the batch size to the model construction phase removes DQNAgent dimensionality error but throws the MaxPooling2D incompatibility error.
I am really stucked.
Issue is with input_shape. input_shape=input_shape[1:]
Working sample code
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten, Convolution2D, MaxPooling2D, Activation
from tensorflow.keras.optimizers import Adam
input_shape = (3, 210, 160, 3)
model = Sequential()
model.add(Convolution2D(32, (8,8), strides=(4,4), activation="relu", input_shape=input_shape[1:], data_format="channels_last"))
model.add(MaxPooling2D(pool_size=(2,2), data_format="channels_last"))
model.add(Convolution2D(64, (4,4), strides=(1,1), activation="relu"))
model.add(MaxPooling2D(pool_size=(2, 2), data_format="channels_last"))
model.add(Convolution2D(64, (3,3), activation="relu"))
model.add(Flatten())
model.add(Dense(512, activation="relu"))
model.add(Dense(256, activation="relu"))
model.add(Dense(2, activation="linear"))
model.summary()
Output
Model: "sequential_7"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_9 (Conv2D) (None, 51, 39, 32) 6176
max_pooling2d_5 (MaxPooling (None, 25, 19, 32) 0
2D)
conv2d_10 (Conv2D) (None, 22, 16, 64) 32832
max_pooling2d_6 (MaxPooling (None, 11, 8, 64) 0
2D)
conv2d_11 (Conv2D) (None, 9, 6, 64) 36928
flatten_1 (Flatten) (None, 3456) 0
dense_4 (Dense) (None, 512) 1769984
dense_5 (Dense) (None, 256) 131328
dense_6 (Dense) (None, 2) 514
=================================================================
Total params: 1,977,762
Trainable params: 1,977,762
Non-trainable params: 0