Is there any way to get around this error? I have a model with a 15x15 input grid, which leads to two outputs. Each output has 15 possible values, which are x or y coordinates. I did this because it is significantly simpler than having 225 separate outputs for every location on the grid. The problem is that when i try to train the model using this code:
def build_agent(model,actions)
policy = BoltzmannQPolicy()
memory = SequentialMemory(limit=100000, window_length=1)
dqn = DQNAgent(model=model, memory=memory,policy=policy,nb_actions=actions,nb_steps_warmup=100, target_model_update=1e-2)
return(dqn)
dqn = build_agent(model, np.array([15,15]))
dqn.compile(Adam(learning_rate = 0.01), metrics=['mae'])
dqn.fit(env, nb_steps=10000, action_repetition=1, visualize=False, verbose=1,nb_max_episode_steps=10000)
plt.show()
I get the error: "Model has more than one output. DQN expects a model that has a single output". The model summary is below so you can see there are 2 output layers.
Model: "model_1"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 1, 15, 15)] 0 []
conv2d_2 (Conv2D) (None, 12, 13, 13) 120 ['input_2[0][0]']
conv2d_3 (Conv2D) (None, 10, 11, 3) 354 ['conv2d_2[0][0]']
flatten_1 (Flatten) (None, 330) 0 ['conv2d_3[0][0]']
dropout_1 (Dropout) (None, 330) 0 ['flatten_1[0][0]']
dense_2 (Dense) (None, 15) 4965 ['dropout_1[0][0]']
dense_3 (Dense) (None, 15) 4965 ['dropout_1[0][0]']
==================================================================================================
Total params: 10,404
Trainable params: 10,404
Non-trainable params: 0
__________________________________________________________________________________________________
Standard Keras allows a model with multiple outputs using the functional api but from the errpr message i assume that feature is just not supported for Keras-rl? If thats true, is there any way to get around this issue?
The solution was that i had to just use one output of 225. This didn't work great, but it was the best i could find. Two different outputs will not work using keras-rl, so this was all i could think of. Another possibility would be using a different library such as stable baselines2, but that would be completely different to the already built code.