Setting up deep reinforcement learning environment on my MAC M1 Air

Question

I recently set up my MAC M1 Air to implement deep reinforcement learning. 
But, when I started following this tutorial - Deep Reinforcement Learning Tutorial for Python https://www.youtube.com/watch?v=cO5g5qLrLSo&list=PLgNJO2hghbmjlE6cuKMws2ejC54BTAaWV&index=2, I got errors with DQN Agent as
 

In this part of the code.
Build Agent with Keras-RL
  from rl.agents import DQNAgent
  from rl.policy import BoltzmannQPolicy
  from rl.memory import SequentialMemory
  def build_agent(model, actions):
      policy = BoltzmannQPolicy()
      memory = SequentialMemory(limit=50000, window_length=1)
      dqn = DQNAgent(model=model, memory=memory, policy=policy, 
              nb_actions=actions, nb_steps_warmup=10, 
      target_model_update=1e-2)
      return dqn
  dqn = build_agent(model, actions)
  dqn.compile(Adam(lr=1e-3), metrics=['mae'])
  dqn.fit(env, nb_steps=50000, visualize=False, verbose=1)

TypeError: Keras symbolic inputs/outputs do not implement __len__. You may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register to dispatch, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model. This error will also get raised if you try asserting a symbolic input/output directly.

I searched a lot for it and they are asking to downgrade TensorFlow for its compatibility with Keras-rl2. But, I don't see any such version available for the MAC silicon chip. Can someone please guide me, I have been stuck on it for far too long.




Any help is appreciated.

Answer 1

The problem you are experiencing is the result of using an older version of keras-rl. That version relies on deprecated methods on the model class.

You need to switch to keras-rl2.