Rollout summary statistics not being monitored for CustomEnv using Stable-Baselines3
I am trying to train a custom environment using PPO via Stable-Baselines3 and OpenAI Gym. For some reason the rollout statistics are not being reported for this custom environment when I try to train the PPO model.
The code that I am using is below ( I have not included the code for the CustomEnv for brevity):
env = CustomEnv(mode = "discrete")
env = Monitor(env, log_dir)
model = PPO("MlpPolicy", env, verbose=1, tensorboard_log = log_dir)
timesteps = 5000
for i in range(3):
model.learn(total_timesteps = timesteps, reset_num_timesteps = False, tb_log_name = "PPO")
model.save(f"{models_dir}/car_model_{timesteps * i}")
Below is an image demonstrating the output from the above code (on the right of the image), and the left side of the image demonstrates the usual output from a dummy environment that I am using for debugging.
I have already tried adding the line of code:
env = Monitor(env, log_dir)
But that doesnt change the output.
SOLVED: There was an edge case where the environment was not ending, and the done variable remained False indefinitely.
After fixing this bug, the Rollout statistics reappeared.
- Reinforcement Learning Gymnasium ValueError
- Too many values in Observation space: Box
- Action masking for continuous action space in reinforcement learning
- torchrl: Using SyncDataCollector with a custom pytorch dqn
- Stable Baselines3 PPO() - how to change clip_range parameter during training?
- Q-learning with a state-action-state reward structure and a Q-matrix with states as rows and actions as columns
- List all environment id in openai gym
- DQN model either doesn't work or it is extremely slow in training
- Deep reinforcement learning - how to deal with boundaries in action space
- QLearning and never-ending episodes
- Good implementations of reinforcement learning?
- Understanding policy and value functions reinforcement learning
- Best practices for exploration/exploitation in Reinforcement Learning
- exploration and exploitation in Q-learning
- iterations and reward in q-learning
- Reinforcement learning toy project
- Reinforcement Learning
- Reinforcement learning And POMDP
- Reinforcement learning with neural networks
- Reinforcement learning algorithms for continuous states, discrete actions
- Supervised learning v.s. offline (batch) reinforcement learning
- How to do backpropagation in PyTorch when training AlphaZero?
- Gymnasium custom environment "too many values to unpack" error
- How to train an artificial neural network to play Diablo 2 using visual input?
- PPO agent is not learning
- Pytorch Geometric graph batching not using DataLoader for Reinforcement learning
- Keras-rl2 error Compability with Tensorflow
- Monte Carlo Method for Blackjack: strange Q-values table
- Performance issue with gradient-bandit agent
- Probability 0 in Importance Sampling