OpenAI Gymnasium, are there any libraries with algorithms supporting it?

Question

OpenAI has released a new library called Gymnasium which is supposed to replace the Gym library. There are many libraries with implamentations of RL algorithms supporting gym environments, however the interfaces changes a bit with Gymnasium. Are there any libbraries with algorithms supporting Gymnasium? I tried CleanRL, KerasRL, RLib and some others and none of them work, the only way for now is to implement them manually, am I correct?

Answer 1

Stable Baselines3 doesn't have a release with Gymnasium support yet, but this pull request explains how to install and use it with gymnasium.

First install the version for that PR:

$ pip install git+https://github.com/carlosluis/stable-baselines3@fix_tests

And assuming you have gymnasium installed already, you can run:

# Important step to override `gym` as `gymnasium`.
import sys
import gymnasium
sys.modules["gym"] = gymnasium

# Sample code which works
from stable_baselines3 import PPO
env = gymnasium.make("CartPole-v1", render_mode="rgb_array")
model = PPO("MlpPolicy", env, verbose=1)