What is missing in OpenAI Gym registration?

Question

I get ValueError: xxx not found in gym registry, you maybe meant when trying to register a custom environment in stable baselines 3. I tried the following commands:

apt-get install swig cmake ffmpeg freeglut3-dev xvfb
git clone --recursive https://github.com/DLR-RM/rl-baselines3-zoo
cd rl-baselines3-zoo
pip3 install -r requirements.txt
cd ..
git clone https://github.com/MatePocs/gym-basic.git
cd gym-basic
pip3 install -e .
cd ..
cd rl-baselines3-zoo
python3 train.py --algo td3 --env basic-v0 --eval-freq 1000 --save-freq 5000

The result is:

Traceback (most recent call last):
  File "train.py", line 107, in <module>
    raise ValueError(f"{env_id} not found in gym registry, you maybe meant {closest_match}?")
ValueError: basic-v0 not found in gym registry, you maybe meant CubeCrash-v0?

Do you see which is the mistake?

Answer 1

This problem can be resolved with --gym-packages gym_basic

python3 train.py --algo td3 --env basic-v0 --eval-freq 1000 --save-freq 5000 --gym-packages gym_basic

But then it shows other problem with hyperparameters.
For standard environments it has predefined tunned hyperparameters in folder hyperparameters. For --algo td3 in hyperparameters/td3.yml.
But there is no parameters for basic-v0

If I duplicate other parameters and use name basic-v0

basic-v0:
  n_timesteps: 300000
  policy: 'MlpPolicy'
  noise_type: 'ornstein-uhlenbeck'
  noise_std: 0.5

then (theoretically) it resolves this problem

... but it gives next problem.

train want to get env.action_space.shape[0] and basic-v0 doesn't have [0] in shape

So all this many need more changes.