Search code examples
reinforcement-learning

What are vectorized environments in reinforcement learning?

I'm having a hard time wrapping my head around what and when vectorized environments should be used. If you can provide an example of a use case, that would be great.

Documentation of vectorized environments in SB3: https://stable-baselines3.readthedocs.io/en/master/guide/vec_envs.html

Solution

  • Vectorized Environments are a method for stacking multiple independent environments into a single environment. Instead of executing and training an agent on 1 environment per step, it allows to train the agent on multiple environments per step.

    Usually you also want these environment to have different seeds, in order to gain more diverse experience. This is very useful to speed up training.

    I think they are called "vectorized" since each training step the agent observes multiple states (inserted in a vector), outputs multiple actions (one for each environment), which are inserted in a vector, and receives multiple rewards. Hence the "vectorized" term