Dictionary Observation Spaces with Gymnasium Vector Environments

Question

I'm trying to adapt some cleanrl code to a robot learning environment. Cleanrl expects the observation space to have a "shape" property, but many simulated robot environments use dictionary observations. Is there a simple way to tell pytorch's SyncVectorEnv how to unwrap a dictionary observation into one it can use?

Answer 1

For simple cases, I guess the best way would be to concatenate the observations before SyncVectorEnv, using a custom wrapper. There is a convenient way to do it with Gym:

class DictConcatWrapper(gym.ObservationWrapper):
    def observation(self, obs):
        # (if dict, concatenate its elements here...)
        return obs

and later:

env = ...
env = DictConcatWrapper(env)
env = SyncVectorEnv(env)