Are Large Non-Vectorized Environments Equivalent to Small Vectorized Environments?

RL newbie, just came across the term vectorized environments.

Based on this SO answer I understand a vectorized environment receives multiple environments per step.

Does a vectorized environment only help with speeding up training (agent trains faster)? Or does it also help with the quality of training (vectorized training leads to better agents)?

Are these training environments equivalent in quality?

Vectorized Environment: 10 environments per step, training with 1,000 steps. That’s 10,000 total environments.

Non-Vectorized Environment: 1 environment per step, but training with 10,000 steps. Also 10,000 total environments.

Would agents trained in both environments be roughly equivalent? Or would the vectorized environment lead to better trained agents?

submitted by /u/mokenyon
[link] [comments]

Leave a Reply

The Future Is A.I. !
To top