When Proximal Policy Optimization (PPO) was released, it was accompanied by a paper describing it.

Later, the authors at OpenAI introduced a second version of PPO, called PPO2 (whereas the original version is now commonly referred to as PPO1). Unfortunately, the several changes made between PPO1 and PPO2 are pretty much undocumented (as stated over here).

Someone associated with OpnenAI's baselines Deep Reinforcement Learning repository commented that the main advancement of PPO2 (compared to PPO1) was the use of a more advanced parallelism strategy, leading to improved performance. Unfortunately, the person omitted naming further changes made.

Now, I was wondering if anyone is aware of a (reliable) source of information or (preferably) even some published literature that lists all the numerous differences between PPO1 and PPO2.



You must log in to answer this question.