Afaik, investigating meta reinforcement learning algorithms requires a collection of two or more environments which have similar structure but are still different enough. When I read this paper it was unclear to me what the meta-training and meta-testing environments were.

For eg., a graph is given for Ant-Fwd-Bkwd showing its performance with number of gradient steps. I'm guessing these are the meta-testing performances. So, which environment was it 'meta-trained' on?

Was it meta-trained on the same Ant-Fwd-Bkwd environment?


1 Answer 1


According to this paper (PEARL):

These locomotion task families require adaptation across reward functions (walking direction for Half-CheetahFwd-Back, Ant-Fwd-Back, Humanoid-Direc-2D, target velocity for Half-Cheetah-Vel, and goal location for Ant-Goal2D) or across dynamics (random system parameters for Walker-2D-Params).

It looks like different versions of the same environment with differing reward functions are used. For eg., Forward direction might be rewarded positively in one version, Negative in another, Both directions rewarded positively in yet another.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .