I've recently been looking into multi-objective reinforcement learning problems for college, mainly the MO Deep Sea Treasure problem discussed here.

I'm applying a single policy algorithm called w-learning, whereby an agent is assigned to each objective and proposes its own action and the reward it will miss out on if it's not listened to. The agent with the highest 'lost' reward gets their action executed.

I can't seem to grasp how to extract the Pareto front for my implementation, so as to compare it to the actual frontier (the distributed treasure items). Say for a 5 column Deep Sea Treasure environment (and therefore 5 treasure items), to calculate the Pareto frontier after 100 episodes, would I have to train an agent 5 times and only include 1 treasure item each time, then merge my results to get the frontier?

The linked paper above shows a graph of the hypervolume for different algorithms in 4.3, which is something I'm hoping to calculate for my own implementation.

For anyone new to multi-objective evaluation like me, I also found this paper to be quite helpful.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.