2
$\begingroup$

I created a virtual 2D environment where an agent aims to find a correct pose corresponding to a target image. I implemented a DQN to solve this task. When the goal is fixed, e.g. the aim is to find the pose for position (1,1), the agent is successful. I would now like to train an agent to find the correct pose while the goal pose changes after every episode. My research pointed me to the term "Multi-Objective Deep Reinforcement Learning". As far as I understood, the aim here is to train one or multiple agents to achieve a policy approximation that fits all goals. Am I on the right track or how should I deal with different goal states?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

The simplest thing you can do is to add data regarding the target pose to the state vector. This will allow any generalisations that the agent learns that apply to similar poses to be used directly.

Clearly in normal use, where the target pose remains fixed during the episode, then that part of the state information will not change either during the episode. You will also need to train with a large variety of target poses - so training will take longer.

Multi-Objective Deep Reinforcement Learning is slightly different in that it attempts to resolve prioritising between multiple sub-goals. It would also be a more complex solution, whilst augmenting the state vector should allow you to continue using a very similar DQN set up as you have already.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your explanation. So the state vector would maybe look like (...,current_pose, target_pose), where target_pose changes after every episode? I already included those information in my current implementation. I think my mistake was that I changed the target_pose after the agent found the previous one during my first tries. Very appreciated your help. $\endgroup$
    – PhilG
    Jun 23, 2020 at 14:51
  • $\begingroup$ Yes. If you can, then you should randomise target_episode according to expected distribution in production for each episode whilst training. If the initial pose can vary then you should randomise that too - again the ideal is to match distribution it would work with in production. $\endgroup$
    – Neil Slater
    Jun 23, 2020 at 16:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .