2
$\begingroup$

I'm working on my thesis concerning a reinforcement learning problem and am trying to prioritise my time on different components of it:

  • Formalising the agent environment (like the design of state-, action-space and reward-structure)
  • Selection of learning algorithm
  • Selection of network architecture and size
  • Design of the training setup

It is an agent in a 3D environment with simulated physics (in Unity), the domain being a real-time strategical game. It is an environment with constraint training data, so sample efficiency is very important.

Now my question: I do anticipate that the design of the state- and action space will have a big impact on the training result, especially in this environment with little training data.

However, is there a way one can clearly prioritise what components will be the most important ones for an RL setting?

Time is limited, and, for me, as a beginner, it seems to be quite difficult to determine what component will be the most important one and needs the most focus. Testing only the hyper-parameters of a learning algorithm thoroughly will take in itself a long time. And obviously disregarding any component will result in bad results.

Is there a way to know on which component one should focus more?

$\endgroup$
1
$\begingroup$

I don't think there's a strategy that applies to all cases. In some cases, the reward function may need to be carefully designed (e.g. a self-driving car), but, in other cases, the reward function might be quickly designed (e.g. chess) and other parts of the RL system may require more care.

Here you can find tips to approach an RL problem. For example, one tip that I find useful, although probably obvious once you hear of it, is to compare your policy with a random policy. See also these tips.

$\endgroup$
1
  • $\begingroup$ I wanted to infer the amount of time spent for each component by some literature, but this seems not to work out then. Anyway, the links you provided are very useful for designing the task, thank you very much! $\endgroup$
    – kitaird
    Dec 16 '21 at 16:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.