3
$\begingroup$

My task is to solve an optimization problem with deep reinforcement learning. I read about several algorithms like DQN, PPO, DDPG, and A2C/A3C but use cases always seem to be problems like video games (sparse rewards, etc.) or robotics (continuous action spaces, etc.). Since my problem is an optimization issue, I wonder which algorithm is appropriate for my setting:

  • limited number of discrete actions (like 20)
  • high-dimensional states (like 250 values)
  • instant reward after every single action (not only at the end of an episode)
  • a single action can affect the state quite a lot

There's no "goal" like in a video game, an episode ends after a certain number of actions. I'm not quite sure which algorithm is appropriate for my use case.

Improve this question
$\endgroup$
2
  • $\begingroup$ How long is the average episode? And on how many episodes will you train? $\endgroup$
    – BlindKungFuMaster
    Mar 30, 2020 at 13:12
  • $\begingroup$ Getting feedback from the environment is pretty slow so there's a max. number of actions in each episode. The number greatly depends on the hyperparameters so I can't really tell for now. $\endgroup$
    – annow
    Mar 30, 2020 at 14:48

1 Answer 1

Reset to default
1
$\begingroup$

Theoretically video games and robotics problems are also about optimization(getting maximum reward). So, just like other reinforcement learning problems, I would expect PPO to be the most efficient in your case too. I don't think a "goal" is necessary for rl, all you need is the rewards.

Improve this answer
$\endgroup$
1
  • $\begingroup$ In RL, the reward defines the goal. RL isn't the most appropriate approach to solve many optimization problems, because it can be very expensive to train good agents for real-world problems. Games are actually a very limited scenario. Even games like Go, which are combinatorially difficult to solve, are limited, in the sense that there aren't many external sources of noise, etc. $\endgroup$
    – nbro
    Apr 2, 2020 at 20:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .