My task is to solve an optimization problem with deep reinforcement learning. I read about several algorithms like DQN, PPO, DDPG, and A2C/A3C but use cases always seem to be problems like video games (sparse rewards, etc.) or robotics (continuous action spaces, etc.). Since my problem is an optimization issue, I wonder which algorithm is appropriate for my setting:

  • limited number of discrete actions (like 20)
  • high-dimensional states (like 250 values)
  • instant reward after every single action (not only at the end of an episode)
  • a single action can affect the state quite a lot

There's no "goal" like in a video game, an episode ends after a certain number of actions. I'm not quite sure which algorithm is appropriate for my use case.

  • $\begingroup$ How long is the average episode? And on how many episodes will you train? $\endgroup$ Mar 30 '20 at 13:12
  • $\begingroup$ Getting feedback from the environment is pretty slow so there's a max. number of actions in each episode. The number greatly depends on the hyperparameters so I can't really tell for now. $\endgroup$
    – annow
    Mar 30 '20 at 14:48

Theoretically video games and robotics problems are also about optimization(getting maximum reward). So, just like other reinforcement learning problems, I would expect PPO to be the most efficient in your case too. I don't think a "goal" is necessary for rl, all you need is the rewards.

  • $\begingroup$ In RL, the reward defines the goal. RL isn't the most appropriate approach to solve many optimization problems, because it can be very expensive to train good agents for real-world problems. Games are actually a very limited scenario. Even games like Go, which are combinatorially difficult to solve, are limited, in the sense that there aren't many external sources of noise, etc. $\endgroup$
    – nbro
    Apr 2 '20 at 20:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.