I would like to know a list of model-based and model-free reinforcement learning algorithms, like Q-learning, SARSA, TD, Dyna-Q.

  • 4
    $\begingroup$ Possible duplicate of What's the difference between model-free and model-based reinforcement learning? $\endgroup$ – Dennis Soemers Mar 7 at 12:58
  • $\begingroup$ Are you asking for a list of algorithms that are model-based and model-free, or asking you asking the difference between the two approaches? $\endgroup$ – nbro Mar 7 at 16:12
  • $\begingroup$ @DennisSoemers This could not be a duplicate of the other question, if she is asking about examples of model-free and mode-based algorithms. I think it would be interesting to have such a list (if the question is that one). $\endgroup$ – nbro Mar 7 at 22:51
  • $\begingroup$ @nbro Yeah if the question is just about enumerating the algorithms, I guess it's slightly different. I wasn't 100% sure if that's what the question was though, especially not in its unedited form which is when I raised the flag. On the other hand, I'm not sure if enumerating all the model-based and model-free approaches is really a realistic option either. $\endgroup$ – Dennis Soemers Mar 8 at 10:01
  • $\begingroup$ I want yo know the list of algorithms / methods that are used for Model-based Reinforcment learning and those using for Model-free Reinforcment learning $\endgroup$ – zommita sabrine Mar 8 at 17:10

The surprising fact is, that model-free algorithm like q-learning can be used for model-free and model-based learning as well. The forward model can be stored in the q-matrix and it can be modified by changing the parameters. What we can say in general is, that model-free algorithms are discussed very often, and model-based learning is some kind of non-conformist idea. The combination of reinforcement learning plus model-based control is a promising technology which will allow to solve complex domains. Sometimes, the principle is called sample-efficient because the algorithm doesn't contain trial-by-error but is working with a plan.

It is a bit complicated to separate between learning algorithm for model-based and model-free assumption. What we can say in general is, that the idea is to learn the parameters. Which means it has to do with raw-data stored in a database. And build a model, a policy and a reward function on top of the data.

Q-learning is often described as a model-free technique. But it is only a special case. It can be model-based as well. The forward model is described as a markov decision process. The transition from one state to another of the system are given in the q-table. In case of online system identification, It's called bayesian q-learning.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.