I've looked into policy gradient RL the last few months. As I find the topic quite interesting, I've been readings lots of papers about it. My aim is to write my master thesis in Maths about it. I already started out, the preliminary title being "Techniques for variance reduction in policy gradient reinforcement learning". Of course, I can sum up latest results, but a Master thesis' aim should be to create sth. new, or apply sth. to a new setting. Does anybody have an idea for a nice application? It was my idea to write the thesis in ML. My professor is not that much into ML but is happy to advise and evaluate the thesis.

Advice highly appreciated!

  • 1
    $\begingroup$ Could you clarify: You appear want someone to invent or suggest a new (in context RL policy gradients) variance reduction technique so that you can apply it and write it up as your thesis? $\endgroup$ – Neil Slater May 3 '18 at 13:36
  • 3
    $\begingroup$ The chances are that if you have read and understood latest papers on policy gradient methods, that you are far more advanced in RL than most of the people on this site. So really you are hoping that the one or two people here at your level happen to have some unexplored idea that they are happy to advise you on. In addition, the answer you get here would not be useful to anyone else (because the subject would no longer be new/unpublished once you had used it). I think you are much better off finding an academic contact within ML research community and asking about unsolved problems. $\endgroup$ – Neil Slater May 3 '18 at 13:42
  • $\begingroup$ @NeilSlater is this question in line with community guidelines? $\endgroup$ – DuttaA May 3 '18 at 16:50
  • $\begingroup$ @DuttaA: I wouldn't want to speak for the community in AI, as I am relatively new here. However, asking for person-specific one-off advice for career or study would be off topic in nearly all Stack Exchanges IMO. That could be avoided in part by asking a variation of the question - such as how one would approach finding out the interesting questions and nice ideas that the OP wants. $\endgroup$ – Neil Slater May 3 '18 at 17:02
  • 1
    $\begingroup$ If phrased like that though, I am still not sure if it could be answered here, because this is a small site, and people with knowledge of cutting-edge RL at an academic level are rare. I expect the best answer will be to do what the OP is already doing - read papers - plus contact academics in the appropriate field. $\endgroup$ – Neil Slater May 3 '18 at 17:06

You can look into applying on-policy learning in environments generated via GAN architectures. There’s work on “imagination augmented agents”, where environment information from rollouts are passed into a policy learning network.

https://arxiv.org/abs/1802.03006 This work unrolls the model in latent space, but is in the same vein of passing environment information to a policy network via generative model.

https://arxiv.org/abs/1804.00379 Apparently the search space for forward rollouts in latent space are very large, and this work on backtracking in rollouts attempts to manage this complexity.

Sources for info: mailing lists, friends, colleagues

Good luck with your thesis!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.