# 3D environment for RL research in Academia

I'm doing my thesis on Reinforcement Learning. My focus on Partially Observable Environments like 3D Games. I want to choose a 3D platform for testing and doing research.

I know some of them. DeepMind Lab and OpenAi Universe. But my question is that which of these environments is good for me? Is there any environment for this purpose that is benchmark and reliable?

I want a platform that accepted in Academia and reliable. For example DeepMind is not a standard or Open Source friendly, Is it rational to use their platform for research in academia?

What i have to do?

Long answer: Courses and tools grouped around Python programming, OpenAI and “everybody can program” motto has a non-professional direction. That means, the target audience are beginners, amateurs, students and people outside of Academia. Python and OpenAI are used to increase the number of people who have a simple understanding of AI programming. It is a mass culture, which is not focused on implementing complex system but simplify the learning process.

If OpenAI isn't used in Academia, what is the alternative for longterm computer scientists who have already an understanding about Reinforcement learning and want to build reliable systems for professional needs? The answer is Forth. The Forth programming language is the most professional setup which is available in Academia. It is used by academic experts to implement realtime applications. Two examples are given in the following paper:

• Zhu, Qiuming, et al. "Synergetic Human-System Integration for Reliable and Efficient C2 Operations."
• Ear, Harold. A Biologically Inspired Four-Legged Walking Robot (Robo-Dog). Diss. Murdoch University, 2017
• Ermm... pretty much everyone in academia (at least when it comes to Reinforcement Learning) is using python. Don't think I've ever heard of Forth... I'm sure it exists, but Python is used many orders of magnitude more often in academia. – Dennis Soemers Nov 1 '18 at 16:37
• It is correct, that Python is used in first semester courses and especially by distance learning universities. But in higher academic spheres and for senior professors, Python is not a great choice, because it needs an underlying operating system, isn't efficient and has too much overhead. In contrast, Forth is much better suited. Quote “Although the basic ideas and principals of the ANDF system are very close to those of Forth [Moo74], it is implemented at a higher level. It is receiving large funding grants for research.” (Knaggs, Peter J: A Look at Forth's Academic Standing) – Manuel Rodriguez Nov 1 '18 at 16:55
• If you're thinking about parts of academia running code on non-standard hardware (like robotics), maybe you're right, I'm not too much involved with that. This question is about the parts of academia involved in more fundamental Reinforcement Learning research in simulations/games/other kinds of toy domains. Those are almost always done using Python. Not just by first semester people, but I'm talking about top tier publications here. See pretty much every publication from DeepMind, OpenAI, well-known universities like McGill, Sutton's group, etc. Almost always all Python. – Dennis Soemers Nov 1 '18 at 17:06

For example DeepMind [Lab] is not a standard or Open Source friendly

I'm not sure where you got that info from... as far as I'm aware, DeepMind Lab is definitely used in various publications (maybe primarily publications from DeepMind, but still). Considering the github repo has the GNU GPL 2 license, it also seems Open Source-friendly to me.

Another framework of which I'm sure that it would widely be considered suitable within academia would be the Unity ML-Agents Toolkit, which uses the Unity game engine.

I suppose you could also consider using ViZDoom, which is also used in various publications, but (as far as I'm aware) it only supports one specific game (Doom).

I do not have enough experience with using any of the above personally to be able to recommend one of them over the others... but they would all seem suitable to me.