I have a steady hex-map and turn-based wargame featuring WWII carrier battles I would like to improve the fixed policy for the AI using reinforcement learning and have a bunch of noob questions which I will try to spread them on several posts.

  1. Software — 
Is Python mandatory? 
The goal is to learn through self-play and the program is written in Objective-C with the aim to migrate it to Unity/C#.

  2. Hardware 
— Is it hopeless to achieve anything with a single computer?


Is Python mandatory?

Not at all. There is nothing magic about the Python language that makes it more suitable for learning algorithms. However, there are some very good machine learning and reinforcement learning libraries available for free that use Python. You might search for longer, or have more need to create your own code when using Objective-C.

Is it hopeless to achieve anything with a single computer ?

That depends on the complexity of your game. The big projects using deep reinforcement learning use multi-million pound installations to research and train their game-playing algorithms. For instance, here is the spec used by Open AI learning to play DoTA:

OpenAI Five plays 180 years worth of games against itself every day, learning via self-play. It trains using a scaled-up version of Proximal Policy Optimization running on 256 GPUs and 128,000 CPU cores — a larger-scale version of the system we built to play the much-simpler solo variant of the game last year.

Most of us cannot afford such a setup. If your game is as complex as DoTA, then this is an indication that you probably don't have the hardware to train an agent.

However, complexity works on a huge range of scales in RL. For example, it is possible to train an AlphaZero clone to play Connect 4 on a single machine with a decent GPU in around a day.

You will need to explore simpler games as you learn the algorithms, so that will give you a sense of your challenge. You can also present your bot with sub-problems within your game to learn. Doing this should help you get a better feel for the challenges of writing learning agents for your project.

  • $\begingroup$ Thanks ! I feel that coding it in the language of the game might be simpler for integration but also to understand what I am doing Simplifying the problem seems promising - smaller state space - concentrate only on naval move : main flaw of current IA and also less actions to perform I am wondering also if playing at the beginning against the heuristic IA may not help tp reduce the training time, at least for the first trials $\endgroup$ – Carrier Battles Dec 23 '18 at 10:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.