I'm quite a newbie when it comes to practically working with Deep Learning techniques, although I studied them quite a lot theoretically in the last months. However, now I'm facing my first practical task.

The setting is the following: There is an air hockey game that exists both physically and as a simulation which could be run in parallel. Currently, the robotic opponent is programmed with a fixed algorithm and should further on be replaced by a trainable AI. We are not operating in a high-dimensional space (such as video frames) as the environment is solely represented as a two-dimensional surface (x and y coordinates of the mallets and the puck).

Following my research, I decided to go for an approach incorporating Model-Based Reinforcement Learning in order to achieve a descent sample efficiency as the simulation won't run much faster than real time. Therefore, I plan to pre-train the network using classic Supervised Learning methods with data generated by the already existing simulation to learn about the environment's model beforehand.

As I'm lacking some practical knowledge and I read about several problems regarding the convergence of deep Q-learning methods, I'm not sure how to proceed with the Reinforcement Learning part. Also, I'm pretty overwhelmed by the publications made in this field throughout the last years. Thus, I would appreciate some practical recommendations, which kind of algorithms (specific policy gradient, Q-learning or hybrid methods) might be promising for my application.

Thank you in advance!


Air hockey is a more complicated version of the pong game. The player is able to move the racket in more directions and this allows a better control over the ball. Model based-learning is equal to store domain knowledge into the Artificial Intelligence. This is done by splitting the domain into subtasks. It make sense to divide the problem into two subproblems: “approaching the ball” and “hit the ball”.

Both problems have to be trained separately by a reinforcement learning algorithm. It's done by analyzing the input-output relations. Features for the racket position, the ball position, the direction and the opponent's position are forming the learning space to hold the policy. Because it's a beginner project, a normal black box neural network for storing the q-table values works great. The neural network gets trained to store the numerical values of the features.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.