5
$\begingroup$

For implementing a neural network algorithm that can play air hockey, I had two ideas for input, and I'm trying to figure out which design would be most viable.

The output must be two analog values that dictate the best position on half of the table for the robot to be at a specific point in time, evaluated 60 times per second.

Having consulted with a professor who has experience with implementing parallel algorithms, it was recommended to me to use a convolutional neural network with a single hidden layer, and directly process the image data as the input layer, after processing it to visualize a direct view of the table and somehow emphasize the puck and mallets with pre-processing. I have already started work on this and have successfully implemented object detection for the puck and mallets using OpenCV to get the center coordinates for all 3 entities.

However, having been able to successfully and accurately pre-process these data at 60 times per second, my thought was to feed those (after normalizing the values using the error function) directly on the input layer and possibly implement a deep learning algorithm that employs more than one hidden layer. The problem is that I don't have any experience implementing neural networks, so I'm not even sure what type of layer would be best for this, or how I should seed the weights.

Another reason I want to consider my idea is that given the relatively few inputs and outputs, probably won't need a GPU to execute forward-propagation 60 times per second, whereas with the convolutional network my professor recommended, I know I'll need to implement using CUDA somehow.

Which of the two input methods would be most recommended for this, and if I were to try and implement my idea, what types of layers should I consider? Also any recommendations for existing frameworks to use for either approach would be highly appreciated.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

I recommend you read up on reinforcement learning. Seeing how AirHockey is similar to the old Atari game Pong, here is a write-up (with code) about how to implement a simple neural network, that plays the game; Deep Reinforcement Learning: Pong from Pixels.

As far as the choice of layers go, again, you should read up on deep reinforcement learning, perhaps study some existing models that work, for a similar task and take their architecture as your starting point. The above link is a nice introduction into such thinking.

As a continuation, I would recomend you employ one of the well known, and developed, libraries, such as TensorFlow or pyTorch. They will allow you to run your models on GPU, like you desire.

For a toolkit, that specializes to reinforcement learning, see openAI gym.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ The first link was immensely helpful, and based on the information, I believe I'll go with the convolutional neural network like my professor recommended, using the preprocessed image frames as input. However, I believe based on my updated understanding that my 2 outputs should be the velocity of moving up and to the right, (after subtracting 0.5 from each and normalizing to the highest velocity our robot can support), that way it's action based rather than location based. $\endgroup$
    – Patrick Roberts
    Commented Sep 29, 2017 at 17:07
  • $\begingroup$ I'm glad. Hope your project goes well. $\endgroup$
    – Žiga Sajovic
    Commented Sep 29, 2017 at 17:13
1
$\begingroup$

I also recommend you take a look at the following work by Uber AI Labs who used an interesting approach to computer games:

https://eng.uber.com/deep-neuroevolution/

Improve this answer
$\endgroup$
1
  • $\begingroup$ Nice link. This topic has definitely come up on the forum. $\endgroup$
    – DukeZhou
    Commented Feb 18, 2018 at 21:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .