2
$\begingroup$

I've had a big interest in machine learning for a while, and I've followed along a few tutorials, but have never made my own project. After losing many games of connect 4 with my friends, I decided to try to make a replica of that, then create a neural network and AI to play against me (Or at least something where I can enter the current board scenario, and it will output which row is the best move). This may be an ambitious first project, but I'm willing to put in the work and research to create something I'm proud of. I created the game using p5.js, and though it may be simple, I'm really happy with how it turned out, as it's one of my first more interesting and unique projects in computer science. Now I don't know a ton about ML, so bear with me. I would like the use pytorch, but I'm open to tensorflow/keras as well.

Here are a few of my questions:

  1. What output do I need to train? My game currently doesn't have a win condition. Would an array or matrix filled with a 0 where there isn't a chip, a 1 where a red chip is, and a 2 for a yellow? Ie
[0,0,0,0
 1,0,0,0
 1,0,0,0
 1,0,2,0
 1,2,2,2]

and enter a 1 somewhere to signify this as a win for player 1? Could an AI recognize this 4 in a row pattern as what needs to be done to win?

  1. What is the best way to simulate a lot of games to get my training data? I'm imagining using an RNG to drop chips randomly, export the data output to a file and then enter whether it was a win for p1, p2, or a tie?

  2. Any other general words of wisdom or links to read?

Thanks so much for reading this and any help you can offer?

$\endgroup$
  • 1
    $\begingroup$ On your first question, numerically for an AI,the difference between a 1 and a 2 could be difficult for it to decipher. Instead, I would suggest using a CNN and adding 2 different channels to the input, one for player 1 and another for player 2, then set the values in these channels to be a 1 for a chip or a 0 otherwise. This will form a sort of image of the board where, if overlayed, you might get like a blue pixel for player 1 and a red pixel for player 2, and a CNN could perform quite well on that. $\endgroup$ – Recessive Dec 2 '19 at 2:53
  • $\begingroup$ @Recessive I will definitely look into that. Out of curiosity, why is it better to do this as an image and then turn it into data rather than just feed it data like a matrix? Would I input whole games or one move at a time? Are you saying to do one image for player 1 chips and 1 image for player 2 chips, overlay them and keep the others empty? Sorry I'm new at this but I appreciate the help! $\endgroup$ – joelanbanks3 Dec 2 '19 at 3:21
  • 1
    $\begingroup$ While CNN's are designed for images, they work exceptionally well on any 2 dimensional data, as you can imagine any matrix with 2 dimensions as a really concise image. In this case, the data is still being fed in as a matrix, just into a CNN as CNN's handle 2d data very well. The data isn't changed in any way, it's only being looked at from a different perspective. Also, no, feeding whole games would be redundant, so just the current board state will do for determining a move. As for you last question - sort of. Search up "Channels in CNN's" to get a better understanding of this. $\endgroup$ – Recessive Dec 2 '19 at 3:30
  • $\begingroup$ @Recessive Ok so you're saying use images of the board 1 move at a time? I'm looking at CNNs and yes, they seem like the best thing to use. So if I understand correctly, normally with CNNs you have to assign each image an output, like with the fashion database you assign each one a class, what would the outputs/classes be for each move? $\endgroup$ – joelanbanks3 Dec 2 '19 at 14:57
  • $\begingroup$ Very good question. This is where it gets tough. If you wanted to do it best, you would use a en.wikipedia.org/wiki/Monte_Carlo_tree_search (a sort of modified minimax). However this might prove difficult. For the sake of simplicity, I would recommend just representing all board positions as a big output vector (7x6, so 42 output nodes) with each representing the next move. Using softmax, only one will be chosen. $\endgroup$ – Recessive Dec 3 '19 at 3:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.