I've made a connect 4 game in javascript, and I want to design an AI for it. I made a post the other day about what output would be needed, and I think I could use images of the board and a CNN. I did some research into Reinforcement learning, and I think that's what I need to do. I don't have much experience with ML in general, much less RL with Q-learning, but that is what I'd like to do.

Now, I don't really know how to start out with such a big project. I have a few questions first:

  1. What do I do with my input? I'm thinking I give the AI 7 options for moves to make, one for each row of the board. How do I implement a way that it can "look" at the board? Can I just import an image of the current board state?

  2. How do I make a reward table? How should I do the points system for the Q-learning? I'm thinking something like: If it drops a chip it gets a point, if it lines up 2 chips in a row it gets 5 points, if it gets 3 in a row it gets 30, and if it gets 4 in a row it gets 100. Would that be an effective way to do this? How do I implement this?

  3. Is there a library I can use to do any of the work where I make an algorithm and board states and reward tables? Or do I have to hard code any of it?

  4. I've done some research, are there any links or tutorials you think I should read or follow along with? Any other general advice or help for me?

I greatly appreciate anyone who answers one or all of these questions! Thank you so much!

  • $\begingroup$ You don't need reinforcement learning for this, or any kind of ML, but it would work and is a fun thing to study and apply in this kind of game environment. Could you clarify whether your question is about AI routines for connect 4 (and you don't really care whether it is ML or RL), or is it about using connect 4 as a case study for Reinforcement Learning? Or do you just want a connect 4 bot (I expect they already exist)? $\endgroup$ – Neil Slater Dec 4 '19 at 10:22
  • $\begingroup$ @NeilSlater I want to use ML or RL as a case study. I'm really interested in making my own AIs or bots, and learning how to write neural networks and things like that. I don't want just the solution, I want to design it and make it the hard way. I just don't know how to start. $\endgroup$ – joelanbanks3 Dec 4 '19 at 12:18
  • $\begingroup$ You're asking too many questions here. Please, ask only one question per post. $\endgroup$ – nbro Dec 4 '19 at 14:39

1) your input should be so that you describe your entire environment. This could be done by 8 (length)* 8 (height)* 3 (either empty space, opponent chip or your chip) = 192 input neurons. you can just import a image of the current boardstate (which is width pixels * height pixels input neurons), but this means you task the neural network with learning to read the image as wel as playing the game.

2) rewards for a game like this are (most of the time) 1 for a winning move and 0 for a losing move. if you do it your way it might have side effects like prioritizing lining 2 chips up instead of going for a winning move.

3) Tensorflow is most commenly used. (this is python and not javascript though)

4) I liked the book Reinforcement learning, an introduction and would recommand this to anyone.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.