1
$\begingroup$

I've made a connect 4 game in javascript, and I want to design an AI for it. I made a post the other day about what output would be needed, and I think I could use images of the board and a CNN. I did some research into Reinforcement learning, and I think that's what I need to do. I don't have much experience with ML in general, much less RL with Q-learning, but that is what I'd like to do.

Now, I don't really know how to start out with such a big project. I have a few questions first:

  1. What do I do with my input? I'm thinking I give the AI 7 options for moves to make, one for each row of the board. How do I implement a way that it can "look" at the board? Can I just import an image of the current board state?

  2. How do I make a reward table? How should I do the points system for the Q-learning? I'm thinking something like: If it drops a chip it gets a point, if it lines up 2 chips in a row it gets 5 points, if it gets 3 in a row it gets 30, and if it gets 4 in a row it gets 100. Would that be an effective way to do this? How do I implement this?

  3. Is there a library I can use to do any of the work where I make an algorithm and board states and reward tables? Or do I have to hard code any of it?

  4. I've done some research, are there any links or tutorials you think I should read or follow along with? Any other general advice or help for me?

I greatly appreciate anyone who answers one or all of these questions! Thank you so much!

Improve this question
$\endgroup$
3
  • $\begingroup$ You don't need reinforcement learning for this, or any kind of ML, but it would work and is a fun thing to study and apply in this kind of game environment. Could you clarify whether your question is about AI routines for connect 4 (and you don't really care whether it is ML or RL), or is it about using connect 4 as a case study for Reinforcement Learning? Or do you just want a connect 4 bot (I expect they already exist)? $\endgroup$
    – Neil Slater
    Dec 4, 2019 at 10:22
  • $\begingroup$ @NeilSlater I want to use ML or RL as a case study. I'm really interested in making my own AIs or bots, and learning how to write neural networks and things like that. I don't want just the solution, I want to design it and make it the hard way. I just don't know how to start. $\endgroup$
    – joelanbanks3
    Dec 4, 2019 at 12:18
  • $\begingroup$ You're asking too many questions here. Please, ask only one question per post. $\endgroup$
    – nbro
    Dec 4, 2019 at 14:39

1 Answer 1

Reset to default
1
$\begingroup$

1) your input should be so that you describe your entire environment. This could be done by 8 (length)* 8 (height)* 3 (either empty space, opponent chip or your chip) = 192 input neurons. you can just import a image of the current boardstate (which is width pixels * height pixels input neurons), but this means you task the neural network with learning to read the image as wel as playing the game.

2) rewards for a game like this are (most of the time) 1 for a winning move and 0 for a losing move. if you do it your way it might have side effects like prioritizing lining 2 chips up instead of going for a winning move.

3) Tensorflow is most commenly used. (this is python and not javascript though)

4) I liked the book Reinforcement learning, an introduction and would recommand this to anyone.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged .