I've been reading a lot about TD-Gammon recently as I'm exploring options for AI in a video-game I'm making. The video game is a turn-based positional sort of game, i.e. a "units", or game piece's, position will greatly impact it's usefulness in that board state.
To work my way towards this, I thought it prudent to implement a Neural Network for a few different games first.
The idea I like is encoding the board state for the Neural Network with a single output neuron which gives that board states relative strength compared to other board states. As I understand, this is how TD-Gammon worked.
However, when I look at other people's code and examples/tutorials, there seems to be a lot of variance in the way they represent the board-state. Even for something as simple as tic-tac-toe.
So; specifically for tic-tac-toe, which is a better, or what is the correct representation for the board state? I have seen:
- 9 input neurons, one for each square. A
0indicating a free-space,
-1the opponent and
- 9 input neurons, but using different values such as
0for the opponent,
0.5for free and
- Could you use larger values? LIke
- 27 input neurons. The first 3 being square 1, the next 3 being square 2 etc. Every neuron is
0. The first of the set of three indicates whether this square is free or not; the second indicating whether the square is occupied by your opponent or not. In the end, only one in every 3 neurons will have a
1, the other two will have a
- 18 input neurons. The first being
1for the X player, the second being
1for the O player and both being
0for a blank
Then; when branching into games where the specific pieces abilities come into play, like in chess, how would you represent this?
Would it be as simple as using higher input values for more valuable pieces? I.e.
-20 for an opponents Queen and
+20 for your own queen? Or would you need something more complex where you define 10+ values for each square, one for each unit-type and player combination?