I've been reading a lot about TD-Gammon recently as I'm exploring options for AI in a video game I'm making. The video game is a turn-based positional sort of game, i.e. a "units", or game piece's, position will greatly impact its usefulness in that board state.
To work my way towards this, I thought it prudent to implement a Neural Network for a few different games first.
The idea I like is encoding the board state for the Neural Network with a single output neuron, which gives that board state relative strength compared to other board states. As I understand, this is how TD-Gammon worked.
However, when I look at other people's code and examples/tutorials, there seems to be a lot of variance in the way they represent the board state. Even for something as simple as tic-tac-toe.
So, specifically, for tic-tac-toe, which is a better, or what is the correct representation for the board state?
I have seen:
9 input neurons, one for each square. A
0
indicates a free-space,-1
the opponent, and1
yourself.9 input neurons, but using different values, such as
0
for the opponent,0.5
for free, and1
for yourself?Could you use larger values? Like
0
,1
and2
?27 input neurons. The first 3 being square 1, the next 3 being square 2, etc. Every neuron is
1
or0
. The first of the set of three indicates whether this square is free or not; the second indicates whether the square is occupied by your opponent or not. In the end, only one in every 3 neurons will have a1
, the other two will have a0
.18 input neurons. The first being
1
for the X player, the second being1
for the O player, and both being0
for a blank
Then, when branching into games where the specific pieces' abilities come into play, like in chess, how would you represent this?
Would it be as simple as using higher input values for more valuable pieces? I.e. -20
for an opponents Queen and +20
for your own queen? Or would you need something more complex where you define 10+ values for each square, one for each unit-type and player combination?