I implemented the Actor-Critic with n-step TD prediction to learn to play the 2048 game
For the environment, I don't use this 2048 implementation. I use a simple one without any graphical interface, just pure matrices. The input for the neural network is the log2 of the game board.
The structure of my neural network is:
- Input layer
- Hidden layer with 16 units
- Softmax layer with 4 units (up, down, left, right) for the actor
- Linear regression for the critic
The hidden layer is shared between the actor and critic, so the same hidden layer (point 2) is connected to both the softmax layer of the actor and the linear regression layer of the critic.
The reward in the original game is the value of the merged cells. For example, if two 4s merged, then the reward is 8. My reward function is almost the same, except I take the log2 of it.
I tried these parameters and I also tweaked the learning rate, the $\gamma$, but I couldn't achieve any good result.
Could you recommend what should I change?