I'm trying to implement DQN to play flappy bird. The action space contains two actions: flap and do nothing. The input for the network is four stacked $80 \times 80$ images.
I've been playing with the hyperparameters, but I can't still make the bird go through the first gap. I can't find a bug in the code, but I guess there is at least one. The network does not predict NaN/Inf values.
Could someone help me?
My implementation https://github.com/Neuroosi/Flappybird-DQN/blob/master/DQN.py