Do we have to use CNN for Deep Q Learning?

Question

I read top articles on Google Search about Deep Q-Learning:

• https://medium.freecodecamp.org/an-introduction-to-deep-q-learning-lets-play-doom-54d02d8017d8
• https://skymind.ai/wiki/deep-reinforcement-learning
• https://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/
• Q-Learning page on wikipedia*

and then I noticed that they all use CNN as approximator. If deep learning has a broader definition than just CNN, can we use the term "Deep Q-Learning" on our model if we don't use CNN? or is there a more appropriate definition for that kind of Q-Learning model? for example, if my model only using deep fully-connected layer.

*_{it doesn't say explicitly Deep RL means CNN on RL, but it uses the DeepMind (that uses CNN) as an example on Deep Q-Learning}

Answer 1

No. DQN and other deep RL methods work well with fully connected layers. Here's an implementation of DQN which doesn't use CNNs: github.com/keon/deep-q-learning/blob/master/dqn.py

DeepMind mostly use CNN because they use image as input state, and that because they tried to evaluate performance of their methods vs humans performance. Humane performance is easy to measure at games with image as input state, and that's why CNN based methods present so promptly in RL now.

Answer 2

The approximator can be any artificial neural network architecture, including deep fully-connected networks.