I read top articles on Google Search about Deep Q-Learning:

and then I noticed that they all use CNN as approximator. If deep learning has a broader definition than just CNN, can we use the term "Deep Q-Learning" on our model if we don't use CNN? or is there a more appropriate definition for that kind of Q-Learning model? for example, if my model only using deep fully-connected layer.

*it doesn't say explicitly Deep RL means CNN on RL, but it uses the DeepMind (that uses CNN) as an example on Deep Q-Learning


2 Answers 2


No. DQN and other deep RL methods work well with fully connected layers. Here's an implementation of DQN which doesn't use CNNs: github.com/keon/deep-q-learning/blob/master/dqn.py

DeepMind mostly use CNN because they use image as input state, and that because they tried to evaluate performance of their methods vs humans performance. Humane performance is easy to measure at games with image as input state, and that's why CNN based methods present so promptly in RL now.

  • $\begingroup$ I start thinking, is that because of the meaning of "deep" on DeepRL means "End-to-End without feature engineering" so most people use the term along with CNN? $\endgroup$
    – malioboro
    Commented Mar 15, 2019 at 3:44
  • $\begingroup$ @malioboro CNN's are essential for using images an inputs to a neural network. If you have image inputs to your network, then you will probably use a CNN. If you don't have images as inputs to your network, you probably won't (not getting into time-series inputs and the like). $\endgroup$
    – Omegastick
    Commented Mar 15, 2019 at 4:18

The approximator can be any artificial neural network architecture, including deep fully-connected networks.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .