I read top articles on Google Search about Deep Q-Learning:

and then I noticed that they all use CNN as approximator. If deep learning has a broader definition than just CNN, can we use the term "Deep Q-Learning" on our model if we don't use CNN? or is there a more appropriate definition for that kind of Q-Learning model? for example, if my model only using deep fully-connected layer.

*it doesn't say explicitly Deep RL means CNN on RL, but it uses the DeepMind (that uses CNN) as an example on Deep Q-Learning


No. DQN and other deep RL methods work well with fully connected layers. Here's an implementation of DQN which doesn't use CNNs: github.com/keon/deep-q-learning/blob/master/dqn.py

DeepMind mostly use CNN because they use image as input state, and that because they tried to evaluate performance of their methods vs humans performance. Humane performance is easy to measure at games with image as input state, and that's why CNN based methods present so promptly in RL now.

  • $\begingroup$ I start thinking, is that because of the meaning of "deep" on DeepRL means "End-to-End without feature engineering" so most people use the term along with CNN? $\endgroup$
    – malioboro
    Mar 15 '19 at 3:44
  • $\begingroup$ @malioboro CNN's are essential for using images an inputs to a neural network. If you have image inputs to your network, then you will probably use a CNN. If you don't have images as inputs to your network, you probably won't (not getting into time-series inputs and the like). $\endgroup$
    – Omegastick
    Mar 15 '19 at 4:18

The approximator can be any artificial neural network architecture, including deep fully-connected networks.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.