Is it

  1. number of units in a layer
  2. number of layers
  3. overall complexity of the network (both 1 and 2)
  • $\begingroup$ Usually it refers to the number of layers that are sequential (i.e. one after the other). However, it isn't used much like a quantity (e.g. this network has a depth of 8), rather a qualitative characteristic of the network (e.g. this network is deeper than the other). $\endgroup$
    – Djib2011
    Feb 18 '20 at 9:33

Very good question, I have heard it refered to all the 3 ways you described. I am not sure if there is a purely objective answer so I will answer with what I understand by "depth" generally:

  • If speaking of model architecture: the number of layers of the model.
  • If speaking of CNN layers: the number of filters (not to be confused with the size of the kernel).
  • If speaking of RNN layers: the number of time steps (temporal "depth").

On the other hand I have never heard about depth regarding to FC layers. As I said the term "depth" is very subjective depending on the speakers. It should not be this way, and maybe theoretically is not, but I speak from what I heard in my experience.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.