Depth of the neural network is equal to the total number of layers in the neural network except input layer. so, neural network with more number of layers are called deep neural networks.

Width, in general, is a term used to a layer in a neural network. It is equal to the number of neurons in that particular layer. So, it may be apt to use the phrase "width of a layer in a neural network".

But, is it valid to use the phrase "width of a neural network"?

I got this doubt because the phrase "wide neural network" is widely used. The phrase gives the impression that width as a property of a neural network. So, I am thinking whether width of a neural network might have a definition. For example, say, width of a neural network is the number of neurons in the widest layer of that neural network.


This width of a neural network *layer is an agreed upon term. *The width of a neural network is generally the width of the widest layer of the neural network. *I would caution how you use the phrase "width of a neural network" due to interpretability and scale, *and the fact that neural networks often contain layers with varying numbers of neurons, depending on the layer.

From this wikipedia page on "Large width limits of neural networks"

The number of neurons in a layer is called the layer width.

From a nice machine learning resource page

Finally, there are terms used to describe the shape and capability of a neural network; for example:

  • Size: The number of nodes in the model.
  • Width: The number of nodes in a specific layer.

A node [is] also called a neuron or Perceptron

*Edits to add clarity, pointed out by @nbro

  • $\begingroup$ I don't think that this answer really clarifies the doubt. So, is the width used to refer to what? Can we talk about "the width of NN" when NNs can have different layers with a different number of neurons? If yes, in that case, what is the width of the NN? You don't answer these questions directly. $\endgroup$
    – nbro
    Sep 26 at 3:49
  • $\begingroup$ @nbro I believe I have added some clarity, I appreciate you pointing out the areas for improvement $\endgroup$
    – jros
    Sep 27 at 11:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.