5
$\begingroup$

The depth of a neural network is equal to the total number of layers in the neural network (excluding the input layer by convention). A neural network with "many layers" is called a deep neural network.

On the other hand, the width is the name of a property of a layer in a neural network: it is equal to the number of neurons in that particular layer. So, it may be apt to use the phrase "the width of a layer in a neural network".

But, is it valid to use the phrase "width of a neural network"?

I got this doubt because the phrase "wide neural network" is widely used. The phrase gives the impression that the width is a property of a neural network. So, I am wondering whether the width of a neural network has a definition. For example, say, the width of a neural network is the number of neurons in the widest layer of that neural network.

Improve this question
$\endgroup$
1
  • $\begingroup$ I changed a little bit your post. I think your question was more what I put in the title, i.e. you were wondering if there is a widely accepted definition of the "width of a neural network". Feel free to change your post again. $\endgroup$
    – nbro
    May 22, 2022 at 21:36

1 Answer 1

Reset to default
5
$\begingroup$

The width of a neural network layer is an agreed upon term.

According to Lou et al., in the paper The Expressive Power of Neural Networks: A View from the Width (page 4), the width of a neural network is the width of the widest layer of the neural network.

The architecture of neural networks often specified by the width and the depth of the networks. The depth $h$ of a network is defined as its number of layers (including output layer but excluding input layer); while the width $d_m$ of a network is defined to be the maximal number of nodes in a layer
(emphasis added)

So, I would caution you to be careful with how you use the phrase "the width of a neural network", due to interpretability and scale, and the fact that neural networks often contain layers with varying numbers of neurons, depending on the layer.

From this Wikipedia page on "Large width limits of neural networks":

The number of neurons in a layer is called the layer width.

From a nice machine learning resource page

Finally, there are terms used to describe the shape and capability of a neural network; for example:

  • Size: The number of nodes in the model.
  • Width: The number of nodes in a specific layer.

A node [is] also called a neuron or Perceptron

Improve this answer
$\endgroup$
1
  • $\begingroup$ I edited your answer to, hopefully, improve its clarity. I would remove the part "From this wikipedia page ..." until the end of your answer because the OP already knows that we can talk about "the width of a layer". That's a pretty much standard definition, so there's no need to say that again. He's also aware of what "depth" means. $\endgroup$
    – nbro
    May 24, 2022 at 15:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .