The depth of a neural network is equal to the total number of layers in the neural network (excluding the input layer by convention). A neural network with "many layers" is called a deep neural network.
On the other hand, the width is the name of a property of a layer in a neural network: it is equal to the number of neurons in that particular layer. So, it may be apt to use the phrase "the width of a layer in a neural network".
But, is it valid to use the phrase "width of a neural network"?
I got this doubt because the phrase "wide neural network" is widely used. The phrase gives the impression that the width is a property of a neural network. So, I am wondering whether the width of a neural network has a definition. For example, say, the width of a neural network is the number of neurons in the widest layer of that neural network.