In a neural network, by how much does the number of neurons typically vary from layer to layer?
Note that I am NOT asking how to find the optimal number of neurons per layer.
As a hardware design engineer with no practical experience programming neural networks, I would like to glean for example
By how much does the number of neurons in hidden layers typically vary from that of the input layer?
What is the maximum deviation in the number of hidden layer neurons to the number of input layer neurons?
How commonly do you see a large spike in the number of neurons?
It likely depends on the application so I would like to hear from as many people as possible. Please tell me about your experience.