# Is better to spend parameters on weights or bias?

If a neural network has a limited number of neuron parameters to find, -let's say only 1000 parameters-, it is generally better to spend the parameters on weights or neuron bias?

For example, if each neuron has 2 weights and one bias, it uses 3 parameters per neuron, so only 333 neurons would be available.

But if each neuron uses no bias parameter, then 500 neurons are available with 1000 parameters.

I'm concerned with overfiting by using too many parameters, so I want to minimize the number of parameters meanwhile maximizing the quality of the result.

• Are you just interested in 1. having or not biases or 2. in general, how many connections a neuron has? From your title, it seems like you're trying to understand whether you should or not have biases. However, from the body of your post, it seems like you're also interested in how you should define all the connections (including biases) between neurons (e.g. if a neural network should be sparse or not). – nbro Jun 3 '20 at 16:43
• @nbro I'm asking about the choice between (more neurons) or (less neurons with bias parameters). I guess that more neurons require more computing, and more cycles for convergence, but more neurons can achieve higher accuracy. If the constrain is the number of parameters, then I guess that more neurons should be preferable. – tutizeri Jun 3 '20 at 17:46
• The capacity of a network is proportional to the number of parameters and not necessarily the number of neurons, so I think you're making a possible wrong assumption here. I think your question is more related to the sparsity of the parameters (i.e. how many connections you have and how they affect learning). – nbro Jun 3 '20 at 18:00

Bias isn't just an extra parameter to fit, it is an important adjustable parameter that sets the offset of the separating hyperplane represented by each neuron. Think of the simple equation $$ax+b$$, there's no way to shift the line unless you use the $$b$$ (bias) part.