# Does the input layer have bias and are there bias neurons?

I have seen two different representations of neural networks when it comes to bias. Consider a "simple" neural network, with just an input layer, a hidden layer and an output layer. To compute the value of a neuron in the hidden layer, the weights and neurons from the input layer are multiplied, shifted by a bias and then activated by the activation function. To compute the values in the output layer, you may choose not to have a bias and have an identity activation function on this layer, so that this last calculation is just "scaling".

Is it standard to have a "scaling" layer? You could say that there is a bias associated with each neuron, except those in the input layer correct (and those in the output layer when it is a scaling layer)? Although I suppose you could immediately shift any value you're given. Does the input layer have a bias?

I have seen bias represented as an extra unchanging neuron in each layer (except the last) having value 1, so that the weights associated with the connections from this neuron correspond to the biases of the neurons in the next layer. Is this the standard way of viewing bias? Or is there some other way to interpret what bias is that is more closely described by "a number that is added to the weighted sum before activation"?

• I can't give an adequate reason as to why this is - but in all cases I've seen the input layer never has any bias associated to it. You just multiply it by the weights of between input and 1st hidden/output layer. Jul 8 '20 at 4:43
• @mark yeah, that's just "scaling" without bias, bias is for shifting the separation line Apr 5 at 4:59

However, although I have never seen it (or I don't recall having seen it), I would not exclude the existence of an input layer that transforms or augments the inputs before passing them to the next layer. For example, one could implement a neural network that first scales the input to a certain range, and the input layer could do this, although, in practice, this is typically done by some object/class that does not belong to the neural network (e.g. tf.data.Dataset).