Is a linear activation function (in the output layer) equivalent to an identity function?

Question

I have a simple question about the choice of activation function for the output layer in feed-forward neural networks.

I have seen several codes where the choice of the activation function for the output layer is linear.

Now, it might well be that I am wrong about this, but isn't that simply equivalent to a rescaling of the weights connecting the last hidden layer to the output layer? And following this point, aren't you just as well off with just using the identity function as your output activation function?

Answer 1

And following this point, aren't you just as well off with just using the identity function as your output activation function?

When someone declares that the output of a neural network layer is linear this is exactly what they mean. It can also be described as "no activation function".

Saying that a NN layer has linear activation is a kind of short-hand for saying "the whole layer is a linear function of its inputs", which is true without adding any activation function, just using the weights and bias.

There is usually no separate linear function applied, and libraries such as Keras include the term 'linear' only for completeness, or so that the choice can be made explicit in the code, as opposed to an unseen default.

Note that the link to Keras activation definition above says:

Linear (i.e. identity) activation function.