I have a neural network that is composed of an input layer, two hidden layers and an output layer. The topology is [151, 200, 100, 1] I am using ReLU activation function on the neurons that are in the hidden layers and no activation function on the neuron that is in the output layer. I am wondering if when calculating the delta value of the neuron in the output layer, I should be using a derivative or if I should just subtract the expected output? Here I will put my line of code that this question concerns:

this.deltas[this.NN.length-2][0] = this.NN[this.NN.length-1][0] - expected;
//In forward propagation this neuron has no activation function.

1 Answer 1


Having no activation function means your activation function is the identity, namely $g(z)=z$.

Therefore, any derivative of $g(z)$ wrt to a parameter is simply the corresponding derivative of $z$, with no extra factor.

You would get the same result if you include the derivative, as it is multiplying by $1$.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .