Suppose I am utilising a neural network to predict the next state, $s'$ based on the current $(s, a)$ pairs.

all my neural network inputs are between 0 and 1 and the loss function for this network is defined as the mean squared error of the Difference between the current state and next state. Because the variables are all between 0 and 1, the MSE difference between the actual difference and predicted difference is smaller than the actual difference.

Suppose the difference in next state and current state for $s \in R^2$ is $[0.4,0.5]$ and the neural network outputs a difference of $[0.2,0.4]$. The mean squared loss is therefore 0.05 $(0.2^2 + 0.1^2) = 0.05$ whereas the neural network does not really predict the next state very well due to a difference of $(0.2, 0.1)$.

Although whichever loss function is used does not matter, It was deceiving to think that despite the loss function outputting low values, it is mainly due to the squared term that keeps the value small.

Is Mean Squared Error loss function still a good loss function to be used here ?

  • $\begingroup$ Mean is usually done over batch size, not elements of the output vector. Besides, nothing is stopping you from having $n$ outputs nodes and having separate loss function for each element, that would be equal to summing the loss of all elements $\endgroup$ – Brale Apr 12 at 15:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.