What is the meaning of these equations in Noise2Noise paper?

Question

I am trying to understand what is meant by following equations in the Noise2Noise paper by Nvidia.

What is meant by the equation in this image? What is $\mathbb{E}_y\{y\}$? And how should I try to visualize these equations?

Answer 1

The equation you are referring to is called Mean Squared Error (or $L_2$ loss) and it is used for regression tasks, where the goal is to predict a real value given some input.

In your case, the inputs are measurements of temperature $y$, either at a certain point in time or point in space or both or none, this is not clear from the image. Now, the goal would be to predict the temperature at a new point in space, time, or both, where we don't have access to a measurement. That is we would like to find a function $f$ (e.g. a simple linear function) which we can use for prediction. But how can we measure which function is "best"? We introduce a loss function $L(f,y)$, another function which tells us how good our proposed function is.

Visually it looks like this (image source):

Red crossed are measurements, the black line is our function we use for prediction and the green dotted lines are the errors (the distance from our prediction to the real measurement). In this example salary depends on experience.

Now, the paper introduces the constant mean of all measurements as $y$, $z = \mathbb{E}_y\{y\} = \frac{1}{N}\sum_i^N y_i$, as our function $f$, which is known to be the minimizer for the $L_2$ loss in the case where there is no dependence on other variables (e.g. time or space).