What is the meaning of these equations in Noise2Noise paper?

I am trying to understand what is meant by following equations in the Noise2Noise paper by Nvidia.

What is meant by the equation in this image? What is $$\mathbb{E}_y\{y\}$$? And how should I try to visualize these equations?

• Which equations are you referring to? There are at least 3 equations there and other math formulas.
– nbro
Nov 21 '20 at 14:15

The equation you are referring to is called Mean Squared Error (or $$L_2$$ loss) and it is used for regression tasks, where the goal is to predict a real value given some input.

In your case, the inputs are measurements of temperature $$y$$, either at a certain point in time or point in space or both or none, this is not clear from the image. Now, the goal would be to predict the temperature at a new point in space, time, or both, where we don't have access to a measurement. That is we would like to find a function $$f$$ (e.g. a simple linear function) which we can use for prediction. But how can we measure which function is "best"? We introduce a loss function $$L(f,y)$$, another function which tells us how good our proposed function is.

Visually it looks like this (image source):

Red crossed are measurements, the black line is our function we use for prediction and the green dotted lines are the errors (the distance from our prediction to the real measurement). In this example salary depends on experience.

Now, the paper introduces the constant mean of all measurements as $$y$$, $$z = \mathbb{E}_y\{y\} = \frac{1}{N}\sum_i^N y_i$$, as our function $$f$$, which is known to be the minimizer for the $$L_2$$ loss in the case where there is no dependence on other variables (e.g. time or space).

• Note that the expectation is not exactly an average, unless you have access to all possible $y_i$ values. Moreover, you should probably comment on the symbol $\mathbb{E}$.
– nbro
Nov 21 '20 at 14:22
• $\mathbb{E}_y\{y\}$ is for expected value of y? Nov 25 '20 at 10:54
• Yes, correct. $\mathbb{E}_y\{y\}$ is the expected value of the variable y.
– Tinu
Nov 25 '20 at 12:26