# Explanation of this L2 minimization equation

I am trying to understand the last two lines of this math notation. How Var and double summation of Cov came to the equation. The first two lines I understood something like $$(a-b)^2 = a^2 -2ab +b^2$$.

As in the second line, the first two terms are $$\mathbb{E}_{\hat{y}}$$, it means the variable of the expectation is $$\hat{y}_i$$ and you can take out $$y_i$$s from $$\mathbb{E}_{\hat{y}}$$s. Now we can use $$\mathbb{E}\{\hat{y}_i\} = y_i$$ and rewrite the terms of the second line likes the following:

$$\mathbb{E}_{\hat{y}}(\sum_i y_i)^2 = (\sum_i y_i)^2 \\ \mathbb{E}_{\hat{y}}\left[(\sum_i y_i)(\sum_i \hat{y}_i)\right] = (\sum_i y_i) \mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i) = (\sum_i y_i)^2$$

The second line is written based on linearity of the expectation and $$\mathbb{E}\{\hat{y}_i\} = y_i$$. Hence, we can rewrite the second line like the following as $$\sum_i y_i = \mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)$$:

$$\frac{1}{N^2}\left[\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)^2 - \left(\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)\right)^2\right]$$

And the final step is using this formula $$Var(X) = E(X^2) - (E(X))^2$$ and take $$X = \sum_i \hat{y}_i$$:

$$\frac{1}{N^2}\left[\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)^2 - \left(\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)\right)^2\right] = \frac{1}{N^2} Var(\sum_i \hat{y}_i)$$

From variance to covariance, you can use this formula: $$Var(\sum_{i=1}^nX_i) = \sum_{i=1}^n\sum_{j=1}^n cov(X_i, X_j)$$