# What are the differences between loss surfaces that "derive"from different observations?

If I understand right that each observation whithin a dataset, creates a different loss surface where we want to find the global minimum.

How different those surfaces one from another? Would it be correct to say that they differ like (for example) those two parabolas: f(x) = 5x^2 +4x+2 versus f(x) = 5x^2 +1x+8 which can be seen as same parabola located in another place of xy plane.

Thank you

Let's take in consideration linear regression. You have a dataset composed by $$x,y$$ pairs, and you assume they are linearly related, thus you model this problem with LR: $$y = wx+b$$ Now, you want to find the $$w$$ and $$b$$ that best describe your data, thus you set a loss function, say MSE, and you minimize it: $$L(w, b) = \sum_{(x,y)\in D} (y - (wx+b))^2$$
At that point, you try to minimize the average loss (the $$1/n$$ is discarded because it does not effect the minimization), so say you take "the average parabola across the dataset"
• @Igor given a distance metric (loss), your global minima would be by definition $L=0$, which means that the requirement is that the function approximated by your model intersects all the data in your dataset... however, there are infinite functions that can pass from N points, no matter the dimensionality, and the parameters of such functions might be arbitrarily far, thus global optima (s) do not overlap each other Aug 13 at 22:38