In LMS(least mean square) since, we use a quadratic error function, and quadratic functions are generally parabola in (some convex like shape). I wonder whether that is the reason why we use least square error metric? If that is not the case(its not ALWAYS convex or reason WHY we use LMS), what is the reason then? why this metric changes for deep learning/neural networks but works for regression problems?
[EDIT]: Will this always be a convex function or is there any possibility that it will not be convex?