Assume we are given a training dataset $D = \{ (x_i, y_i)\}_{i=1}^{N}$.
My question is: which is better?
- A multivariate regression with basis expansion with independent matrix $X$ and dependent matrix $Y$, such that $X \in K; K \subset \mathbb R^n$ and $Y \in \mathbb R^m$ with training data $D$.
Or
- A neural network which takes $n$ input variables and returns $m$ output with training data $D$
Without a doubt, the multivariate regression option is better with its basis polynomials because it can adapt any curve required in any dimension and doesn't need a large number of datasets than neural networks. Then, why neural networks are used more than multivariate regression?
Note: Prefer explaining the mechanism of neural network used as regression in your answers. To help us know the degree of flexibility of both.
Edit: You may prefer choosing your own loss function in case you need.
