# Which NN would you choose to estimate a continuous function $f:\mathbb R^2 \rightarrow \mathbb R$?

Suppose we want to estimate a continuous function $$f:\mathbb R^2 \rightarrow \mathbb R$$ based on a sample using a NN (around 1000 examples). This function is not bounded. Which architecture would you choose ? How many layers/neurons ? Which activation functions ? Which loss function ?

Intuitively, I would go with one hidden layer, 2 neurons, $$L^2$$ loss, and maybe the Bent identity for the output and a sigmoid in the hidden layer ?

What are the advantages of doing something "fancier" than that ?

Would you also have chosen to use a NN for this job or would you have considered a regression SVM for example or something else (knowing that precision is the goal)?

It depends on the complexity of your problem. $$\mathbb{R}^2 \rightarrow \mathbb{R}^1$$ looks simple, but I can give you some nonsense complicated examples that need a deep network. So, the complexity of the problem sets the number of layers and neurons. The kind of problem will determine the architecture of your network (if it needs memory or not). In most cases, the mean squared error is ok. However, for the activation function, I would go with the ReLU.