Let's say one wants to use a neural net to learn some function $g(x)$. Let's say that we know that $g$ is a combination of two functions (or two sub-problems), $g(x)=f_2(f_1(x))$, and that we have two datasets
- composed of $x$ samples and their corresponding $g(x)$ labels, and
- composed of $x$ samples and their corresponding $f_1(x)$ labels.
Should we use two nets, one to learn the mapping from $x$ samples to $f_1(x)$ using dataset 1 and another net to learn the mapping from $f_1(x)$ to $g(x)$ (note that we can build a dataset composed of $f_1(x)$ samples and $g(x)$ labels with the trained net), or just one net to learn mappings from $x$ to $g(x)$ using dataset 1?
Intuitively, the first option seems to be better since we take advantage of our knowledge that $f_1$ is a "sub-problem" of $g$.