I can reproduce this problem for an even more easily separable dataset:
The ideal tree for it should be as follows:
However, when I run DecisionTreeClassifier with the maximal depth = 2 in scikit-learn many times, it splits the dataset randomly and never gets it right.
This is an example of 4 different runs:
The problem is that scikit-learn has only two ...
Yes, ML can fit a curve based on examples that include hyperparameters but not a model specification. To do this, you need to specify a family of models that is large enough to include the true model. You can then treat this as learning a relationship from 4 inputs to a single output.
For example, suppose you are willing to make only the following ...