What is a working configuration of a neuronal network (number of layers, lerning rate and so on) for a specific dataset?

I try to solve some easy functions with a neuronal network (aforge-lib):

This is how I generate the dataset:

const int GesamtAnzahl = 200;
float[,] tempData = new float[GesamtAnzahl, 2];
float minX = float.MaxValue;
float maxX = float.MinValue;

Random rnd = new Random();
var granzen = new List<int>()
{
rnd.Next(1, GesamtAnzahl-1),
rnd.Next(1, GesamtAnzahl-1),
rnd.Next(1, GesamtAnzahl-1),
rnd.Next(1, GesamtAnzahl-1),
};
granzen.Sort();

for (int i = 0; i < GesamtAnzahl; i++)
{

var x = i;
var y = -1;
if ((i > granzen[0] && i < granzen[1]) ||
(i > granzen[2] && i < granzen[3]))
{
y = 1;
}
tempData[i, 0] = x;
tempData[i, 1] = y;
}


So this is quite easy: The output is 1 if the input is between the 2 lower random generated "borders" or between the 2 higher numbers. Otherwise the output is 1.

The input values are standardices to fit between -1 and 1. So 0 is -1 and 200 is 1.

As a network I used a BackPropagationLearning with a BipolarSigmoidFunction and several configurations like:

Learning Rate: 0,1
Momentum: 0
Sigmoids alpha value: 2
Hidden Layer 1: 4 neurons
Hidden Layer 2: 2 neurons

Learning Rate: 0,1
Momentum: 0
Sigmoids alpha value: 2
Hidden Layer 1: 4 neurons
Hidden Layer 2: 2 neurons
Hidden Layer 3: 2 neurons

Learning Rate: 0,2
Momentum: 0
Sigmoids alpha value: 2
Hidden Layer 1: 4 neurons
Hidden Layer 2: 2 neurons
Hidden Layer 3: 2 neurons


and so on. None of them worked. As described here: https://towardsdatascience.com/beginners-ask-how-many-hidden-layers-neurons-to-use-in-artificial-neural-networks-51466afa0d3e it should be enough to have 2 hidden layers. The first one with 4 neurons and the second one with 2.

The configurations which worked best were:

Learning Rate: 0,01
Momentum: 0
Sigmoids alpha value: 2
Hidden Layer 1: 4 neurons
Hidden Layer 2: 4 neurons
Hidden Layer 3: 4 neurons

Learning Rate: 0,02
Momentum: 0
Sigmoids alpha value: 2
Hidden Layer 1: 4 neurons
Hidden Layer 2: 2 neurons


This solves the problem about 50 % of the times.

As this is a quite simple problem I wonder if I am doing something wrong. I think there has to be a configuration which has better results.

What is the best configuration for this problem and why?

• Having more data does not help. I created 5000 a dataset of 5000 points ( GesamtAnzahl = 5000). Then the networks have a even worse sucess rate.
• I tried to add an extra constant input (always 1) to the dataset but this also lowered the sucess rate