Why is batch gradient descent performing worse than stochastic and minibatch gradient descent?

I have implemented a neural network from scratch (only using numpy) and I am having problems understanding why the results are so different between stochastic/minibatch gradient descent and batch gradient descent:

The training data is a collection of point coordinates (x,y). The labels are 0s or 1s (below or above the parabola).

As a test, I am doing a classification task. My objective is to make the NN learn which points are above the parabola (yellow) and which points are below the parabola (purple).

Here is the link to the notebook: https://github.com/Pign4/ScratchML/blob/master/Neural%20Network.ipynb

• Why is the batch gradient descent performing so poorly with respect to the other two methods?
• Is it a bug? But how can it be since the code is almost identical to the
• I am using the same (randomly chosen with try and error) hyperparameters for all three
neural networks. Does batch gradient descent need a more accurate
technique to find the correct hyperparameters? If yes, why so?
• Hi. Can you please clarify exactly how we should read these plots? How can you say that batch GD is performing worse than the others? – nbro Sep 27 '19 at 0:01
• @nbro I edited the question. It's performing worse because it shoud label the points above the parabola as 1s (i.e. yellow instead of purple) – Pigna Sep 27 '19 at 11:56