I am a complete beginner in the area. I implemented my first neural network following the online book "Neural Networks and Deep Learning" by Micheal Nielsen. It works fine with classifying handwritten digits. Achieving ~9500/10000 accuracy on the test data. I am trying to train the network to determine whether $x > 5$ where $x$ is in the interval $[0,10)$, which should be a much simpler task than classifying handwritten digits. However, no learning happens and the accuracy ever the test data stays exactly the same with every epoch. I tried different structures and different learning rates but always the same thing happened. Here is the code I wrote that uses libraries in Nielsen's book:
import networkCopy import numpy as np # Creating training data x =  y =  for n in range(1000): to_add = 100*np.random.rand() x.append(np.array([to_add]).reshape(1,1)) y.append(np.array([float(to_add > 50)]).reshape(1,1)) training_data = zip(x, y) # Creating test data tx =  ty =  for n in range(1000): to_add = 100*np.random.rand() tx.append(np.array([to_add]).reshape(1,1)) ty.append(np.array([float(to_add > 50)]).reshape(1,1)) test_data = zip(tx, ty) # Creating and training the network net = networkCopy.Network([1, 5, 1]) # [1, 5, 1] contains the number of neurons for each layer net.SGD(training_data, 300, 100, 5.0, test_data=test_data) # 300 is the number of epochs, 100 is the mini batch size #5.0 is the learning rate
The way I generated the data may not be optimal, it is an ad hoc solution to make the data in the proper form for the network. This is my first question so I apologize for any mistakes that might be in the format of the question.