I am a complete beginner in the area. I implemented my first neural network following the online book "Neural Networks and Deep Learning" by Micheal Nielsen. It works fine with classifying handwritten digits. Achieving ~9500/10000 accuracy on the test data. I am trying to train the network to determine whether $x > 5$ where $x$ is in the interval $[0,10)$, which should be a much simpler task than classifying handwritten digits. However, no learning happens and the accuracy ever the test data stays exactly the same with every epoch. I tried different structures and different learning rates but always the same thing happened. Here is the code I wrote that uses libraries in Nielsen's book:

import networkCopy
import numpy as np
# Creating training data
x = []
y = []
for n in range(1000):
    to_add = 100*np.random.rand()
    y.append(np.array([float(to_add > 50)]).reshape(1,1))
training_data = zip(x, y)
# Creating test data
tx = []
ty = []
for n in range(1000):
    to_add = 100*np.random.rand()
    ty.append(np.array([float(to_add > 50)]).reshape(1,1))
test_data = zip(tx, ty)

# Creating and training the network
net = networkCopy.Network([1, 5, 1])  # [1, 5, 1] contains the number of neurons for each layer
net.SGD(training_data, 300, 100, 5.0, test_data=test_data)
# 300 is the number of epochs, 100 is the mini batch size
#5.0 is the learning rate 

The way I generated the data may not be optimal, it is an ad hoc solution to make the data in the proper form for the network. This is my first question so I apologize for any mistakes that might be in the format of the question.

  • $\begingroup$ I don’t understand the point of your task. If you have got it to predict the digit already then why not just add some code that takes the predicted digit and returns prediction > 5? $\endgroup$ – David Ireland Jul 9 at 20:23
  • $\begingroup$ Yes the task is pointless, I am doing it just to build a better understanding of how neural networks behave. This is the reason I chose the task extremely simple. $\endgroup$ – Random_AI_Student Jul 10 at 7:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.