0
$\begingroup$

I have created an ANN in Python (without libs). On beginning, it had been learned in target of solve linear problems like distinguishing between negative and positive numbers, where the layer widths were [1, 2, 1]. I have decided to learn recognizing small digits saved as 20x20 black & white PNG files. Now the array of layer widths is:

[1200, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 10]

I tried other similar ones.... With the above array of layer widths training took 8 hours (NN had seen 600.000 images from 5000 images of learning set) and when I look at results, each output is equal about 10%-15%. Nothing is certain.

This code is a core of my NN

and that is main code:

net = nnmv.NeuralNetwork()
    net.createNew([1200,100,100,100,100,100,100,100,100,100,100,100,100,10],0.15,0.07)
    for step in range(0,1000):
        for i in range(0,500):
            for number in range(0,10):
                print("Currently learning: ",number,'x',i," in step: ",step)
                pixels = list(Image.open("judgment/"+str(number)+'x'+str(i)+".png").convert("RGB").getdata())
                output = list()
                for itr in range(0,number):
                    output.append(0)
                output.append(1)
                for itr in range(0,9-number):
                    output.append(0)
                net.teach(Stuff.reorganisePixelData(pixels),output)
                print("Error: ",net.calculateError(output))
        Saver.Saver().save(net, "digitrecognizer")

There is 1200 inputs because there are 400 pixels and each pixel is saved in RGB model. Stuff.reorganisePixelData:

def reorganisePixelData(pixels):
    output = []
    for i in range(0,len(pixels)):
        output.append(pixels[i][0])
        output.append(pixels[i][1])
        output.append(pixels[i][2])
    return output

What have I to do? Add or remove layers, change some or all of the layer widths? Or something in concept of learning?

My error calculator prints error like 0.30203135930914193, and it changes only a bit.

$\endgroup$
  • $\begingroup$ Welcome to AI.SE! I'm sorry, this site focuses on social/architectural/scientific questions about artificial intelligence, as opposed to implementation/programming issues. For the latter, you might be able to get help on Data Science. For an intro to our site, see the tour. $\endgroup$ – Ben N Jul 27 '18 at 16:21
  • $\begingroup$ Ok, but this question is not off-topic. It is about architectural problem, the code is only addon for present the problem! $\endgroup$ – mvxxx Jul 27 '18 at 18:35
  • $\begingroup$ Ah, I see. Sorry about that, reopened! $\endgroup$ – Ben N Jul 27 '18 at 18:37
  • $\begingroup$ Just wanted to check in to make sure the recent edit to your question is acceptable. (You can roll back the edit if you desire.) $\endgroup$ – DukeZhou Aug 13 '18 at 20:49
1
$\begingroup$

The most important parameter, which can modify the learning in a neural network model is the structure of the layers. That means, how many layers were used and how many neurons each layer has. The best neural network is a minimalist one. It is a good idea to start with only 1 hidden layer which has exactly one neuron. Surprisingly this kind of simple neural network is able to do certain tasks. If the error-rate is not good enough it makes sense to increase the number of neurons slowly.

There is a special neural network architecture available which can do this automatically, called NEAT neuroevolution. The idea is, that the network determines it's structure alone. The problem is, that for practical examples NEAT has no advantage over handcrafted network layout. What most programmers are doing is first to try out some typologies by trial and error, and if they are not able to minimize the error rate, they are using convolution neural networks (CNN) which are optimized for digit recognition. The work of Yann LeCun was dedicated to this special task and his first topology “LeNet” has a very low error rate.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.