I have read some about AI and I know these bubble graphs and how they supposed to work, the different training concepts ect.

But I totally fail to understand, how to put it in code. Most explainations about AI are high level, how the concept works and so on but not how to actually code it.

I would like to start experimenting myself but I don't know where to start. Are the neurons functions? Variables? Loops?

In python they often use tensor flow but I have difficulties to use something when I don't understand how it's constructed under the hood.

What are simple examples to start trying myself for learning purposes?

| improve this question | | | | |
  • 4
    $\begingroup$ There are multiple sources online, I'd suggest googling "neural network from scratch" if you're looking for a better low-level understanding on how they work. $\endgroup$ – Djib2011 Aug 30 '19 at 22:51
  • $\begingroup$ if you really want to understand neural network, you can try to code it from scratch as mentioned by @Djib2011. If you want to implement easily, you can try to learn scikit-learn, there is a package called MLPClassifier which is a simple neural network package. $\endgroup$ – malioboro Aug 31 '19 at 1:22
  • $\begingroup$ @Djib2011 I read a little around and there are quite a few examples... but I still struggle to understand them. Like when you have an NN that gets a random set of [0,1,0,1,1] and it spits out something between 0 and 1... what's the purpose? I don't understand how to actually do something with that. $\endgroup$ – user6329530 Sep 3 '19 at 10:21

A simple way to gain understanding about neural networks is to, first, understand the Inference part.

Forget about complex neural network architectures like CNNs and RNNs. First try to understand how inference happens step by step in a fully connected multi-layer neural network.

Further, to start with, you should try to understand how a single neuron works. Then it's all about connecting many neurons to form a layer and then creating multiple such layers. Then it will become the bubble graph like you mentioned.

Single neuron

Yes, you can think neurons as functions. It first performs the following operation on inputs and weights.

$$ \sum_{i=1}^n w_i x_i $$

Then, the neuron also applies some kind of activation function on the result like tanh, sigmoid, RELU to get y. So, for each bubble in your bubble graph, you perform the following two operations,

  • A sigma operation on weights and inputs

  • Activation on the result of first step.

Don't get confused by the equations shown in literature. Literature mostly uses linear algebra to convert all the above operations into simple matrix multiplications.

If you understand a forward pass, the operation that happens during inference, understanding backward pass that also happens along with forward pass in training is the next level. But, as the name indicates, to simply put, forward pass works from input to output and backward pass works from output to input.

Note: To simplify things, I have avoided using biases.

| improve this answer | | | | |
  • $\begingroup$ Thank you. The problem for me is always that I struggle learning theoretically. I need to experiment and learn while doing. I don't know how to put inputs and weights into code. That's why I need simple, practical examples that I can play around with. $\endgroup$ – user6329530 Aug 31 '19 at 11:18
  • $\begingroup$ "I don't know how to put inputs and weights into code". Then you just need to read a lot of resources, and take weeks of reading efforts. We don't know your mathematical education, and you need some of it to understand machine learning techniques (including NN) $\endgroup$ – Basile Starynkevitch Sep 6 '19 at 5:41
  • $\begingroup$ @user6329530. Like i mentioned in my answer, first try to understand how single neuron works. Then you can expand it to neural networks. Don't get bogged down by the problems AI/ML are solving. If you try to see what it actually doing, its just multiply and accumulate weights and inputs. That's all it does. I suggest you read this article which just says how to emulate logic gates (and/or) with simple neural nets. towardsdatascience.com/… $\endgroup$ – Kartik Podugu Sep 6 '19 at 15:39

I think you should go in a top-down approach, I mean first you need to implement a neural network and see it working and then you can start going into the details(the depth to which you want to go depends on the individual).This approach worked for me.

I suggest you trying out the MNIST hand written digit classification problem which I guess is a hello world problem in Neural networks. This is a 28x28 image of a number and the task of the neural network is to recognize them. To try out this problem you need very basic knowledge in computer vision and some mathematics.

Follow this useful link

Here you have detailed description of the basic Neural network and the libraries that are required. Take some time and start experimenting with the codes step by step and you don't need to know everything about keras or other libraries.Just learn all that is required for this task and keep doing more examples and you will become clear. By the way keras is a high-level API in Deep learning, it uses tensorflow as a backend(you dont need to code in tensorflow which is slightly difficult than keras).

In keras you can build and deploy models very easily.(You will find for yourself)

So once you understand how the simple Densely connected layer works, slowly start referring the theory as well like feed forward, backpropagation, how the weights are updated and so on.

You have a lot of learning resources online like Coursera , Stanford AI Lectures in you tube or their website, DataCamp and a lot of other materials.

But they will be useful if you experiment them yourself. So take a long time and learn the basics through this example and then you can learn about applying CNN architectures to this same problem which will give you better results and then you can proceed to popular and standard architectures.

Hope this helps you and other beginners in deep learning.

| improve this answer | | | | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.