I am a little confused on how, you can find online papers that describe complex Machine Learning formulas in a mathematical/probabilistic way, and, in the other hands, easy tutorials that teach you how to use frameworks to create neural networks, without mentioning the maths behind.

What is not clear is, what is the correlation between these two worlds? What are the "parameters" that make you understand i.e. how many layer to code, what kind of perceptrons to use, etc?

to make an example:

Let's take this formula, which in Wikipedia Italy is described as "the standard learning algorytm":

enter image description here

And suppose that size of w and x is 4. , and g(x) and f(x) are, for examples, linear functions. What next? Where do I start coding a neural network that solves this problem?

It would seem more logical to me to code this "directly" without defining perceptrons, convolution, layers etc.

  • 1
    $\begingroup$ Hi @Barsaas. Please include your main question in the title $\endgroup$
    – mugoh
    Jun 5 '21 at 5:11

Basic feed forward neural nets (MLPs) are essentially just computing sequences of matrix multiplications (with nonlinear activations in between), so this is in fact easy to code "directly" like you mentioned. The more difficult part is computing the gradients with respect to the parameter matrices (usually with backpropagation). However there really isn't anything that fancy behind the basic neural network model, it really is just sequences of simple blocks.

You certainly want at least one hidden layer usually, because without it you'll just have a generalized linear model. Neural networks are useful for creating arbitrarily nonlinear models, which can be achieved by adding more layers or more neurons per layer.

If you want some clarification about how to code a neural net, I recommend taking the classic Andrew Ng ML course on Coursera.

With regard to the amount of layers, number of neurons, etc, these are hyperparameters -- there is no known way to determine the correct values for them without experimentation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.