1
$\begingroup$

In the paper The Perceptron: A probabilistic model for information storage and organization in the brain, Rosenblatt used the probability theory to model his perceptron.

My professor told me that today's neural networks are not modeled with probability theory anymore. Why is that?

$\endgroup$
1
  • $\begingroup$ I appreciate this query as an undergrad math major doing a project on probability theory in neural machine translations. $\endgroup$ Nov 22 '21 at 1:25
1
$\begingroup$

Your professor is wrong (or maybe you misunderstood what he wanted to say, or he did not explain correctly what he wanted to say). It may be a good idea to ask your professor for some clarification and tell him about the info given in this answer.

Probability theory is widely used to model problems in machine learning, including in the context of today's neural networks. There are many examples of research papers or books where you will see probabilities and probability distributions. For example, you can take a look at the neural machine translation paper.

More importantly, we often formulate the problem of learning (in the context of neural networks) as a minimization of an objective function, which is equivalent to the maximization of a likelihood of the parameters (given the data). It's also often the case that neural networks produce a probability vector (from the so-called logits, often, by using a softmax function).

There are many other (maybe clearer) examples of the use of probability theory to model problems in machine learning. For example, you can take a look at Bayesian neural networks, variational auto-encoders, or generative adversarial networks. You will see a lot of probabilities and probability distributions in these linked papers.

You may also be interested in this answer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.