3
$\begingroup$

It is a sentence that I hear a lot and I guess I don't get what it means.

It seems that the weight optimization procedure is very well understood and there is, to some extent, theoretical/empirical foundations that justify the different techniques.

So what exactly about DNNs is not understood?

Any pointing to a reference would already be helpful.

$\endgroup$
2
  • 1
    $\begingroup$ What's definitely not understood: in which part of the matrix multiplications does sentient thought occur? $\endgroup$ Commented Mar 13, 2023 at 22:09
  • $\begingroup$ Usually this is referring to Explainability. $\endgroup$
    – Rexcirus
    Commented Mar 15, 2023 at 16:24

1 Answer 1

3
$\begingroup$

There are broadly two phenomena that are often referred to as not understanding deep learning:

  1. It is not well understood how the network arrives at any particular solution. Trying to analyze the weights into human understandable knowledge is often futile. The model acts as a black box: it seems to do well, but we do not understand the rules it uses to operate. The field of AI that attempts to deal with this is called explainability.

  2. It is not well understood why neural networks, which are heavily over parametrized, do not seem to overfit. The field of study that attempts to tackle this is called Computational Learning Theory.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .