34 votes
Accepted

Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?

The concept you are looking for is called epistemic uncertainty, also known as model uncertainty. You want the model to produce meaningful calibrated probabilities that quantify the real confidence of ...
user avatar
15 votes

Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?

Your classifier is specifically learning the ways in which 0s are different from other digits, not what it really means for a digit to be a zero. Philosophically, you could say the model appears to ...
user avatar
7 votes

Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?

Broken assumptions Generalization relies on making strong assumptions (no free lunch, etc). If you break your assumptions, then you're not going to have a good time. A key assumption of a standard ...
user avatar
  • 853
3 votes

Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?

Apollys, That's a very well thought out response. Particularly, the philosophical discussion of the essence of "0-ness." I haven't actually performed this experiment, so caveat emptor... I wonder ...
user avatar
3 votes
Accepted

How would AI be able to self-examine?

Several AI systems will come up with a level of confidence to the solution found. For example, neural networks can indicate how relatable is the input problem to the ones it was trained with. ...
user avatar
  • 448
3 votes

How does the Dempster-Shafer theory differ from Bayesian reasoning?

Demster-Shafer Theory and Bayesian Networks were both techniques that rose to prominence within AI in the 1970's and 1980's, as AI started to seriously grapple with uncertainty in the world, and move ...
user avatar
2 votes
Accepted

Is there any research on models that provide uncertainty estimation?

Yes, there is some research on this topic. It's often called Bayesian machine learning or Bayesian deep learning (but I don't think this is a good name because there are models that aren't really ...
user avatar
  • 33.8k
2 votes

Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?

I'm an amateur with neural networks, but I will illustrate my understanding of how this problem comes to be. First, lets see how trivial neural network classifies 2D input into two classes : But ...
user avatar
  • 121
2 votes
Accepted

Why would the "improvement" be the result of random initialization, and so why should we use multiple runs?

Neural networks use random number generators in multiple places. Most notably for weight initialisation, but also for features such as dropout, selecting minibatches within epochs, and train/cv split ...
user avatar
  • 23.8k
2 votes
Accepted

Why is my Keras prediction always close to 100% for one image class?

Traditional neural networks can be over-confident (i.e. give a probability close to $0$ or $1$) even when they are wrong, so you should not interpret the probability that it produces as a measure of ...
user avatar
  • 33.8k
1 vote

Why is my Keras prediction always close to 100% for one image class?

Without more details about the nature of the dataset, it is impossible to know for sure. However, here are a few likely causes: You were calling predict on training data, not testing data. The ...
user avatar
1 vote

How would AI be able to self-examine?

I would concur with the answer given to you by Lovecraft. One of the major problems with A.I. programmers is that they are always trying to push computers to do things which are designed for "mature" ...
user avatar
  • 49
1 vote

How would AI be able to self-examine?

Would AI be able to self-examine objectively and determine if it is capable of doing the task? Our ability to self-examine comes definitively from the memory of our experiences; indeed, for this ...
user avatar
  • 332

Only top scored, non community-wiki answers of a minimum length are eligible