35
votes
Accepted
Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?
The concept you are looking for is called epistemic uncertainty, also known as model uncertainty. You want the model to produce meaningful calibrated probabilities that quantify the real confidence of ...
- 1,252
15
votes
Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?
Your classifier is specifically learning the ways in which 0s are different from other digits,
not what it really means for a digit to be a zero.
Philosophically, you could say the model appears to ...
7
votes
Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?
Broken assumptions
Generalization relies on making strong assumptions (no free lunch, etc). If you break your assumptions, then you're not going to have a good time. A key assumption of a standard ...
- 883
4
votes
How does the Dempster-Shafer theory differ from Bayesian reasoning?
Demster-Shafer Theory and Bayesian Networks were both techniques that rose to prominence within AI in the 1970's and 1980's, as AI started to seriously grapple with uncertainty in the world, and move ...
- 9,207
3
votes
Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?
Apollys,
That's a very well thought out response. Particularly, the philosophical discussion of the essence of "0-ness."
I haven't actually performed this experiment, so caveat emptor... I wonder ...
- 131
3
votes
How do language models know what they don't know - and report it?
The data it is trained on includes variants of "I don't know". For instance, if you ask me what is the meaning of life and I reply I don't know, then that is the information schema the AI ...
3
votes
Accepted
How would AI be able to self-examine?
Several AI systems will come up with a level of confidence to the solution found. For example, neural networks can indicate how relatable is the input problem to the ones it was trained with. ...
- 458
2
votes
Accepted
Is there any research on models that provide uncertainty estimation?
Yes, there is some research on this topic. It's often called Bayesian machine learning or Bayesian deep learning (but I don't think this is a good name because there are models that aren't really ...
- 39.6k
2
votes
Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?
I'm an amateur with neural networks, but I will illustrate my understanding of how this problem comes to be.
First, lets see how trivial neural network classifies 2D input into two classes :
But ...
- 121
2
votes
How do language models know what they don't know - and report it?
It makes sense to assume that reinforcement learning from human feedback (RLHF) has some merit, at least. I'll explain myself.
In RL we have a reward (the human feedback), a policy (which should be ...
- 2,443
2
votes
Accepted
Why would the "improvement" be the result of random initialization, and so why should we use multiple runs?
Neural networks use random number generators in multiple places. Most notably for weight initialisation, but also for features such as dropout, selecting minibatches within epochs, and train/cv split ...
- 30.3k
2
votes
Accepted
Why is my Keras prediction always close to 100% for one image class?
Traditional neural networks can be over-confident (i.e. give a probability close to $0$ or $1$) even when they are wrong, so you should not interpret the probability that it produces as a measure of ...
- 39.6k
1
vote
Accepted
How Mutual Information is related to uncertainty
Yes, you can think of mutual information as entropy carried by that information $Y$
Say you have a dice $X$ and you want to predict the outcome, then your belief is a discrete uniform one with $p=1/6$ ...
- 1,358
1
vote
Why is my Keras prediction always close to 100% for one image class?
Without more details about the nature of the dataset, it is impossible to know for sure. However, here are a few likely causes:
You were calling predict on training data, not testing data. The ...
- 2,300
1
vote
How would AI be able to self-examine?
I would concur with the answer given to you by Lovecraft. One of the major problems with A.I. programmers is that they are always trying to push computers to do things which are designed for "mature" ...
- 47
1
vote
How would AI be able to self-examine?
Would AI be able to self-examine objectively and determine if it is capable of doing the task?
Our ability to self-examine comes definitively from the memory of our experiences; indeed, for this ...
- 332
Only top scored, non community-wiki answers of a minimum length are eligible