# Is there an AI model with “certainty” built in?

If I see a hundred elephants and fifty of them are grey I'd say the probability of an elephant being grey is 50%. And my certainty of that probability is high.

However, if I see two elephants and one of them is grey. Still the probability is 50%. But my certaintity of this is low.

Are there any AI models where not only the probability is given by the AI but it's certainty is also?

"Certainty" might be thought of as the probability that the probability is correct.

This could go up more levels.

Is there any advantage in doing this?

One way I can envisage this working is instead of a weight, the NN stores two integers $$(P,N)$$ which represent positive and negative evidence and the weight is given by $$P/(P+N)$$. And each iteration $$P$$ or $$N$$ can only be incremented by 1.

• I think you might need to explain this : "Certainty" might be thought of as the probability that the probability is correct. a bit more. This is just not helping the question. – naive Aug 18 '19 at 6:11
• I think you might be looking for confidence intervals – naive Aug 18 '19 at 6:12

A popular methodology to this is Bayesian modeling. In other words given some data, $$\Omega$$, you want to create some form of descriminative model $$p(y|x;\theta)$$ where $$\theta$$ denotes the parameters that are random variables. This is unline NN's where they also learn some descriminative model $$p(y|x;\theta)$$ where $$\theta$$ are fixed. This difference is key to your goal, because if $$\theta$$ is fixed, $$Var(Y|X) = 0$$ since any time you put in the same $$X$$ into the network, itll always come out the same. On the other hand a bayesian model will have a variance, $$Var(Y|X) = E[(Y - E[Y|X])^2 | X]$$ and $$(Y - E[Y|X])^2 | X$$ is no longer gauranteed to be 0 and can be empirically measured through some MC method (or analytically based on your model).
Note that your idea is on the right track, that the more data points you have, the more confident your estimates will be, but the only issue is that just considering $$\frac{P}{(P+N)}$$ in a neural network will be just a heuristic and is difficult to quantify its association to the variance.