Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
Is there any research on machine learning models that provide uncertainty estimation?
If I train a denoising autoencoder on words and put through a noised word, I'd like it to return a certainty that it is correct given the distribution of data it has been trained on.
Answering these questions or metrics for uncertainty are both things I am curious about. Just general ways for models to just say "I'm not sure" when it receives something far outside the inputs it's been trained to approximate.
Yes, there is some research on this topic. It's often called Bayesian machine learning or Bayesian deep learning (but I don't think this is a good name because there are models that aren't really based on a direct application of Bayesian statistics). Some ML/DL models that provide some kind of uncertainty estimation are, for example, Monte Carlo Dropout (MC dropout) or Bayesian neural networks. In theory, these techniques look promising. In practice, I don't think they are the ultimate solution to the problem of uncertainty estimation in deep learning. In fact, e.g. in the case of Bayesian neural networks, they have some disadvantages, such as more parameters to tweak and save.
