0
$\begingroup$

How does ChatGPT assign confidence level to its answer? What is the mathematical description of it?

Here is my guess. It produces many answers from auto-regression. It already has a probability assigned to each of its outputs. It can just output this probability or some function of this probability for the chosen output. Is this correct?

$\endgroup$
2
  • $\begingroup$ Where is it written that ChatGPT assigns "confidence levels" to its answers? I've just used ChatGPT again and I don't see that. So, either you're mixing concepts here, or you're asking about "how ChatGPT decides what's the most likely answer". $\endgroup$
    – nbro
    Commented Apr 29 at 11:24
  • $\begingroup$ @nbro: Hmm, you may well be right. Someone was saying it gave confidence level. I thought he might be talking about the paid version... $\endgroup$
    – Hans
    Commented Apr 30 at 3:45

1 Answer 1

1
$\begingroup$

Uncertainty is usually seen under the lens of entropy (aka chaos).

The higher the entropy, the less predictable the outcome. Indeed, it's a paraphrasis of what you are saying... if the entropy is high, the generations will be very different (you are confused). For example, think about asking an LLM to predict a coin flip... it will assign 0.5 probability of both outcomes, and thus if you generate, you will have "very confused" answers.

Instead, for events that are certain, theoretically the distribution will be deterministic, because there is no entropy (you have close to no uncertainty in which state London is, thus it's low entropy)

Given this, the entropy of a distribution is given by $\int p(x) \log p(x)$, and given that an LLM is nothing more than a function that predict distribution, specifically factorized as $p(x) = \prod p(x_i|x_{j<i})$, you can used a factorized entropy to find the overall entropy

In poor words, at every token generation step, you check the softmax distribution the LLM is predicting, you calculate the entropy, and at the end you sum all of the entropies from the generation... the higher it is, the more uncertain is the LLM about the generation

A proxy of entropy, is called perplexity, and it's a very well known metric in the LLM community, here's a nice blogpost about it

$\endgroup$
7
  • $\begingroup$ Are you saying the confidence level is the entropy or perplexity of the answer ChatGPT puts out? If so, do you have any published reference to confirm this claim? $\endgroup$
    – Hans
    Commented Apr 28 at 21:08
  • $\begingroup$ @Hans arxiv is public for everybody... arxiv.org/html/2404.15993v1 "Existing uncertainty estimation approaches for LLMs usually involve designing uncertainty metrics for their outputs. For black-box LLMs, these metrics are computed by examining aspects like the generated outputs’ consistency, similarity, entropy, and other relevant characteristics" $\endgroup$
    – Alberto
    Commented Apr 28 at 21:23
  • $\begingroup$ This paper proposes a measure for uncertainty. What I am asking is how ChatGPT constructs its confidence level for each answer. Is their method the same or close to what ChatGPT uses? $\endgroup$
    – Hans
    Commented Apr 28 at 21:38
  • $\begingroup$ @Hans chatGPT is a proprietary system, they do not disclose such information, I'm answering to the _ What is the mathematical description of it_ $\endgroup$
    – Alberto
    Commented Apr 29 at 0:46
  • $\begingroup$ @Hans not that there are not many other formalization of uncertainty without having a bayesian neural network, which chatgpt is not made of definitely $\endgroup$
    – Alberto
    Commented Apr 29 at 0:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .