I asked ChatGPT 'how certain you are to your answer to my previous question' after I asked it some question, and it replied me the following: 'As an AI developed by OpenAI, I provide information based on a large dataset of text from the internet up until my last training cut-off in September 2021. The definition of a blender as an electrically powered kitchen appliance used for mixing, chopping, or liquefying foods is a standard definition found in many dictionaries and widely accepted. Therefore, I can say with high confidence that this definition is accurate. However, please note that in different contexts, the word "blender" might have different meanings. For example, Blender is also the name of a popular open-source 3D computer graphics software.'
Now, from what I understand, Bayesian people would not accept LLMs have a proper way to quantify uncertainty, since they are just trained to maximize likelihood (i.e., trained to do MLE). According to them a model with true uncertainty quantification should be something like Bayesian neural networks where the distribution of weights is given. But by the same logic, I can argue neither are we humans capable of truly quantifying uncertainty, since we don't have a distribution of how likely a neural is activated in our brain (there is only one instantiation of it).
QUESTION: why do probabilistic/Bayesian machine learning researchers/practiationers claim deep learning models do not have truth uncertainty quantification when even humans do not have the same kind of uncertainty quantification they deem to be genuine, while as humans we clearly have a sense of uncertainty? Is our sense of uncertainty not true then by the same token?