All Questions
Tagged with probabilistic-deep-learning or bayesian-deep-learning
27 questions
1
vote
2
answers
59
views
Why the company such as openai has not build probabilistic LLM which tells the user "How sure the model is about its answer"?
I am curious why the LLM model is not built to give a probabilistic answer. Whether people are working on it or is it not necessary to be accomplished.
- 29
0
votes
1
answer
58
views
Predicting Values with Bayesian Neural Network
I want to use a Bayesian Neural Network for a regression task.
To do that I converted a BNN from this paper to Python 3. The provided training script runs and I receive a pickle file, which I want to ...
- 101
0
votes
0
answers
49
views
Should we sum or mean reduce the KL loss in Bayes by Backprop?
I'm unsure if you're supposed to use sum or mean reduction of KL loss for Bayes by Backprop. For example, the BayesianTorch library does both: it reduces by mean across each individual tensor (as seen ...
- 454
1
vote
2
answers
91
views
KL loss not changing in a Bayesian Neural Network?
I've been trying to train a Bayesian Neural Network and I noticed that the KL loss (which enforces the prior) isn't changing over time. And it occurred to me that while in standard Bayesian inference ...
- 454
0
votes
1
answer
35
views
Can Conditional Normalizing Flows Cheat?
Consider the Conditional Normalizing Flow (informally): $f(x, z) = y$ with its inverse $f^{-1}(x, y) = z$ where $x$ is the prior and $z$ is the posterior.
Is $f^{-1}$ able to "cheat" by ...
- 155
0
votes
1
answer
66
views
What is the difference between an input and observed data in a Bayesian neural network?
I'm new to the Bayesian perspective and would appreciate clarity on this.
In a few resources concerning Bayesian deep learning (such as this one), I see this notation:
$p(y|x, D) = \int p(y|x, \theta)...
- 3
1
vote
0
answers
170
views
Samples from a reverse diffusion process with cosine noise schedule blow up
I have implemented a diffusion probabilistic model, and I am finding some of the model behavior unexpected.
When I draw samples from an untrained reverse diffusion process with 20 denoising steps ...
- 11
1
vote
0
answers
16
views
variational inference but with a weighted loglikelihood
I would like to know if it's correct if I substitute in the ELBO formula
a weighted sum of the loglikelihood
$$\sum E_{q_{\theta}(w)}[w_i \ln{p(y_i|f^{w}(x_i))}]$$
in place of the traditional sum.
...
- 111
1
vote
1
answer
525
views
What does AUSE metric mean in uncertainty estimation
I am reading the paper "Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision", I do not understand the definition of AUSE metric in this sentence "but only in ...
- 11
2
votes
0
answers
153
views
Today's Practicality of Bayesian Neural Networks
Just having heard lately about BNNs (wow, ANNs and CNNs are clear; now there's a B? What's that? Ahh, Bayesian ;-)) and quickly getting their main idea and focus, that is, weights not being pure ...
- 153
0
votes
1
answer
188
views
Are some low dimensional distributions known to be hard to model with VAEs?
I am trying to implement a toy VAE project.
My goal is to use a VAE to model the moon dataset from scikit-learn, with an extra constant (but noisy) z-dimension.
To this end I use an approximate ...
- 21
2
votes
0
answers
34
views
What are the practical problems where full bayesian treatment is affordable?
Suppose, I have a problem, where there is rather a small number of training samples, and transfer learning from ImageNet or some huge NLP dataset is not relevant for this task.
Due to the small number ...
- 2,754
2
votes
1
answer
436
views
How could Bayesian neural networks be used for transfer learning?
In transfer learning, we use big data from similar tasks to learn the parameters of a neural network, and then fine-tune the neural network on our own task that has little data available for it. Here, ...
- 171
1
vote
0
answers
38
views
Does MobileNet SSD v2 only capture aleatoric uncertainty (and so not the epistemic one)?
Regarding the MobileNet SSD v2 model, I was wondering to what extend it captures uncertainty of the predictions.
There are 2 types of uncertainty, data uncertainty (aleatoric) and model uncertainty (...
- 11
2
votes
0
answers
92
views
How to add prior information when predicting using deep learning models?
Background
I'm building a binary classification model for a pair match problem using CNN, e.g. whether person A1 likes product B1 or not. Model input features are sequence features (text descriptions) ...
- 21
5
votes
1
answer
814
views
What is the intuition behind variational inference for Bayesian neural networks?
I'm trying to understand the concept of Variational Inference for BNNs. My source is this work. The aim is to minimize the divergence between the approx. distribution and the true posterior
$$\text{KL}...
- 99
1
vote
1
answer
793
views
What's the likelihood in Bayesian Neural Networks?
I'm trying to understand the concept behind BNN.
Their are based on the Bayes Theorem:
$$p(w \mid \text{data}) = \frac{p(\text{data} \mid w)*p(w)}{p(\text{data})}$$
which boils down to
$$\text{...
- 19
3
votes
0
answers
88
views
Why does this formula $\sigma^2 + \frac{1}{T}\sum_{t=1}^Tf^{\hat{W_t}}(x)^Tf^{\hat{W_t}}(x_t)-E(y)^TE(y)$ approximate the variance?
How does:
$$\text{Var}(y) \approx \sigma^2 + \frac{1}{T}\sum_{t=1}^Tf^{\hat{W_t}}(x)^Tf^{\hat{W_t}}(x_t)-E(y)^TE(y)$$
approximate variance?
I'm currently reading What Uncertainties Do We Need in ...
- 805
2
votes
1
answer
1k
views
Why is neural networks being a deterministic mapping not always considered a good thing?
Why is neural networks being a deterministic mapping not always considered a good thing?
So I'm excluding models like VAEs since those aren't entirely deterministic. I keep thinking about this and my ...
- 805
1
vote
2
answers
1k
views
Why is my Keras prediction always close to 100% for one image class?
I am using Keras (on top of TF 2.3) to train an image classifier. In some cases I have more than two classes, but often there are just two classes (either "good" or "bad"). I am ...
- 165
1
vote
0
answers
49
views
Will adding memory to a supervised learning system makes it into a Bayesian learning system?
Seung et.al recently published GameGAN paper, GameGAN learned and stored the whole Pacman game and was able to reproduce it without a game engine. The uniqueness of GameGAN is that it had added memory ...
- 1,096
4
votes
1
answer
128
views
Is there any research on models that provide uncertainty estimation?
Is there any research on machine learning models that provide uncertainty estimation?
If I train a denoising autoencoder on words and put through a noised word, I'd like it to return a certainty that ...
- 805
1
vote
1
answer
309
views
How can a machine learning problem be reduced as a communication problem?
I once heard that the problem of approximating an unknown function can be modeled as a communication problem. How is this possible?
- 187
2
votes
0
answers
74
views
Are bayesian neural networks suited for text (or document) classification?
I've tried to do my research on Bayesian neural networks online, but I find most of them are used for image classification. This is probably due to the nature of Bayesian neural networks, which may be ...
- 21
7
votes
1
answer
2k
views
How should the neural network deal with unexpected inputs?
I recently wrote an application using a deep learning model designed to classify inputs. There are plenty of examples of this using images of irises, cats, and other objects.
If I trained a data ...
- 173
37
votes
6
answers
11k
views
Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?
I trained a simple CNN on the MNIST database of handwritten digits to 99% accuracy. I'm feeding in a bunch of handwritten digits, and non-digits from a document.
I want the CNN to report errors, so I ...
- 1,369
3
votes
2
answers
2k
views
How can supervised learning be viewed as a conditional probability of the labels given the inputs?
In the literature and textbooks, one often sees supervised learning expressed as a conditional probability, e.g.,
$$\rho(\vec{y}|\vec{x},\vec{\theta})$$
where $\vec{\theta}$ denotes a learned set of ...
- 33