11
votes
Accepted
How should the neural network deal with unexpected inputs?
This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and identically distributed, i....
- 41.4k
9
votes
Accepted
What is the intuition behind variational inference for Bayesian neural networks?
Your description of what is going on is more or less correct, although I am not completely sure that you have really understood it, given your last question.
So, let me enumerate the steps.
The ...
- 41.4k
3
votes
Accepted
Bayesian Perceptron: How is it compatible to Bayes Theorem?
Thanks for asking the question. I'm the author of the paper.
The key point is that the weights $w$ cannot be updated directly with the new data as $w$ is not directly related with the output $y$ (see ...
- 46
2
votes
Accepted
Is there any research on models that provide uncertainty estimation?
Yes, there is some research on this topic. It's often called Bayesian machine learning or Bayesian deep learning (but I don't think this is a good name because there are models that aren't really ...
- 41.4k
2
votes
Accepted
Why can Variational Autoencoders (VAEs) approximate arbitrary distributions?
The fact that you can approximate any distribution is given by the definition ELBO, which is a lower bound in order to learn $p(x)$
Theoretically speaking, if you are able to make that difference ...
- 2,632
2
votes
Accepted
KL loss not changing in a Bayesian Neural Network?
This looks like more underfitting than other issue.
But you are overlooking that Bayes by Backprop is an approximation to Bayesian principles, because a true Bayesian NN is not tractable, you are ...
- 1,355
1
vote
KL loss not changing in a Bayesian Neural Network?
Your observation is correct. In BNNs trained with methods like Bayesian by backprop (BBB), the KL loss serves as a regularizer that encourages the posterior distribution of weights to approximate the ...
- 5,507
1
vote
Accepted
What is the difference between an input and observed data in a Bayesian neural network?
You're pretty close:
$x,y$ correspond to a single data point
$\mathcal{D}$ is the whole dataset
Given this, you can read the posterior $p(y|x,D)$ as " what's the distribution of $y$ given that ...
- 2,632
1
vote
Bayesian Perceptron: Why to marginalize over neuron's output instead of it's weights?
We want a distribution over $w$, don't we?
Yes. You want to obtain a distribution over the parameters, which models the uncertainty about the parameters. This distribution over the parameters can ...
- 41.4k
1
vote
What's the likelihood in Bayesian Neural Networks?
The likelihood depends on the task that you are solving, so this is similar to traditional neural networks (in fact, even these neural networks have a probabilistic/Bayesian interpretation!).
For ...
- 41.4k
1
vote
Accepted
Why is neural networks being a deterministic mapping not always considered a good thing?
Your intuition is right. The main reason why a deterministic function can be undesirable (or even dangerous, as I will explain below with an example) is that we may not have enough data to learn the ...
- 41.4k
1
vote
Accepted
Why is the E step in expectation maximisation algorithm called so?
In expectation step, firstly we calculate the posterior of latent variable $Z$ and then the $Q(θ | θ^{(t)})$ is defined as the expected value of the log likelihood of $θ$, with respect to the current ...
- 927
Only top scored, non community-wiki answers of a minimum length are eligible