11 votes
Accepted

How should the neural network deal with unexpected inputs?

This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and identically distributed, i....
user avatar
  • 33.8k
4 votes
Accepted

What is the intuition behind variational inference for Bayesian neural networks?

Your description of what is going on is more or less correct, although I am not completely sure that you have really understood it, given your last question. So, let me enumerate the steps. The ...
user avatar
  • 33.8k
2 votes
Accepted

Is there any research on models that provide uncertainty estimation?

Yes, there is some research on this topic. It's often called Bayesian machine learning or Bayesian deep learning (but I don't think this is a good name because there are models that aren't really ...
user avatar
  • 33.8k
2 votes
Accepted

Bayesian Perceptron: How is it compatible to Bayes Theorem?

Thanks for asking the question. I'm the author of the paper. The key point is that the weights $w$ cannot be updated directly with the new data as $w$ is not directly related with the output $y$ (see ...
user avatar
1 vote

Bayesian Perceptron: Why to marginalize over neuron's output instead of it's weights?

We want a distribution over $w$, don't we? Yes. You want to obtain a distribution over the parameters, which models the uncertainty about the parameters. This distribution over the parameters can ...
user avatar
  • 33.8k
1 vote

What's the likelihood in Bayesian Neural Networks?

The likelihood depends on the task that you are solving, so this is similar to traditional neural networks (in fact, even these neural networks have a probabilistic/Bayesian interpretation!). For ...
user avatar
  • 33.8k
1 vote
Accepted

Why is neural networks being a deterministic mapping not always considered a good thing?

Your intuition is right. The main reason why a deterministic function can be undesirable (or even dangerous, as I will explain below with an example) is that we may not have enough data to learn the ...
user avatar
  • 33.8k
1 vote
Accepted

Why is the E step in expectation maximisation algorithm called so?

In expectation step, firstly we calculate the posterior of latent variable $Z$ and then the $Q(θ | θ^{(t)})$ is defined as the expected value of the log likelihood of $θ$, with respect to the current ...
user avatar
  • 889

Only top scored, non community-wiki answers of a minimum length are eligible