Skip to main content

Questions tagged [bayesian-neural-networks]

For questions related to Bayesian neural networks, which are artificial neural networks that incorporate uncertainty in their parameters.

Filter by
Sorted by
Tagged with
0 votes
0 answers
28 views

Should we sum or mean reduce the KL loss in Bayes by Backprop?

I'm unsure if you're supposed to use sum or mean reduction of KL loss for Bayes by Backprop. For example, the BayesianTorch library does both: it reduces by mean across each individual tensor (as seen ...
profPlum's user avatar
  • 434
1 vote
2 answers
64 views

KL loss not changing in a Bayesian Neural Network?

I've been trying to train a Bayesian Neural Network and I noticed that the KL loss (which enforces the prior) isn't changing over time. And it occurred to me that while in standard Bayesian inference ...
profPlum's user avatar
  • 434
1 vote
1 answer
126 views

How does using the ELBO in VAEs make the problem tractable?

I'm studying Variational Autoencoders and a lot of the literature says that the posterior is intractable because the marginal distribution p(x) is intractable since the space of z is so large we ...
Kiran Manicka's user avatar
0 votes
1 answer
49 views

What is the difference between an input and observed data in a Bayesian neural network?

I'm new to the Bayesian perspective and would appreciate clarity on this. In a few resources concerning Bayesian deep learning (such as this one), I see this notation: $p(y|x, D) = \int p(y|x, \theta)...
Seo's user avatar
  • 3
2 votes
1 answer
108 views

Why can Variational Autoencoders (VAEs) approximate arbitrary distributions?

I am trying to reason to myself why is it that VAEs can approximate arbitrary probability distributions even though π‘žπœ™(𝑧|π‘₯) and π‘πœƒ(π‘₯|𝑧) are Gaussian. I understand that the parameters are ...
Decaying Tails's user avatar
2 votes
0 answers
144 views

Today's Practicality of Bayesian Neural Networks

Just having heard lately about BNNs (wow, ANNs and CNNs are clear; now there's a B? What's that? Ahh, Bayesian ;-)) and quickly getting their main idea and focus, that is, weights not being pure ...
Mathy's user avatar
  • 153
2 votes
1 answer
408 views

How could Bayesian neural networks be used for transfer learning?

In transfer learning, we use big data from similar tasks to learn the parameters of a neural network, and then fine-tune the neural network on our own task that has little data available for it. Here, ...
samsambakster's user avatar
1 vote
2 answers
259 views

Bayesian Perceptron: How is it compatible to Bayes Theorem?

I found a very interesting paper on the internet that tries to apply Bayesian inference with a gradient-free online-learning approach: [Bayesian Perceptron: Bayesian Perceptron: Towards fully Bayesian ...
f_3464gh's user avatar
0 votes
1 answer
214 views

Bayesian Perceptron: Why to marginalize over neuron's output instead of it's weights?

I found a very interesting paper on the internet that tries to apply Bayesian inference with a gradient-free online-learning approach: Bayesian Perceptron: Towards fully Bayesian Neural Networks. I ...
f_3464gh's user avatar
5 votes
1 answer
720 views

What is the intuition behind variational inference for Bayesian neural networks?

I'm trying to understand the concept of Variational Inference for BNNs. My source is this work. The aim is to minimize the divergence between the approx. distribution and the true posterior $$\text{KL}...
f_3464gh's user avatar
1 vote
1 answer
699 views

What's the likelihood in Bayesian Neural Networks?

I'm trying to understand the concept behind BNN. Their are based on the Bayes Theorem: $$p(w \mid \text{data}) = \frac{p(\text{data} \mid w)*p(w)}{p(\text{data})}$$ which boils down to $$\text{...
Micha Christ's user avatar
3 votes
0 answers
88 views

Why does this formula $\sigma^2 + \frac{1}{T}\sum_{t=1}^Tf^{\hat{W_t}}(x)^Tf^{\hat{W_t}}(x_t)-E(y)^TE(y)$ approximate the variance?

How does: $$\text{Var}(y) \approx \sigma^2 + \frac{1}{T}\sum_{t=1}^Tf^{\hat{W_t}}(x)^Tf^{\hat{W_t}}(x_t)-E(y)^TE(y)$$ approximate variance? I'm currently reading What Uncertainties Do We Need in ...
user8714896's user avatar
2 votes
1 answer
1k views

Why is neural networks being a deterministic mapping not always considered a good thing?

Why is neural networks being a deterministic mapping not always considered a good thing? So I'm excluding models like VAEs since those aren't entirely deterministic. I keep thinking about this and my ...
user8714896's user avatar
1 vote
1 answer
122 views

Why is the E step in expectation maximisation algorithm called so?

The E step on the EM algorithm asks us to set the value of the variational lower bound to be equal to the posterior probability of the latent variable, given the data points and parameters. Clearly we ...
Manish Kausik Hari Baskar's user avatar
4 votes
1 answer
126 views

Is there any research on models that provide uncertainty estimation?

Is there any research on machine learning models that provide uncertainty estimation? If I train a denoising autoencoder on words and put through a noised word, I'd like it to return a certainty that ...
user8714896's user avatar
2 votes
0 answers
73 views

Are bayesian neural networks suited for text (or document) classification?

I've tried to do my research on Bayesian neural networks online, but I find most of them are used for image classification. This is probably due to the nature of Bayesian neural networks, which may be ...
Nicole's user avatar
  • 21
7 votes
1 answer
1k views

How should the neural network deal with unexpected inputs?

I recently wrote an application using a deep learning model designed to classify inputs. There are plenty of examples of this using images of irises, cats, and other objects. If I trained a data ...
Jeff's user avatar
  • 173