# What is the difference between an input and observed data in a Bayesian neural network?

I'm new to the Bayesian perspective and would appreciate clarity on this.

In a few resources concerning Bayesian deep learning (such as this one), I see this notation:

$$p(y|x, D) = \int p(y|x, \theta)p(\theta|D)d\theta$$

for input input $$x$$, corresponding output $$y$$, network weights and biases $$\theta$$, likelihood function $$p(y|x, \theta)$$, and "posterior distribution over the weights and biases given the observed data $$D$$" $$p(\theta|D)$$.

I would have thought that the observed data would be the input, but apparently $$x$$ and $$D$$ are not the same thing. My question is what is $$D$$ in this context, and how is it different than $$x$$?

You're pretty close:

• $$x,y$$ correspond to a single data point
• $$\mathcal{D}$$ is the whole dataset

Given this, you can read the posterior $$p(y|x,D)$$ as " what's the distribution of $$y$$ given that I observe $$x$$ and my dataset is $$D$$"

At this point, they say that reasoning over a whole dataset is not feasible, so they introduce a new random variable $$\theta$$, from which you can reason over how to solve this

• That makes a lot of sense. Thank you!
– Seo
Sep 19, 2023 at 6:21
• @Seo you’re welcome Sep 19, 2023 at 17:30