0
$\begingroup$

I'm new to the Bayesian perspective and would appreciate clarity on this.

In a few resources concerning Bayesian deep learning (such as this one), I see this notation:

$p(y|x, D) = \int p(y|x, \theta)p(\theta|D)d\theta$

for input input $x$, corresponding output $y$, network weights and biases $\theta$, likelihood function $p(y|x, \theta)$, and "posterior distribution over the weights and biases given the observed data $D$" $p(\theta|D)$.

I would have thought that the observed data would be the input, but apparently $x$ and $D$ are not the same thing. My question is what is $D$ in this context, and how is it different than $x$?

$\endgroup$

1 Answer 1

1
$\begingroup$

You're pretty close:

  • $x,y$ correspond to a single data point
  • $\mathcal{D}$ is the whole dataset

Given this, you can read the posterior $p(y|x,D)$ as " what's the distribution of $y$ given that I observe $x$ and my dataset is $D$"

At this point, they say that reasoning over a whole dataset is not feasible, so they introduce a new random variable $\theta$, from which you can reason over how to solve this

$\endgroup$
2
  • 1
    $\begingroup$ That makes a lot of sense. Thank you! $\endgroup$
    – Seo
    Sep 19, 2023 at 6:21
  • $\begingroup$ @Seo you’re welcome $\endgroup$
    – Alberto
    Sep 19, 2023 at 17:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .