Questions tagged [probability-theory]
For questions related to probability theory in the context of artificial intelligence.
19
questions
0
votes
0
answers
23
views
How to guess a card in the game of Cambio with limited information?
I need help with a probability problem in the card game of Cambio. In this game, two players are dealt four cards each from a deck of 52 cards . At the start of the game, the bottom cards of each ...
1
vote
1
answer
35
views
What is x, y, p(x), p(y) in generative model domain?
Background
Generative modeling
Generative modeling aims to model the probability of observing an observation x.
$$
p(x) = \frac{p(y\cap x)}{p(y|x)}
$$
...
- 11
0
votes
0
answers
100
views
Resources to understand the math behind diffusion models
I started learning about diffusion models, but I couldn't follow the math proofs (probability distribution terms and other stuff), is there any sources you suggest to understand what they're talking ...
- 1
17
votes
2
answers
2k
views
Are softmax outputs of classifiers true probabilities?
BACKGROUND: The softmax function is the most common choice for an activation function for the last dense layer of a multiclass neural network classifier. The outputs of the softmax function have ...
- 922
0
votes
1
answer
61
views
Why isn't the evidence $p(x) = 1$ if it's an observed variable?
Every explanation of variational inference starts with the same basic premise: given an observed variable $x$, and a latent variable $z$,
$$ p(z|x)=\frac{p(x,z)}{p(x)} $$
and then proceeds to expand $...
- 1
2
votes
1
answer
272
views
Why don't we also need to approximate $p(x \mid z)$ in the VAE?
In the VAE, we approximate the probability distribution $p(z \mid x)$, where $z$ is the latent vector and $x$ is our data. The reason is that $p(z \mid x)$ becomes impossible to calculate for ...
- 145
1
vote
1
answer
99
views
Which formula of p(x, y) to use?
The probability distribution $p(x, y)$ can be calculated in two ways :
$p(x, y) = p(y \mid x) p(x)$
$p(x, y) = p(x \mid y) p(y)$
But according to the book Deep Generative Modeling (page number 3 ...
- 145
1
vote
1
answer
230
views
Why are today's neural networks not modeled with probability theory?
In the paper The Perceptron: A probabilistic model for information storage and organization in the brain, Rosenblatt used the probability theory to model his perceptron.
My professor told me that ...
- 11
3
votes
1
answer
44
views
What do we mean by "orderly opinions" in this sentence in the context of Bayes theorem?
In this page, it's written (emphasis mine)
If probabilities are thought to describe orderly opinions, Bayes theorem describes how the opinions should be updated in the light of new information
What ...
- 31
1
vote
1
answer
128
views
How would the probability of a document $P(d)$ be computed in the Naive Bayes classifier?
In naive Bayes classification, we estimate the class of a document as follows
$$\hat{c} = \arg \max_{c \in C} P(c \mid d) = \arg \max_{c \in C} \dfrac{ P(d \mid c)P(c) }{P(d)} $$
It has been said in ...
- 3,581
2
votes
2
answers
329
views
Why is the equation $\mathbb{E} \left[ (Y - \hat{Y})^2 \right] = \left(f(X) - \hat{f}(X) \right)^2 + \operatorname{Var} (\epsilon)$ true?
In the book An Introduction to Statistical Learning, the authors claim (equation 2.3, p. 19, chapter 2)
$$\mathbb{E} \left[ (Y - \hat{Y})^2 \right] = \left(f(X) - \hat{f}(X) \right)^2 + \operatorname{...
- 39.1k
0
votes
1
answer
192
views
Bayesian Perceptron: Why to marginalize over neuron's output instead of it's weights?
I found a very interesting paper on the internet that tries to apply Bayesian inference with a gradient-free online-learning approach: Bayesian Perceptron: Towards fully Bayesian Neural Networks.
I ...
- 99
4
votes
1
answer
446
views
How is the formula for the Bayes error rate with an integral derived?
My questions concern a particular formulation of the Bayes error rate from Wikipedia, summarized below.
For a multiclass classifier, the Bayes error rate may be calculated as follows: $$p = 1 - \sum_{...
- 145
1
vote
1
answer
98
views
Understanding how to calculate $P(x|c_k)$ for the Bernoulli naïve Bayes classifier
I'm looking at the Bernoulli naïve Bayes classifier on Wikipedia and I understand Bayes theorem along with Gaussian naïve Bayes. However, when looking at how $P(x|c_k)$ is calculated, I don't ...
- 65
0
votes
0
answers
66
views
How can the V and Q functions take the expectation over a sum where the number of summands is random?
Assume the existence of a Markov Decision Process consisting of:
State space $S$
Action space $A$
Transition model $T: S \times A \times S \to [0,1]$
Reward function $R: S \times A \times S \to \...
- 435
1
vote
3
answers
98
views
Is it possible to compute $P( F \mid S )$ given $P(F \mid S,A)$ and $P(F \mid S, \lnot A)$ in Bayesian network?
I have a bayesian network, which has the following data:
$P(S) = 0.07$
$P(A) = 0.01$
$P(F \mid S,A) = 1.0$
$P(F \mid S, \lnot A) = 0.7$
$P(F \mid \lnot S, A) = 0.9$
$P(F \mid \lnot S, \lnot A) =...
- 11
8
votes
1
answer
395
views
What are the main benefits of using Bayesian networks?
I have some trouble understanding the benefits of Bayesian networks.
Am I correct that the key benefit of the network is that one does not need to use the chain rule of probability in order to ...
- 181
6
votes
5
answers
1k
views
How do compute the table for $p(s',r|s,a)$ (exercise 3.5 in Sutton & Barto's book)?
I am trying to study the book Reinforcement Learning: An Introduction (Sutton & Barto, 2018). In chapter 3.1 the authors state the following exercise
Exercise 3.5 Give a table analogous to that ...
- 255
2
votes
1
answer
151
views
Why is the equation $r(s', a, s') =\sum_{r \in \mathcal{R}} r \frac{p\left(s^{\prime}, r \mid s, a\right)}{p\left(s^{\prime} \mid s, a\right)}$true?
I am referring to eq. 3.6 (page 49) based on Sutton's online book and can be found in an image below.
I could not make sense of the final derivation of the equation $r(s, a, s')$. My question is ...
- 123