Questions tagged [probability-theory]

For questions related to probability theory in the context of artificial intelligence.

Filter by
Sorted by
Tagged with
0 votes
0 answers
23 views

How to guess a card in the game of Cambio with limited information?

I need help with a probability problem in the card game of Cambio. In this game, two players are dealt four cards each from a deck of 52 cards . At the start of the game, the bottom cards of each ...
Ali Qaqan's user avatar
1 vote
1 answer
35 views

What is x, y, p(x), p(y) in generative model domain?

Background Generative modeling Generative modeling aims to model the probability of observing an observation x. $$ p(x) = \frac{p(y\cap x)}{p(y|x)} $$ ...
Prakhar's user avatar
  • 11
0 votes
0 answers
100 views

Resources to understand the math behind diffusion models

I started learning about diffusion models, but I couldn't follow the math proofs (probability distribution terms and other stuff), is there any sources you suggest to understand what they're talking ...
hakim47's user avatar
17 votes
2 answers
2k views

Are softmax outputs of classifiers true probabilities?

BACKGROUND: The softmax function is the most common choice for an activation function for the last dense layer of a multiclass neural network classifier. The outputs of the softmax function have ...
Snehal Patel's user avatar
0 votes
1 answer
61 views

Why isn't the evidence $p(x) = 1$ if it's an observed variable?

Every explanation of variational inference starts with the same basic premise: given an observed variable $x$, and a latent variable $z$, $$ p(z|x)=\frac{p(x,z)}{p(x)} $$ and then proceeds to expand $...
Abrrval's user avatar
2 votes
1 answer
272 views

Why don't we also need to approximate $p(x \mid z)$ in the VAE?

In the VAE, we approximate the probability distribution $p(z \mid x)$, where $z$ is the latent vector and $x$ is our data. The reason is that $p(z \mid x)$ becomes impossible to calculate for ...
Nervous Hero's user avatar
1 vote
1 answer
99 views

Which formula of p(x, y) to use?

The probability distribution $p(x, y)$ can be calculated in two ways : $p(x, y) = p(y \mid x) p(x)$ $p(x, y) = p(x \mid y) p(y)$ But according to the book Deep Generative Modeling (page number 3 ...
Nervous Hero's user avatar
1 vote
1 answer
230 views

Why are today's neural networks not modeled with probability theory?

In the paper The Perceptron: A probabilistic model for information storage and organization in the brain, Rosenblatt used the probability theory to model his perceptron. My professor told me that ...
Collo's user avatar
  • 11
3 votes
1 answer
44 views

What do we mean by "orderly opinions" in this sentence in the context of Bayes theorem?

In this page, it's written (emphasis mine) If probabilities are thought to describe orderly opinions, Bayes theorem describes how the opinions should be updated in the light of new information What ...
MC5321's user avatar
  • 31
1 vote
1 answer
128 views

How would the probability of a document $P(d)$ be computed in the Naive Bayes classifier?

In naive Bayes classification, we estimate the class of a document as follows $$\hat{c} = \arg \max_{c \in C} P(c \mid d) = \arg \max_{c \in C} \dfrac{ P(d \mid c)P(c) }{P(d)} $$ It has been said in ...
hanugm's user avatar
  • 3,581
2 votes
2 answers
329 views

Why is the equation $\mathbb{E} \left[ (Y - \hat{Y})^2 \right] = \left(f(X) - \hat{f}(X) \right)^2 + \operatorname{Var} (\epsilon)$ true?

In the book An Introduction to Statistical Learning, the authors claim (equation 2.3, p. 19, chapter 2) $$\mathbb{E} \left[ (Y - \hat{Y})^2 \right] = \left(f(X) - \hat{f}(X) \right)^2 + \operatorname{...
nbro's user avatar
  • 39.1k
0 votes
1 answer
192 views

Bayesian Perceptron: Why to marginalize over neuron's output instead of it's weights?

I found a very interesting paper on the internet that tries to apply Bayesian inference with a gradient-free online-learning approach: Bayesian Perceptron: Towards fully Bayesian Neural Networks. I ...
f_3464gh's user avatar
4 votes
1 answer
446 views

How is the formula for the Bayes error rate with an integral derived?

My questions concern a particular formulation of the Bayes error rate from Wikipedia, summarized below. For a multiclass classifier, the Bayes error rate may be calculated as follows: $$p = 1 - \sum_{...
EntangledLoops's user avatar
1 vote
1 answer
98 views

Understanding how to calculate $P(x|c_k)$ for the Bernoulli naïve Bayes classifier

I'm looking at the Bernoulli naïve Bayes classifier on Wikipedia and I understand Bayes theorem along with Gaussian naïve Bayes. However, when looking at how $P(x|c_k)$ is calculated, I don't ...
Aguy's user avatar
  • 65
0 votes
0 answers
66 views

How can the V and Q functions take the expectation over a sum where the number of summands is random?

Assume the existence of a Markov Decision Process consisting of: State space $S$ Action space $A$ Transition model $T: S \times A \times S \to [0,1]$ Reward function $R: S \times A \times S \to \...
Leroy Od's user avatar
  • 435
1 vote
3 answers
98 views

Is it possible to compute $P( F \mid S )$ given $P(F \mid S,A)$ and $P(F \mid S, \lnot A)$ in Bayesian network?

I have a bayesian network, which has the following data: $P(S) = 0.07$ $P(A) = 0.01$ $P(F \mid S,A) = 1.0$ $P(F \mid S, \lnot A) = 0.7$ $P(F \mid \lnot S, A) = 0.9$ $P(F \mid \lnot S, \lnot A) =...
Diego C. 's user avatar
8 votes
1 answer
395 views

What are the main benefits of using Bayesian networks?

I have some trouble understanding the benefits of Bayesian networks. Am I correct that the key benefit of the network is that one does not need to use the chain rule of probability in order to ...
Sebastian Dine's user avatar
6 votes
5 answers
1k views

How do compute the table for $p(s',r|s,a)$ (exercise 3.5 in Sutton & Barto's book)?

I am trying to study the book Reinforcement Learning: An Introduction (Sutton & Barto, 2018). In chapter 3.1 the authors state the following exercise Exercise 3.5 Give a table analogous to that ...
MrYouMath's user avatar
  • 255
2 votes
1 answer
151 views

Why is the equation $r(s', a, s') =\sum_{r \in \mathcal{R}} r \frac{p\left(s^{\prime}, r \mid s, a\right)}{p\left(s^{\prime} \mid s, a\right)}$true?

I am referring to eq. 3.6 (page 49) based on Sutton's online book and can be found in an image below. I could not make sense of the final derivation of the equation $r(s, a, s')$. My question is ...
alfa_80's user avatar
  • 123