Questions tagged [probability]

For question involving probability as related to AI methods. (This tag is for general usage. Feel free to utilize in conjunction with the "math" and more specific probability tags.)

Filter by
Sorted by
Tagged with
0
votes
1answer
48 views

Does generator in conditonal GAN obey probability laws?

In probability, we have two types of probability functions: unconditional probability $p(x)$ and conditional probability $p(x | y)$. Both are fundamentally different and the latter can be obtained by ...
2
votes
1answer
229 views

What to do when PDFs are not Gaussian/Normal in Naive Bayes Classifier

While analyzing the data for a given problem set, I came across a few distributions which are not Gaussian in nature. They are not even uniform or Gamma distributions(so that I can write a function, ...
1
vote
1answer
60 views

Would it be any more difficult for AI/ML to beat more poker players?

Would the number of players at a poker table alter a capable AI/ML's chances of winning? I don't see how it would, so long as every player was playing fairly. I would assume that it would not matter ...
0
votes
1answer
36 views
2
votes
1answer
114 views

Why do I get small probabilities when implementing a multinomial naive Bayes text classification model?

When applying multinomial Naive Bayes text classification, I get very small probabilities (around $10e^{-48}$), so there's no way for me to know which classes are valid predictions and which ones are ...
1
vote
1answer
44 views

How would the probability of a document $P(d)$ be computed in the Naive Bayes classifier?

In naive Bayes classification, we estimate the class of a document as follows $$\hat{c} = \arg \max_{c \in C} P(c \mid d) = \arg \max_{c \in C} \dfrac{ P(d \mid c)P(c) }{P(d)} $$ It has been said in ...
0
votes
0answers
25 views

Which probability distribution a generator in Generative Adversarial Network (GAN) is capturing: dataset or ground truth?

Consider the following statement from the abstract of the paper titled Generative Adversarial Nets We propose a new framework for estimating generative models via an adversarial process, in which we ...
2
votes
1answer
346 views

What exactly is a Parzen?

I came across the term "Parzen" while reading the research paper titled Generative Adversarial Nets. It has been used in the research paper in two contexts. #1: In phrase "Parzen window&...
2
votes
0answers
67 views

How to Prove This Inequality, Related to Generalization Error (Not Using Rademacher Complexity)?

This is an inequality on page 36 of the Foundations of Machine Learning by Mohri, but the author only states it without proof. $$ \mathbb{P}\left[\left|R(h)-\widehat{R}_{S}(h)\right|>\epsilon\right]...
1
vote
0answers
24 views

How can the probability of two disjoint events be non-zero?

Let $A$ and $B$ be two models for a classification task. Let $x$ be a test set and $M$ be a metric for the classification task. $X$ be a random variable on test sets. Now, $M(A,x) = $ Score of model $...
1
vote
1answer
65 views

Why am I getting the logarithm of the probability bigger than zero when using Neural Spline Flows?

I am using a normalizing flow (Neural Spline Flows) to approximate a probability. After some training, the average loss is around 0.5 (so the logarithm of the probability = -0.5). However, when I am ...
0
votes
0answers
25 views

Product of probabilities raised to own powers that can be used for entropy calculation

Suppose $X$ is a random variable taking $k$ values. $$Val(X) = \{x_1, x_2, x_3, \cdots, x_k\} $$ Then what is the following expression of $N(X)$ called in literature? What does it signify? $$ N(X) = \...
0
votes
0answers
22 views

Derivation of an probability expansion used in Word2Vec classifier model

We are using the following notations, for this question, to calculate the probability values \begin{array}{|c|c|} \hline \text{$w$} & \text{target word embedding vector} \\ \hline \text{$c$} &...
0
votes
0answers
17 views

Self sufficient material(s) on maximum likelihood estimation

While studying techniques related to word embeddings, I came across an objective function named maximum likelihood. Word embeddings can be estimated using maximum likelihood as an objective function. ...
0
votes
1answer
75 views

Predicting probabilities of events using neural networks

I've got a few thousands of sequences like 1.23, 2.15. 3.19, 4.30, 5.24, 6.22 where the numbers denote times on which an event happened (there's just a single ...
0
votes
0answers
30 views

Predicting the probability of a periodically happening event occurring at a given time

I have encountered this problem on how to predict the probability of a periodically happening event occurring at a given time. For example, we have an event called being_an_undergrad. There are many ...
3
votes
2answers
728 views

What does the argmax of the expectation of the log likelihood mean?

What does the following equation mean? What does each part of the formula represent or mean? $$\theta^* = \underset {\theta}{\arg \max} \Bbb E_{x \sim p_{data}} \log {p_{model}(x|\theta) }$$
1
vote
1answer
707 views

How do I calculate the probabilities of the BERT model prediction logits?

I might be getting this completely wrong, but please let me first try to explain what I need, and then what's wrong. I have a classification task. The training data has 50 different labels. The ...
1
vote
0answers
37 views

PPO2: Intuition behind Gumbel Softmax and Exploration?

I'm trying to understand the logic behind the magic of using the gumbel distribution for action sampling inside the PPO2 algorithm. This code snippet implements the action sampling, taken from here: <...
2
votes
1answer
92 views

Aren't scores in the Wasserstein GAN probabilities?

I am quite new to GAN and I am reading about WGAN vs DCGAN. Relating to the Wasserstein GAN (WGAN), I read here Instead of using a discriminator to classify or predict the probability of generated ...
0
votes
0answers
16 views

How to get the prediction probability of random sample image from multiclass classification model?

I am performing classification using AlexNet as transfer learning(simply say performing classification using CNN) for five types of class on 18000 images. These 18000 images are divided into Train, ...
0
votes
1answer
126 views

Can you use machine learning for binary data?

I am totally new to artificial intelligence and neural networks and have a broad question that I hope is appropriate to ask here. I am an ecologist working in animal movement and I want to use AI to ...
1
vote
1answer
38 views

In RL as probabilistic inference, why do we take a probability to be $\exp(r(s_t, a_t))$?

In section 2 the paper Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review the author is discussing formulating the RL problem as a probabilistic graphical model. They ...
0
votes
1answer
29 views

How to calculate probability from fuzzy membership grade?

Suppose we have the fuzzy membership grade for a person $x$ with a set $S = \text{set of tall people}$ be $0.9$, i.e. $\mu_S(x)=0.9$. Does this mean that the probability of person $x$ being tall is $0....
1
vote
0answers
42 views

Estimating $\sigma_i$ according to maximum likelihood method

Let be a Bayesian multivariate normal distribution classifier with distinct covariance matrices for each class and isotropic, i.e. with equal values over the entire diagonal and zero otherwise, $\...
1
vote
1answer
84 views

Why is the entire area of a join probability distribution considered when it comes to calculating misclassification?

In the image given below, I do not understand a few things 1) Why is an entire area colored to signify misclassification? For the given decision boundary, only the points between $x_0$ and the ...
5
votes
5answers
682 views

Reinforcement Learning (RL) how to obtain $p(s',r|s,a)$

I am trying to study the book Reinforcement Learning: An Introduction (Sutton & Barto, 2018). In chapter 3.1 the authors state the following exercise Exercise 3.5 Give a table analogous to that ...
2
votes
1answer
280 views

How can supervised learning be viewed as a conditional probability of the labels given the inputs?

In the literature and textbooks, one often sees supervised learning expressed as a conditional probability, e.g., $$\rho(\vec{y}|\vec{x},\vec{\theta})$$ where $\vec{\theta}$ denotes a learned set of ...
4
votes
1answer
152 views

Bayes error rate formula clarification

My questions concern a particular formulation of the Bayes error rate from Wikipedia, summarized below. For a multiclass classifier, the Bayes error rate may be calculated as follows: $$p = 1 - \...
9
votes
1answer
3k views

Is Nassim Taleb right about AI not being able to accurately predict certain types of distributions?

So Taleb has two heuristics to generally describe data distributions. One is Mediocristan, which basically means things that are on a Gaussian distribution such as height and/or weight of people. The ...
2
votes
0answers
50 views

Is the generator distribution in GAN's continuous or discrete?

I have some trouble with the probability densities described in the original paper. My question is based on Goodfellow's paper and tutorial, respectively: Generative Adversarial Networks and NIPS ...
3
votes
1answer
88 views

How does $\mathbb{E}$ suddenly change to $\mathbb{E}_{\pi'}$ in this equation?

In Sutton-Barto's book on page 63 (81 of the pdf): $$\mathbb{E}[R_{t+1} + \gamma v_\pi(S_{t+1}) \mid S_t=s,A_t=\pi'(s)] = \mathbb{E}_{\pi'}[R_{t+1} + \gamma v_\pi(S_{t+1}) \mid S_{t} = s]$$ How does $...
1
vote
0answers
51 views

Importance sampling eq. 5 in paper "Residual Energy-based Models for Text Generation"

In the paper "Residual Energy-Based Models for Text Generation" (arXiv), on page 5, they write that equation 5 is an instance of importance sampling. Equation 5 is: $$ P(x_t \mid x_{<t}) = P_{LM}(...
1
vote
1answer
52 views

Why is probability that at least one hypothesis out of $k$ being consistent with $m$ training examples $k(1- \epsilon)^m$?

My question is actually related to the addition of probabilities. I am reading on computational learning theory from Tom Mitchell's machine learning book. In chapter 7, when proving the upper bound ...
2
votes
0answers
28 views

What is the most efficient data type to store probabilities?

In ML we often have to store a huge amount of values ranging from 0 to 1, mostly being probabilities. The most common data structure to do so seems to be a floating point? Indeed, the range of ...
0
votes
1answer
74 views

What are the prerequisites to start using the TensorFlow Probability library? [closed]

I have some familiarity with the regular Tensorflow library and have been able to create a number of working models with it. There are more than enough resources out there to get up and running and ...
2
votes
0answers
43 views

Expected duration in a state

I am going through Rabiner 1989 and he writes that the discrete probability density function of duration $d$ in state $i$ (that is, staying in a state for duration $d$, conditioned on starting in that ...
1
vote
0answers
26 views

Neural network seems to just figure out the probability of a specific result

I am really new to neural networks, so i was following along with a video series, created by '3blue1brown' on youtube. I created an implementation of the network he explained in c++. I am attempting ...
2
votes
0answers
73 views

Formulation of a Markov Decision Process Problem

Given a list of $N$ questions. If question $i$ is answered correctly (given probability $p_i$), we receive reward $R_i$; if not the quiz terminates. Find the optimal order of questions to maximize ...
2
votes
0answers
58 views

Convert a PAC-learning algorithm into another one which requires no knowledge of the parameter

This is part of the exercise 2.13 in the book Foundations of Machine Learning (page 28). You can refer to chapter 2 for the notations. Consider a family of concept classes $\left\{\mathcal{C}_{s}\...
2
votes
0answers
37 views

What are some approaches to estimate the transition and observation probabilities in POMDP?

What are some common approaches to estimate the transition or observation probabilities, when the probabilities are not exactly known? When realizing a POMDP model, the state model needs additional ...
1
vote
0answers
41 views

Can anyone explain the pixelwise accuracy metric used in this paper? Also a question to the KL Divergence Loss

So I am making a project based on this paper: https://arxiv.org/ftp/arxiv/papers/1901/1901.07761.pdf In this paper, a U-Net is used to generate optimized mechanical structures. I am trying to ...
1
vote
2answers
197 views

How can I convert the probability score between 0 to 1 to another format?

I have trained a multi-class CNN model using fastai. The model splits out probabilites for each of the three classes, which, of course, sum up to 1. The class with highest probability becomes the ...
1
vote
0answers
14 views

Is there an algorithm for "contextual recognition" with probabilities?

Example 1 An object is composed of 3 sub-objects. Object 1: 90% looks like an eye 10% looks like a wheel Object 2: 50% looks like an eye 50% looks like a wheel Object 3: 90% looks like a mouth 10% ...
1
vote
0answers
21 views

Probabilistic classification - normalize results

I have a probabilistic classifier that produces a distribution over my 3 classes - C1, C2, C3. I want to compare some new points I'm classifying to each other, to see which one is the best fit for a ...
1
vote
2answers
226 views

How to make machine learning model that reports ambiguity of the input?

Suppose I want to build a neural network regression model that takes one input and return one output. Here's the training data: ...
1
vote
0answers
48 views

Unique game problem (ML, DP, PP etc)

Looking for a solution to my below game problem. I believe it to require some sort of reinforcement learning, dynamic programming, or probabilistic programming solution, but am unsure... This is my ...
3
votes
1answer
94 views

Viterbi versus filtering

In Chapter 15 of Russel and Norvig's Artificial Intelligence -- A Modern Approach (Third Edition), they describe three basic tasks in temporal inference: Filtering, Likelihood, and Finding the ...
1
vote
1answer
83 views

Is there an AI model with "certainty" built in?

If I see a hundred elephants and fifty of them are grey I'd say the probability of an elephant being grey is 50%. And my certainty of that probability is high. However, if I see two elephants and one ...
8
votes
2answers
307 views

Are probabilistic models dead ends in AI?

I am a strong believer of Marvin Minsky's idea about Artificial General Intelligence (AGI) and one of his thoughts was that probabilistic models are dead ends in the field of AGI. I would really ...