2

Welcome to AI.SE @vdbuss, and great first question! This point is touched on in Section 15.2.3 (page 576 in my copy), in the second paragraph, and there's a good exercise at the end of the chapter (15.4) that is designed to get you to think through exactly why these are different procedures. If you want to really absorb it, I suggest trying to work out that ...


2

Another specific way to do this if one uses a neural network for this. Use a dropout a layer in your network and instead of scaling the activations at test time, one can sample the activations (just like in training-time) and predict multiple times for a given input, then look at distribution of your outputs. Intuitively this would add "probabilistic, ...


2

A probability distribution in ML is the same as a probability distribution elsewhere. A probability distribution (or probability function, or probability mass function, or probability density function) is any function that accepts as input elements of some specific set $x \in X$, and produces as output, real-valued numbers between 0 and 1 (inclusive), such ...


2

Although this question is slightly primarily opinion-based and too broad (and I will probably close it as such) and a good answer will necessarily depend on your background, I will list some of the main theoretical prerequisites that everyone should ideally be familiar with before diving into TensorFlow Probability (TFP). I am familiar with TFP, given that ...


1

Let $A$ and $B$ be two events. In general, the probability that either $A$ or $B$ occurs is defined as $$ P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B) $$ If $A$ and $B$ are disjoint, i.e. they cannot happen at the same time, then $P(A \text{ and } B) = 0$, so the above formula becomes $$ P(A \text{ or } B) = P(A) + P(B) $$ If the probability of ...


1

This formulation can indeed be misleading, as the output of a neural network is usually deterministic (i.e. given the same input $x$, the output is always the same, so there is no sampling), and there isn't really a probability distribution that models any uncertainty associated with the parameters of the network or the input. People often use this ...


1

You could maybe do something like this, it's a bit hackish \begin{equation} y = C_1\cdot 1 + C_2 \cdot 0.5 + C_3 \cdot 0 \end{equation} $y$ represents the output and its bounded $\in [0, 1]$. $C_i$ is probability for class $i$. This way when $C_1 \approx 1, C_2 \approx 0, C_3 \approx 0$ you have \begin{equation} y \approx 1\cdot 1 + 0.5 \cdot 0 + 0 \cdot 0 \...


1

Random variables You do not necessarily need to understand the concept of a random variable (r.v.) to understand the concept of a probability distribution, but the concept of a random variable is strictly connected to the concept of a probability distribution (given that each random variable has an associated probability distribution), so, before proceeding,...


1

You make a valid point, vanilla neural networks cannot give you more than a point estimate of class confidence. If one wanted to actually gain an idea of variance, you need a framework that allows such a mechanism. A popular methodology to this is Bayesian modeling. In other words given some data, $\Omega$, you want to create some form of descriminative ...


Only top voted, non community-wiki answers of a minimum length are eligible