9 votes
Accepted

What is the meaning of $V(D,G)$ in the GAN objective function?

To understand this equation first you need to understand the context in which it is first introduced. We have two neural networks (i.e. $D$ and $G$) that are playing a minimax game. This means that ...
user avatar
  • 3,093
7 votes

How are the reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ equivalent?

In general the different reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ are not equivalent mathematically, so you will not find any formal proof. It is possible for the functions to resolve to ...
user avatar
  • 23.8k
7 votes
Accepted

How is the policy gradient calculated in REINFORCE?

The first part of this answer is a little background that might bolster your intuition for what's going on. The second part is the more practical and direct answer to your question. The gradient is ...
user avatar
6 votes
Accepted

What does the term $|\mathcal{A}(s)|$ mean in the $\epsilon$-greedy policy?

This expression: $|\mathcal{A}(s)|$ means $|\quad|$ the size of $\mathcal{A}(s)$ the set of actions in state $s$ or more simply the number of actions allowed in the state. This makes sense in the ...
user avatar
  • 23.8k
6 votes
Accepted

What is a filter in the context of graph convolutional networks?

Short answer Check out the paper of Shuman et al. [1], it provides some background on Graph Signal Processing, including answers to your questions in sections II.C and III.A Long Answer Question 1 Yes,...
user avatar
5 votes
Accepted

What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$?

$\|x\| = |x|$ denotes the absolute value norm, which is a special case of the $L_1$ norm defined on the 1-D vector spaces formed by real or complex numbers. $\|\textbf{x}\|_1 = \sum_{i=1}^n|x_i|$ ...
user avatar
  • 360
5 votes
Accepted

Understanding the equation of TD(0) in the paper "Learning to predict by the methods of temporal differences"

When lambda = 0 as in TD(0), how does the method learn? As it appears, with lambda = 0, there will never be a change in weight and hence no learning. I think the detail that you're missing is that ...
user avatar
  • 9,379
5 votes
Accepted

What does the notation $\mathcal{N}(z; \mu, \sigma)$ stand for in statistics?

It means that $z$ has a (multivariate) normal distribution with 0 mean and identity covariance matrix. This essentially means each individual element of the vector $z$ has a standard normal ...
user avatar
4 votes

Why are the value functions sometimes written with capital letters and other times with lower-case letters?

In the Sutton and Barto book $q(s,a)$ is used to denote the true expected value of taking action $a$ in state $s$, whereas capital $Q(s,a)$ is used to denote an estimate of $q(s,a)$. However, there is ...
user avatar
4 votes
Accepted

What do the subscripts mean in $N_{t,n,\sigma,L}$?

Here is a paper with the mathematical definition of each term: Let Nt,n,σ,L be all target functions that can be implemented using a neural network of depth t, size n, activation function σ, and ...
user avatar
  • 870
4 votes

How are the reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ equivalent?

Let $R(s)$ denote a probability distribution over rewards that our agent may get in some MDP as a reward for entering a state $s$. The easiest case is to demonstrate that we can also choose to write ...
user avatar
  • 9,379
4 votes
Accepted

Why is the equation $r(s', a, s') =\sum_{r \in \mathcal{R}} r \frac{p\left(s^{\prime}, r \mid s, a\right)}{p\left(s^{\prime} \mid s, a\right)}$true?

No, the substitution you suggest based on Equation (3.4) is not correct because you forgot about the $\sum_{r \in \mathcal{R}}$ in the right-hand side Equation (3.4). Equation (3.4) says (leaving out ...
user avatar
  • 9,379
4 votes

What does the notation $\nabla_\theta \mathcal{L}$ mean?

This is standard backpropagation. The gradient term you see is in fact a vector of partial derivatives where each element is the partial derivative of the log-likelihood with respect to each element ...
user avatar
4 votes
Accepted

What is the name of this letter $\mathcal{J}$?

It's an uppercase "J" from the math calligraphy alphabet, i.e. \mathcal{J} in latex. $\mathcal{J}$
user avatar
  • 876
4 votes
Accepted

Are the authors of the VAE paper writing the PDFs as a function of the random variables?

When it comes to notation/terminology, often, people in machine learning are (a bit?) sloppy, which causes a lot of confusion, especially for newcomers to the field or people not very math-savvy. I ...
user avatar
  • 33.8k
3 votes

What does the argmax of the expectation of the log likelihood mean?

This equation and more information of it can be found in Expectation Maximization Wikipedia site and the explanation there was as follows (formula there in two parts): Some more explanation from same ...
user avatar
  • 927
3 votes
Accepted

Being confused of distribution notations in Deep Learning book

At page 130 of the same book, the author states that $\hat{p}_\text{data}$ is an empirical distribution defined by the training data. Similarly, at page 129, he states that $p_\text{data}$ is the true ...
user avatar
  • 33.8k
3 votes
Accepted

What is the meaning of the square brackets in ant colony optimization?

The square brackets $[]$ in $[\tau_{ij}]^\alpha$ and $[\eta_{ij}]^\beta$ may be just a way of emphasing that the elements $\tau_{ij} \in \mathbb{R}$ and $\eta_{ij} \in \mathbb{R}$ of respectively the ...
user avatar
  • 33.8k
3 votes
Accepted

What does the notation sup dist mean in distributional RL?

It doesn't seem that it is a "proper" symbol. I guess that $\sup$ simply refers to the supremum, that is, you want to select actions that maximize the quantity that comes to the right of $\sup$, ...
user avatar
3 votes
Accepted

What is a probability distribution in machine learning?

Random variables You do not necessarily need to understand the concept of a random variable (r.v.) to understand the concept of a probability distribution, but the concept of a random variable is ...
user avatar
  • 33.8k
3 votes
Accepted

In variational autoencoders, what does p(x|z) mean?

Whilst you're right that for any continuous distribution $P(X = x) = 0 \;; \forall x \in \mathcal{X}$ where $\mathcal{X}$ is there support of the distribution, they are not referring to probabilities ...
user avatar
3 votes

Why is noise vector represented by letter $z$?

I don't think there's any rationale behind the usage of the letter $z$ to denote the noise (which sometimes is also denoted by $\epsilon$ in other contexts), apart from the fact that $x$ and $y$ are ...
user avatar
  • 33.8k
3 votes
Accepted

Where are the parentheses in the Bellman update rule?

Here's your equation with an additional couple of parenthesis that emphasizes the order of the operations (note that you had a small typo in your original equation). $$v_{\pi}(s) =\sum_a \pi(a \mid s) ...
user avatar
  • 33.8k
3 votes
Accepted

What do the square brackets $[ ]$ and $\mid$ mean in $[G_t \mid S_t=s]$?

The square brackets are part of the expectation operator (i.e. a function of a random variable, which in this case is $G_t$). This is common notation for the expectation. So, it's not $\left[ X \right]...
user avatar
  • 33.8k
2 votes

What is a probability distribution in machine learning?

A probability distribution in ML is the same as a probability distribution elsewhere. A probability distribution (or probability function, or probability mass function, or probability density ...
user avatar
2 votes
Accepted

What does the notation $[m]=\{1, \ldots, m\}$ mean in the equation of the empirical error?

This is a commonly used notation in theoretical computer science. $[m]$ is not the variable $m$, but is instead the set of integers from $1$ to $m$ inclusive. The empirical error equation thus reads ...
user avatar
2 votes

What does the formula $1-\sum_i(e_i-a_i)^2$ mean in this NEAT Python API?

$$1-\sum_i(e_i-a_i)^2$$ $\sum$ - there just means sum. It is the greek letter for S. You can rewrite the above formula as $$1 -[(e_1 - a_1)^2+(e_2-a_2)^2+(e_3-a_3)^2+\ldots ]$$ $\sum$ just helps us ...
user avatar
2 votes
Accepted

Where are the parentheses in the definition of $r(s,a)$?

Your first option is correct: $$r(s,a) = \mathbb{E}\left[R_t|S_{t-1}=s,A_{t-1}=a\right]=\sum_{r\in \mathcal{R}}\left[r\sum_{s'\in \mathcal{S}}p(s',r|s,a)\right]$$ It's partly a matter of taste, but ...
user avatar
  • 23.8k
2 votes
Accepted

What does equation in the "related work" section of the GAN paper mean?

In full: The limit, as standard deviation $\sigma$ tends towards zero, of the gradient with respect to vector $\mathbf{x}$, of the expectation - where perturbation $\epsilon$ follows the normal ...
user avatar
  • 23.8k
2 votes
Accepted

What does the notation ${s'\sim T(s,a,\cdot)}$ mean?

The dot ($.$) at the end of $T(s,a,.)$ shows all possible states that we can go from state $S$ by doing action $a$. As you know there are some probabilities here for choosing those states, that the ...
user avatar
  • 1,663

Only top scored, non community-wiki answers of a minimum length are eligible