Questions tagged [notation]

For questions related to notation (in general).

Filter by
Sorted by
Tagged with
12
votes
1answer
1k views

Understanding notation of Goodfellow's GAN objective function

What is the meaning of $V(D,G)$? How do we get these expectation parts? I was trying to understand it following this article: Understanding Generative Adversarial Networks (D.Seita), but, after many ...
7
votes
1answer
673 views

How is the policy gradient calculated in REINFORCE?

Reading Sutton and Barto, I see the following in describing policy gradients: How is the gradient calculated with respect to an action (taken at time t)? I've read implementations of the algorithm, ...
5
votes
2answers
352 views

How are the reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ equivalent?

In this video, the lecturer states that $R(s)$, $R(s, a)$ and $R(s, a, s')$ are equivalent representations of the reward function. Intuitively, this is the case, according to the same lecturer, ...
5
votes
1answer
94 views

What does the term $|\mathcal{A}(s)|$ mean in the $\epsilon$-greedy policy?

I've been looking online for a while for a source that explains these computations but I can't find anywhere what does the $|A(s)|$ mean. I guess $A$ is the action set but I'm not sure about that ...
5
votes
1answer
400 views

Understanding the equation of TD(0) in the paper "Learning to predict by the methods of temporal differences"

In the paper Learning to predict by the methods of temporal differences (p. 15), the weights in the temporal difference learning are updated as given by the equation $$ \Delta w_t = \alpha \left(P_{t+...
5
votes
1answer
89 views

What is the meaning of the square brackets in ant colony optimization?

I'm studying the paper "Minimizing Total Tardiness on a Single Machine Using Ant Colony Optimization" which has proposed to use Ant colony optimization to SMTWTP. According to this paper: ...
4
votes
2answers
229 views

Why are the value functions sometimes written with capital letters and other times with lower-case letters?

Why are the state-value and action-value functions are sometimes written in small letters and other times in capitals? For instance, why in the Q-learning algorithm (page 131 of Barto and Sutton's ...
4
votes
2answers
113 views

Why do we use $X_{I_t,t}$ and $v_{I_t}$ to denote the reward received and the at time step $t$ and the distribution of the chosen arm $I_t$?

I'm doing some introductory research on classical (stochastic) MABs. However, I'm a little confused about the common notation (e.g. in the popular paper of Auer (2002) or Bubeck and Cesa-Bianchi (2012)...
4
votes
1answer
63 views

What do the subscripts mean in $N_{t,n,\sigma,L}$?

A neural network can apparently be denoted as $N_{t,n,\sigma,L}$. What do these subscripts $t, n, \sigma$ and $L$ mean? Could you link me to a paper, article or webpage with an explanation for this?
4
votes
1answer
57 views

Being confused of distribution notations in Deep Learning book

In chapter 5 of Deep Learning book of Ian Goodfellow, some notations in the loss function as below make me really confused. I tried to understand $x,y \sim p_{data}$ means a sample $(x, y)$ sampled ...
3
votes
1answer
128 views

What does the notation $\mathcal{N}(z; \mu, \sigma)$ stand for in statistics?

I know that the notation $\mathcal{N}(\mu, \sigma)$ stands for a normal distribution. But I'm reading the book "An Introduction to Variational Autoencoders" and in it, there is this notation:...
3
votes
1answer
94 views

What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$?

What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$? I think $|x|$ is the magnitude of $x$.
3
votes
2answers
720 views

What is a probability distribution in machine learning?

If we were learning or working in the machine learning field, then we frequently come across the term "probability distribution". I know what probability, conditional probability, and ...
3
votes
1answer
269 views

What does the notation $\nabla_\theta \mathcal{L}$ mean?

Here's the general algorithm of maximum entropy inverse reinforcement learning. This uses a gradient descent algorithm. The point that I do not understand is there is only a single gradient value $\...
3
votes
1answer
71 views

What does the notation sup dist mean in distributional RL?

I'm trying to understand distributional RL, based on this article. In one of the equations, there is a symbol $\operatorname{sup dist}$. \begin{align} \operatorname{sup dist}_{s, a} (R(s, a) + \...
3
votes
1answer
114 views

Why is the equation $r(s', a, s') =\sum_{r \in \mathcal{R}} r \frac{p\left(s^{\prime}, r \mid s, a\right)}{p\left(s^{\prime} \mid s, a\right)}$true?

I am referring to eq. 3.6 (page 49) based on Sutton's online book and can be found in an image below. I could not make sense of the final derivation of the equation $r(s, a, s')$. My question is ...
3
votes
2answers
70 views

What does the parameter $y$ stand for in function $g(y,\mu,\sigma)$ related to REINFORCE algorithm?

I am wondering what the parameter $y$ in the function $g(y,\mu,\sigma)=\frac{1}{(2\pi)^{1/2}\sigma}e^{-(y-\mu)^{2/2\sigma^2}}$ stands for in Section 6 (page 14) of the paper introducing the REINFORCE ...
3
votes
1answer
79 views

Sutton & Barto's notation $V_{t+n}$ in Chapter 7: $n$-step Bootstrapping

Until Chapter 6 of Sutton & Barto's book on Reinforcement Learning, the authors use $V$ for the current estimate of a state value. Equation (6.1), for example, shows: $$ V(S_t) \leftarrow V(S_t) +...
3
votes
2answers
762 views

What does the argmax of the expectation of the log likelihood mean?

What does the following equation mean? What does each part of the formula represent or mean? $$\theta^* = \underset {\theta}{\arg \max} \Bbb E_{x \sim p_{data}} \log {p_{model}(x|\theta) }$$
2
votes
2answers
61 views

What is the correct notation for an operation that applies to each element of an array independently?

I am looking for the standard notation to define element-wise / Hadamard-style functions, if there is one. That is to say, if the operator I am looking for were represented by a hexagon ⬡, I could use ...
2
votes
1answer
68 views

What do the variables in the cross-correlation formula mean?

I understand what cross-correlation does given a kernel and an input image, but the formula confuses me a little. Given here in Goodfellow's Deep Learning (page 329), I can't quite understand what $m$ ...
2
votes
1answer
40 views

What does the notation ${s'\sim T(s,a,\cdot)}$ mean?

I have been seeing notations on Expectations with their respective subscripts such as $E_{s_0 \sim D}[V^\pi (s_0)] = \Sigma_{t=0}^\infty[\gamma^t\phi(s_t)]$. This equation is taken from https://ai....
2
votes
1answer
38 views

What does the notation $[m]=\{1, \ldots, m\}$ mean in the equation of the empirical error?

The empirical error equation given in the book Understanding Machine Learning: From Theory to Algorithms is My intuition for this equation is: total wrong predictions divided by the total number of ...
2
votes
3answers
122 views

What does the formula $1-\sum_i(e_i-a_i)^2$ mean in this NEAT Python API?

I have looked at the documentation for the NEAT Python API found here, where it's written The error for each genome is $1-\sum_i(e_i-a_i)^2$ I have not yet learned calculus, so I can't understand ...
2
votes
1answer
136 views

Understanding the notation in the definition of the expected reward

I am new to RL and I am trying to work through the book Reinforcement Learning: An Introduction I (Sutton & Barto, 2018). In chapter 3 on Finite Markov Decision Processes, the authors write the ...
2
votes
1answer
41 views

Is the Bandit Problem an MDP?

I've read Sutton and Barto's introductory RL book. They define a policy as a mapping from states to probabilities of selecting each possible action. If the agent is following policy $\pi$ at time $t$, ...
2
votes
1answer
71 views

What is the use of the $\epsilon$ term in this back-propagation equation?

I am currently looking at different documents to understand back-propagation, mainly at this document. Now, at page 3, there is the $\epsilon$ symbol involved: While I understand the main part of the ...
2
votes
1answer
64 views

What is $ \nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?

I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML. I came across this blog: https://lilianweng.github.io/lil-log/2018/11/30/meta-...
2
votes
1answer
57 views

In AlphaZero, do we need to store the data of terminal states?

I have a question about the training data used during the update/back-propagation step of the neural network in AlphaZero. From the paper: The data for each time-step $t$ is stored as ($s_t, \pi_t, ...
1
vote
2answers
75 views

In variational autoencoders, what does p(x|z) mean?

If $x \sim \mathcal{N}(\mu,\,\sigma^{2})$, then it is a continuous variable, and therefore $P(x) = 0$ for any x. One can only consider things like $P(x<X)$ to get a probability greater than 0. So ...
1
vote
1answer
66 views

In the definition of the state-action value function, what is the random variable we take the expectation of?

I know that $$\mathbb{E}[g(X) \mid A] = \sum\limits_{x} g(x) p_{X \mid A}(x)$$ for any random variable $X$. Now, consider the following expression. $$\mathbb{E}_{\pi} \left[ \sum \limits_{k=0}^{\infty}...
1
vote
1answer
143 views

Is it abuse of notation to use tilde operator in this context?

The following is a way to use tilde (∼) in context of random variables or random vectors. In statistics, the tilde is frequently used to mean "has the distribution (of)," for instance, $X∼N(...
1
vote
1answer
48 views

What do the notations $\sim$ and $\Delta (A) $ mean in the paper "Fairness Through Awareness"?

In this paper Fairness Through Awareness, the notation $\mathbb{E}_{x \sim V} \mathbb{E}_{a \sim \mu_x} L(x,a)$ is being used (page 5 top line), where $V$ denotes the set of individuals (so I guess ...
1
vote
1answer
59 views

What does equation in the "related work" section of the GAN paper mean?

I was going through the paper on GAN by Ian Goodfellow. Under the related work section, there is an equation. I cannot decipher the equation. Can anyone help me understand the meaning of the equation? ...
1
vote
1answer
38 views

What is the purpose of the arrow $\leftarrow$ in this formula?

What is the purpose of the arrow $\leftarrow$ in the formula below? $$V(S_t) \leftarrow V(S_t) + \alpha \left[ G_t - V(S_t) \right]$$ I presume it's not the same as 'equals'.
1
vote
1answer
123 views

Why is exp used in encoder of VAE instead of using the value of standard deviation alone?

There's one VAE example here: https://towardsdatascience.com/teaching-a-variational-autoencoder-vae-to-draw-mnist-characters-978675c95776. And the source code of encoder can be found at the ...
1
vote
1answer
29 views

Does a trajectory in reinforcement learning contain the last action?

From what I learn from CS285 and OpenAI's spinning up, a trajectory in RL is a sequence of state-action pairs: $$\tau = \{s_0, a_0, ..., s_t, a_t\}$$ And the resulting trajectory probability is: $$ P(\...
1
vote
1answer
45 views

Why is the behaviour policy denoted by $\beta$ and the exploration policy by $ \mu'$ in the DDPG paper?

I am learning about the deep deterministic policy gradient (DDPG) (Lillicrap et al, 2016) and got confused about the notation of the behavior policy. Lillicrap et al. denote the policy gradient by $$\...
1
vote
1answer
110 views

What does the notation $\partial \theta_{\pi}$ mean in this actor-critic update rule?

One of the steps in the actor-critic algorithm is $$\partial \theta_{\pi} \gets \partial \theta_{\pi} + \nabla_{\theta}\log\pi_{\theta} (a_i | s_i) (R - V_{\theta}(s_i))$$ For me, $\theta$ are just ...
1
vote
1answer
207 views

What is $I$ in the noise described in the paper "Parameter Space Noise for Exploration"?

In the paper Parameter Space Noise for Exploration, the authors describe the noise that they add to the parameter vector as: $$ \tilde{\theta} = \theta + \mathcal{N}(0, \sigma^2I) $$ is $I$ simply ...
0
votes
1answer
39 views

What, exactly, do mlp(64,64) and mlp(64,128,1024) mean in PointNet, and how many input neurons does 1 (x,y,z) point have?

I couldn't find out how to interpret the multilayer perceptron notation given in PointNet. Specifically, I am looking to find out what the numbers inside the parentheses of ...
0
votes
1answer
49 views

Does generator in conditonal GAN obey probability laws?

In probability, we have two types of probability functions: unconditional probability $p(x)$ and conditional probability $p(x | y)$. Both are fundamentally different and the latter can be obtained by ...
0
votes
1answer
109 views

What is the meaning of these equations in Noise2Noise paper?

I am trying to understand what is meant by following equations in the Noise2Noise paper by Nvidia. What is meant by the equation in this image? What is $\mathbb{E}_y\{y\}$? And how should I try to ...
0
votes
1answer
58 views

Do the rows of the design matrix refer to the observations or predictors?

I attempt to understand the formulation of dictionary learning for this paper: Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution Multimodal Task-Driven ...
0
votes
1answer
67 views

How to understand the average l2 loss?

In the snippet below, the highlighted part is the average norm, but since $1/|p_i|$ is outside the summation, it is very confusing to understand. is $|p_i|$ l2-norm(as per wolfram) or l1-norm or ...
0
votes
1answer
25 views

Why using negative integers (as dimensions?) in tensor shapes rather than natural numbers?

Consider the following paragraph from A.1 MULTI-MNIST AND CLEVR of A IMPLEMENTATION DETAILS from the research paper titled ...
0
votes
0answers
30 views

Is there any difference between input and conditional input incase of neural networks?

In the research paper titled Conditional Generative Adversarial Nets by Mehdi Mirza and Simon Osindero, there is a notion of conditioning a neural network on class label. It has been mentioned in <...
0
votes
0answers
16 views

To denote a training example should I use row vector or column vector?

This code accesses the first 3 examples in the iris data set, from sklearn.datasets import load_iris iris = load_iris() print(iris.data[:3]) and gives ...
0
votes
0answers
24 views

Do $V_\theta$ and $V_s$ represent partial or total derivatives in the paper "Learning Continuous Control Policies by Stochastic Value Gradients"?

I was reading up on the Stochastic Value Gradients paper by Heess et al. In the paper, they describe a recursive process to calculate path-wise derivatives via equations (3) and (4), at the bottom of ...
0
votes
0answers
14 views

Is there an arrow missing in the derivation of front-door adjustment formula from do-calculus?

Here is Judea Pearl's derivation of the front-door adjustment formula: Is there an arrow from Genotype to Cancer missing in the second diagram at the right? just like this?