Questions tagged [notation]
For questions related to notation (in general).
36
questions
2
votes
1answer
47 views
What is $ \nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?
I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML. I came across this blog: https://lilianweng.github.io/lil-log/2018/11/30/meta-...
3
votes
2answers
64 views
What does the parameter $y$ stand for in function $g(y,\mu,\sigma)$ related to REINFORCE algorithm?
I am wondering what the parameter $y$ in the function $g(y,\mu,\sigma)=\frac{1}{(2\pi)^{1/2}\sigma}e^{-(y-\mu)^{2/2\sigma^2}}$ stands for in Section 6 (page 14) of the paper introducing the REINFORCE ...
2
votes
1answer
41 views
In AlphaZero, do we need to store the data of terminal states?
I have a question about the training data used during the update/back-propagation step of the neural network in AlphaZero.
From the paper:
The data for each time-step $t$ is stored as ($s_t, \pi_t, ...
0
votes
1answer
62 views
What is the meaning of these equations in Noise2Noise paper?
I am trying to understand what is meant by following equations in the Noise2Noise paper by Nvidia.
What is meant by the equation in this image? What is $\mathbb{E}_y\{y\}$? And how should I try to ...
1
vote
1answer
49 views
In the definition of the state-action value function, what is the random variable we take the expectation of?
I know that
$$\mathbb{E}[g(X) \mid A] = \sum\limits_{x} g(x) p_{X \mid A}(x)$$
for any random variable $X$.
Now, consider the following expression.
$$\mathbb{E}_{\pi} \left[ \sum \limits_{k=0}^{\infty}...
0
votes
1answer
39 views
Do the rows of the design matrix refer to the observations or predictors?
I attempt to understand the formulation of dictionary learning for this paper:
Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution
Multimodal Task-Driven ...
3
votes
1answer
92 views
What does the notation $\mathcal{N}(z; \mu, \sigma)$ stand for in statistics?
I know that the notation $\mathcal{N}(\mu, \sigma)$ stands for a normal distribution.
But I'm reading the book "An Introduction to Variational Autoencoders" and in it, there is this notation:...
4
votes
2answers
105 views
Why do we use $X_{I_t,t}$ and $v_{I_t}$ to denote the reward received and the at time step $t$ and the distribution of the chosen arm $I_t$?
I'm doing some introductory research on classical (stochastic) MABs. However, I'm a little confused about the common notation (e.g. in the popular paper of Auer (2002) or Bubeck and Cesa-Bianchi (2012)...
4
votes
1answer
79 views
What does the term $|\mathcal{A}(s)|$ mean in the $\epsilon$-greedy policy?
I've been looking online for a while for a source that explains these computations but I can't find anywhere what does the $|A(s)|$ mean. I guess $A$ is the action set but I'm not sure about that ...
0
votes
0answers
43 views
What does the notation “for t=T to 1,ā1 do” in terms of time steps, in deep recurrent q network?
In looking at an algorithm in the paper Learning to Communicate with Deep Multi-Agent Reinforcement Learning.
Here is the full algorithm:
What does the notation ...
1
vote
1answer
44 views
What do the notations $\sim$ and $\Delta (A) $ mean in the paper “Fairness Through Awareness”?
In this paper Fairness Through Awareness, the notation $\mathbb{E}_{x \sim V} \mathbb{E}_{a \sim \mu_x} L(x,a)$ is being used (page 5 top line), where $V$ denotes the set of individuals (so I guess ...
4
votes
2answers
171 views
Why are the value functions sometimes written with capital letters and other times with lower-case letters?
Why are the state-value and action-value functions are sometimes written in small letters and other times in capitals? For instance, why in the Q-learning algorithm (page 131 of Barto and Sutton's ...
1
vote
1answer
92 views
What does the notation $\partial \theta_{\pi}$ mean in this actor-critic update rule?
One of the steps in the actor-critic algorithm is $$\partial \theta_{\pi} \gets \partial \theta_{\pi} + \nabla_{\theta}\log\pi_{\theta} (a_i | s_i) (R - V_{\theta}(s_i))$$
For me, $\theta$ are just ...
1
vote
1answer
53 views
What does equation in the “related work” section of the GAN paper mean?
I was going through the paper on GAN by Ian Goodfellow. Under the related work section, there is an equation. I cannot decipher the equation. Can anyone help me understand the meaning of the equation? ...
2
votes
1answer
38 views
What does the notation ${s'\sim T(s,a,\cdot)}$ mean?
I have been seeing notations on Expectations with their respective subscripts such as $E_{s_0 \sim D}[V^\pi (s_0)] = \Sigma_{t=0}^\infty[\gamma^t\phi(s_t)]$. This equation is taken from https://ai....
1
vote
1answer
38 views
What is the purpose of the arrow $\leftarrow$ in this formula?
What is the purpose of the arrow $\leftarrow$ in the formula below?
$$V(S_t) \leftarrow V(S_t) + \alpha \left[ G_t - V(S_t) \right]$$
I presume it's not the same as 'equals'.
1
vote
1answer
60 views
Why is exp used in encoder of VAE instead of using the value of standard deviation alone?
There's one VAE example here:
https://towardsdatascience.com/teaching-a-variational-autoencoder-vae-to-draw-mnist-characters-978675c95776.
And the source code of encoder can be found at the ...
0
votes
1answer
49 views
How to understand the average l2 loss?
In the snippet below, the highlighted part is the average norm, but since $1/|p_i|$ is outside the summation, it is very confusing to understand.
is $|p_i|$ l2-norm(as per wolfram) or l1-norm or ...
3
votes
1answer
92 views
What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$?
What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$? I think $|x|$ is the magnitude of $x$.
3
votes
1answer
62 views
What does the notation sup dist mean in distributional RL?
I'm trying to understand distributional RL, based on this article. In one of the equations, there is a symbol $\operatorname{sup dist}$.
\begin{align}
\operatorname{sup dist}_{s, a} (R(s, a) + \...
4
votes
1answer
58 views
What do the subscripts mean in $N_{t,n,\sigma,L}$?
A neural network can apparently be denoted as $N_{t,n,\sigma,L}$. What do these subscripts $t, n, \sigma$ and $L$ mean? Could you link me to a paper, article or webpage with an explanation for this?
3
votes
1answer
56 views
Sutton & Barto's notation $V_{t+n}$ in Chapter 7: $n$-step Bootstrapping
Until Chapter 6 of Sutton & Barto's book on Reinforcement Learning, the authors use $V$ for the current estimate of a state value. Equation (6.1), for example, shows:
$$ V(S_t) \leftarrow V(S_t) +...
5
votes
1answer
82 views
What is the meaning of the square brackets in ant colony optimization?
I'm studying the paper "Minimizing Total Tardiness on a Single Machine Using Ant Colony Optimization" which has proposed to use Ant colony optimization to SMTWTP.
According to this paper:
...
2
votes
1answer
27 views
Understanding the equation of the empirical error
The empirical error equation given in the book Understanding Machine Learning: From Theory to Algorithms is
My intuition for this equation is: total wrong predictions divided by the total number of ...
5
votes
1answer
346 views
Understanding the equation of TD(0) in the paper “Learning to predict by the methods of temporal differences”
In the paper Learning to predict by the methods of temporal differences (p. 15), the weights in the temporal difference learning are updated as given by the equation
$$
\Delta w_t
= \alpha \left(P_{t+...
4
votes
1answer
52 views
Being confused of distribution notations in Deep Learning book
In chapter 5 of Deep Learning book of Ian Goodfellow, some notations in the
loss function as below make me really confused.
I tried to understand $x,y \sim p_{data}$ means a sample $(x, y)$ sampled ...
6
votes
1answer
441 views
How is the policy gradient calculated in REINFORCE?
Reading Sutton and Barto, I see the following in describing policy gradients:
How is the gradient calculated with respect to an action (taken at time t)? I've read implementations of the algorithm, ...
2
votes
3answers
118 views
What does the formula $1-\sum_i(e_i-a_i)^2$ mean in this NEAT Python API?
I have looked at the documentation for the NEAT Python API found here, where it's written
The error for each genome is $1-\sum_i(e_i-a_i)^2$
I have not yet learned calculus, so I can't understand ...
9
votes
1answer
854 views
Understanding notation of Goodfellow's GAN objective function
What is the meaning of $V(D,G)$? How do we get these expectation parts?
I was trying to understand it following this article: Understanding Generative Adversarial Networks (D.Seita), but, after many ...
4
votes
2answers
149 views
How are the reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ equivalent?
In this video, the lecturer states that $R(s)$, $R(s, a)$ and $R(s, a, s')$ are equivalent representations of the reward function. Intuitively, this is the case, according to the same lecturer, ...
2
votes
1answer
56 views
What is the use of the $\epsilon$ term in this back-propagation equation?
I am currently looking at different documents to understand back-propagation, mainly at this document. Now, at page 3, there is the $\epsilon$ symbol involved:
While I understand the main part of the ...
3
votes
1answer
107 views
Why is the equation $r(s', a, s') =\sum_{r \in \mathcal{R}} r \frac{p\left(s^{\prime}, r \mid s, a\right)}{p\left(s^{\prime} \mid s, a\right)}$true?
I am referring to eq. 3.6 (page 49) based on Sutton's online book and can be found in an image below.
I could not make sense of the final derivation of the equation $r(s, a, s')$. My question is ...
2
votes
1answer
123 views
Understanding the notation in the definition of the expected reward
I am new to RL and I am trying to work through the book Reinforcement Learning: An Introduction I (Sutton & Barto, 2018). In chapter 3 on Finite Markov Decision Processes, the authors write the ...
2
votes
1answer
134 views
What does the notation $\nabla_\theta \mathcal{L}$ mean?
Here's the general algorithm of maximum entropy inverse reinforcement learning.
This uses a gradient descent algorithm. The point that I do not understand is there is only a single gradient value $\...
3
votes
2answers
499 views
What does the argmax of the expectation of the log likelihood mean?
What does the following equation mean? What does each part of the formula represent or mean?
$$\theta^* = \underset {\theta}{\arg \max} \Bbb E_{x \sim p_{data}} \log {p_{model}(x|\theta) }$$
1
vote
1answer
205 views
What is $I$ in the noise described in the paper “Parameter Space Noise for Exploration”?
In the paper Parameter Space Noise for Exploration, the authors describe the noise that they add to the parameter vector as:
$$
\tilde{\theta} = \theta + \mathcal{N}(0, \sigma^2I)
$$
is $I$ simply ...
