Questions tagged [notation]
For questions related to notation (in general).
26
questions
4
votes
1answer
70 views
What does the term $|\mathcal{A}(s)|$ mean in the $\epsilon$-greedy policy?
I've been looking online for a while for a source that explains these computations but I can't find anywhere what does the $|A(s)|$ mean. I guess $A$ is the action set but I'm not sure about that ...
0
votes
0answers
41 views
What does the notation “for t=T to 1,−1 do” in terms of time steps, in deep recurrent q network?
In looking at an algorithm in the paper Learning to Communicate with Deep Multi-Agent Reinforcement Learning.
Here is the full algorithm:
What does the notation ...
1
vote
1answer
40 views
What do the notations $\sim$ and $\Delta (A) $ mean in the paper “Fairness Through Awareness”?
In this paper Fairness Through Awareness, the notation $\mathbb{E}_{x \sim V} \mathbb{E}_{a \sim \mu_x} L(x,a)$ is being used (page 5 top line), where $V$ denotes the set of individuals (so I guess ...
4
votes
2answers
162 views
Why are the value functions sometimes written with capital letters and other times with lower-case letters?
Why are the state-value and action-value functions are sometimes written in small letters and other times in capitals? For instance, why in the Q-learning algorithm (page 131 of Barto and Sutton's ...
1
vote
1answer
86 views
What does the notation $\partial \theta_{\pi}$ mean in this actor-critic update rule?
One of the steps in the actor-critic algorithm is $$\partial \theta_{\pi} \gets \partial \theta_{\pi} + \nabla_{\theta}\log\pi_{\theta} (a_i | s_i) (R - V_{\theta}(s_i))$$
For me, $\theta$ are just ...
1
vote
1answer
43 views
What does equation in the “related work” section of the GAN paper mean?
I was going through the paper on GAN by Ian Goodfellow. Under the related work section, there is an equation. I cannot decipher the equation. Can anyone help me understand the meaning of the equation? ...
2
votes
1answer
35 views
What does the notation ${s'\sim T(s,a,\cdot)}$ mean?
I have been seeing notations on Expectations with their respective subscripts such as $E_{s_0 \sim D}[V^\pi (s_0)] = \Sigma_{t=0}^\infty[\gamma^t\phi(s_t)]$. This equation is taken from https://ai....
0
votes
1answer
36 views
What is the purpose of the arrow $\leftarrow$ in this formula?
What is the purpose of the arrow $\leftarrow$ in the formula below?
$$V(S_t) \leftarrow V(S_t) + \alpha \left[ G_t - V(S_t) \right]$$
I presume it's not the same as 'equals'.
1
vote
1answer
38 views
Why is exp used in encoder of VAE instead of using the value of standard deviation alone?
There's one VAE example here:
https://towardsdatascience.com/teaching-a-variational-autoencoder-vae-to-draw-mnist-characters-978675c95776.
And the source code of encoder can be found at the ...
0
votes
1answer
38 views
How to understand the average l2 loss?
In the snippet below, the highlighted part is the average norm, but since $1/|p_i|$ is outside the summation, it is very confusing to understand.
is $|p_i|$ l2-norm(as per wolfram) or l1-norm or ...
3
votes
1answer
85 views
What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$?
What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$? I think $|x|$ is the magnitude of $x$.
3
votes
1answer
59 views
What does the notation sup dist mean in distributional RL?
I'm trying to understand distributional RL, based on this article. In one of the equations, there is a symbol $\operatorname{sup dist}$.
\begin{align}
\operatorname{sup dist}_{s, a} (R(s, a) + \...
4
votes
1answer
58 views
What do the subscripts mean in $N_{t,n,\sigma,L}$?
A neural network can apparently be denoted as $N_{t,n,\sigma,L}$. What do these subscripts $t, n, \sigma$ and $L$ mean? Could you link me to a paper, article or webpage with an explanation for this?
3
votes
1answer
51 views
Sutton & Barto's notation $V_{t+n}$ in Chapter 7: $n$-step Bootstrapping
Until Chapter 6 of Sutton & Barto's book on Reinforcement Learning, the authors use $V$ for the current estimate of a state value. Equation (6.1), for example, shows:
$$ V(S_t) \leftarrow V(S_t) +...
4
votes
1answer
59 views
What is the meaning of the square brackets in ant colony optimization?
I'm studying the paper "Minimizing Total Tardiness on a Single Machine Using Ant Colony Optimization" which has proposed to use Ant colony optimization to SMTWTP.
According to this paper:
Each ...
2
votes
1answer
27 views
Understanding the equation of the empirical error
The empirical error equation given in the book Understanding Machine Learning: From Theory to Algorithms is
My intuition for this equation is: total wrong predictions divided by the total number of ...
5
votes
1answer
317 views
Understanding the equation of TD(0) in the paper “Learning to predict by the methods of temporal differences”
In the paper Learning to predict by the methods of temporal differences (p. 15), the weights in the temporal difference learning are updated as given by the equation
$$
\Delta w_t
= \alpha \left(P_{t+...
3
votes
1answer
46 views
Being confused of distribution notations in Deep Learning book
In chapter 5 of Deep Learning book of Ian Goodfellow, some notations in the
loss function as below make me really confused.
I tried to understand $x,y \sim p_{data}$ means a sample $(x, y)$ sampled ...
4
votes
1answer
329 views
How is the policy gradient calculated in REINFORCE?
Reading Sutton and Barto, I see the following in describing policy gradients:
How is the gradient calculated with respect to an action (taken at time t)? I've read implementations of the algorithm, ...
2
votes
3answers
104 views
What does the formula $1-\sum_i(e_i-a_i)^2$ mean in this NEAT Python API?
I have looked at the documentation for the NEAT Python API found here, but it shows calculus like this:
The error for each genome is $1-\sum_i(e_i-a_i)^2$
I haven't learned calculus at the moment....
7
votes
1answer
550 views
Understanding notation of Goodfellow's GAN objective function
What is the meaning of $V(D,G)$? How do we get these expectation parts?
I was trying to understand it following this article: Understanding Generative Adversarial Networks (D.Seita), but, after many ...
2
votes
1answer
50 views
What is the use of the $\epsilon$ term in this back-propagation equation?
I am currently looking at different documents to understand back-propagation, mainly at this document. Now, at page 3, there is the $\epsilon$ symbol involved:
While I understand the main part of the ...
2
votes
1answer
117 views
Understanding the notation in the definition of the expected reward
I am new to RL and I am trying to work through the book Reinforcement Learning: An Introduction I (Sutton & Barto, 2018). In chapter 3 on Finite Markov Decision Processes, the authors write the ...
2
votes
1answer
87 views
What does the notation $\nabla_\theta \mathcal{L}$ mean?
Here's the general algorithm of maximum entropy inverse reinforcement learning.
This uses a gradient descent algorithm. The point that I do not understand is there is only a single gradient value $\...
3
votes
2answers
292 views
What does the argmax of the expectation of the log likelihood mean?
What does the following equation mean? What does each part of the formula represent or mean?
$$\theta^* = \underset {\theta}{\arg \max} \Bbb E_{x \sim p_{data}} \log {p_{model}(x|\theta) }$$
1
vote
1answer
199 views
What is $I$ in the noise described in the paper “Parameter Space Noise for Exploration”?
In the paper Parameter Space Noise for Exploration, the authors describe the noise that they add to the parameter vector as:
$$
\tilde{\theta} = \theta + \mathcal{N}(0, \sigma^2I)
$$
is $I$ simply ...
