# Questions tagged [notation]

For questions related to notation (in general).

26 questions
Filter by
Sorted by
Tagged with
70 views

### What does the term $|\mathcal{A}(s)|$ mean in the $\epsilon$-greedy policy?

I've been looking online for a while for a source that explains these computations but I can't find anywhere what does the $|A(s)|$ mean. I guess $A$ is the action set but I'm not sure about that ...
41 views

### What does the notation “for t=T to 1,−1 do” in terms of time steps, in deep recurrent q network?

In looking at an algorithm in the paper Learning to Communicate with Deep Multi-Agent Reinforcement Learning. Here is the full algorithm: What does the notation ...
40 views

### What do the notations $\sim$ and $\Delta (A)$ mean in the paper “Fairness Through Awareness”?

In this paper Fairness Through Awareness, the notation $\mathbb{E}_{x \sim V} \mathbb{E}_{a \sim \mu_x} L(x,a)$ is being used (page 5 top line), where $V$ denotes the set of individuals (so I guess ...
162 views

### Why are the value functions sometimes written with capital letters and other times with lower-case letters?

Why are the state-value and action-value functions are sometimes written in small letters and other times in capitals? For instance, why in the Q-learning algorithm (page 131 of Barto and Sutton's ...
86 views

### What does the notation $\partial \theta_{\pi}$ mean in this actor-critic update rule?

One of the steps in the actor-critic algorithm is $$\partial \theta_{\pi} \gets \partial \theta_{\pi} + \nabla_{\theta}\log\pi_{\theta} (a_i | s_i) (R - V_{\theta}(s_i))$$ For me, $\theta$ are just ...
43 views

### What does equation in the “related work” section of the GAN paper mean?

I was going through the paper on GAN by Ian Goodfellow. Under the related work section, there is an equation. I cannot decipher the equation. Can anyone help me understand the meaning of the equation? ...
35 views

### What does the notation ${s'\sim T(s,a,\cdot)}$ mean?

I have been seeing notations on Expectations with their respective subscripts such as $E_{s_0 \sim D}[V^\pi (s_0)] = \Sigma_{t=0}^\infty[\gamma^t\phi(s_t)]$. This equation is taken from https://ai....
36 views

### What is the purpose of the arrow $\leftarrow$ in this formula?

What is the purpose of the arrow $\leftarrow$ in the formula below? $$V(S_t) \leftarrow V(S_t) + \alpha \left[ G_t - V(S_t) \right]$$ I presume it's not the same as 'equals'.
38 views

### Why is exp used in encoder of VAE instead of using the value of standard deviation alone?

There's one VAE example here: https://towardsdatascience.com/teaching-a-variational-autoencoder-vae-to-draw-mnist-characters-978675c95776. And the source code of encoder can be found at the ...
38 views

### How to understand the average l2 loss?

In the snippet below, the highlighted part is the average norm, but since $1/|p_i|$ is outside the summation, it is very confusing to understand. is $|p_i|$ l2-norm(as per wolfram) or l1-norm or ...
85 views

### What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$?

What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$? I think $|x|$ is the magnitude of $x$.
59 views

### What does the notation sup dist mean in distributional RL?

I'm trying to understand distributional RL, based on this article. In one of the equations, there is a symbol $\operatorname{sup dist}$. \begin{align} \operatorname{sup dist}_{s, a} (R(s, a) + \...
58 views

### What do the subscripts mean in $N_{t,n,\sigma,L}$?

A neural network can apparently be denoted as $N_{t,n,\sigma,L}$. What do these subscripts $t, n, \sigma$ and $L$ mean? Could you link me to a paper, article or webpage with an explanation for this?
51 views

46 views

### Being confused of distribution notations in Deep Learning book

In chapter 5 of Deep Learning book of Ian Goodfellow, some notations in the loss function as below make me really confused. I tried to understand $x,y \sim p_{data}$ means a sample $(x, y)$ sampled ...
329 views

### How is the policy gradient calculated in REINFORCE?

Reading Sutton and Barto, I see the following in describing policy gradients: How is the gradient calculated with respect to an action (taken at time t)? I've read implementations of the algorithm, ...
104 views

### What does the formula $1-\sum_i(e_i-a_i)^2$ mean in this NEAT Python API?

I have looked at the documentation for the NEAT Python API found here, but it shows calculus like this: The error for each genome is $1-\sum_i(e_i-a_i)^2$ I haven't learned calculus at the moment....
551 views

### Understanding notation of Goodfellow's GAN objective function

What is the meaning of $V(D,G)$? How do we get these expectation parts? I was trying to understand it following this article: Understanding Generative Adversarial Networks (D.Seita), but, after many ...
50 views

### What is the use of the $\epsilon$ term in this back-propagation equation?

I am currently looking at different documents to understand back-propagation, mainly at this document. Now, at page 3, there is the $\epsilon$ symbol involved: While I understand the main part of the ...