Questions tagged [notation]
For questions related to notation (in general).
67
questions
0
votes
1
answer
38
views
What is the equation for $\pi_*$ in terms of $q_*(s,a)$?
I am trying to solve the following exercise from Sutton and Barto:
Sutton and Barto Exercise 3.27 Give an equation for $\pi_*$ in terms of $q_*(s,a)$
However, I am struggling to do so. I know that $\...
- 81
0
votes
1
answer
43
views
What do we mean by the notation $\mathbf{x}_{p} \in \mathbb{R}^{N \times\left(P^{2} \cdot C\right)}$?
I was going through this VIT paper, what will it look like in torch , if we are trying to write this expression.
- 101
0
votes
1
answer
40
views
What is the meaning of $p_{\text {data }}(y)$ in the CycleGAN?
In the original CycleGAN paper, on the second page, there is a sentence that I didn't quite understand
In theory, this objective can induce an output distribution over $\hat{y}$ that matches the ...
- 19
1
vote
1
answer
122
views
What are the steps to derive the original GAN loss function from the generalized version?
I am trying to understand how the loss function from the original GAN paper
$$\min_{G} \max_{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[\log D(\boldsymbol{x})]+\...
- 13
0
votes
1
answer
33
views
Does $R_{s}=E[R_{t}|S_{t}=s]$ indicate the reward we might expect on getting on average moving from any other state to $s$?
I'm trying to understand correctly what each "variable" in RL is and I'm not sure about $R_{s}$ the reward function. I used to think that it's the reward we may expect on average after ...
- 509
0
votes
1
answer
49
views
Why $ t=τ+n-1$ instead of $t=τ+n$ in n-step TD?
If $\tau$ is the time, whose state’s estimate is being updated, and $t$ is the current time, then, in n-step TD method, we have $t=\tau+n$ (because we have to wait n-steps, before we can update). ...
- 194
2
votes
1
answer
52
views
Why do we have $t$ as subscript in $V$ instead of $t+1$ in the expression of $G_{t:t+1}$?
In one-step TD updates, the target is the first reward plus the discounted estimated value of the next state, which we call the one-step return (page 143 of Sutton & Barto):
$$
G_{t:t+1} \...
- 194
1
vote
1
answer
108
views
What do the square brackets $[ ]$ and $\mid$ mean in $[G_t \mid S_t=s]$?
Here is the formula of state-value function in Reinforcement Learning.
What do the square brackets $[ ]$ and $\mid$ mean in $[G_t \mid S_t=s]$? Why use square brackets? Why use $\mid$?
Why do ...
- 213
2
votes
1
answer
257
views
Where are the parentheses in the Bellman update rule?
I'm not having a lot of intuition about the equation. I have this Bellman update rule:
$$v_{\pi}(s) =\sum_a \pi(a|s)\sum_{s',r} p(s',r|s,a)[r+ \gamma v_{k}(s')]$$
But where are the parenthesis? Is the ...
- 155
1
vote
1
answer
61
views
What does the complexity equation constitute exactly in “Why Should I Trust You?” LIME paper
I've recently been reading this paper on LIME, an algorithm to interpret ANY machine learning model. I encountered this equation (in red) on page 4 and have just been having a hard time deciphering ...
1
vote
0
answers
40
views
Why do we use $q_{\phi}(z \mid x^{(i)})$ in the objective function of amortized variational inference, while sometimes we use $q(z)$?
In page 21 here, it states:
General Idea of Amortization: if same inference problem needs to be solved many times, can we parameterize a neural network to solve it?
Our case: for all $x^{(i)}$ we ...
- 203
2
votes
3
answers
141
views
Are the authors of the VAE paper writing the PDFs as a function of the random variables?
Usually, I see the conventions:
discrete random variable is denoted as $X$,
the pmf is written as $P(X=x)$ or $p(X=x)$ or $p_{X}(x)$ or $p(x)$, where $x$ is an instance of $X$
a continuous random ...
- 203
0
votes
1
answer
32
views
Which is more popular/common way of representing a gradient in AI community: as a row or column vector?
Consider the following remark about writing gradients from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.
Remark (...
- 3,251
1
vote
1
answer
75
views
What is the name of this letter $\mathcal{J}$?
What is the name of this letter $\mathcal{J}$ in the following deep learning equation? And what alphabet it is from?
$$\mathcal{J} = \frac{1}{m} \sum_{i=1}^m \mathcal{L}^{(i)}$$
- 115
1
vote
1
answer
155
views
What is a filter in the context of graph convolutional networks?
In Section 2.1 of the research paper titled Semi-Supervised Classification with Graph Convolutional Networks by Thomas N. Kipf et al.,
Spectral convolution on graphs defined as
The multiplication of ...
- 125
0
votes
1
answer
62
views
In this example of fuzzy c-means, what is the difference between "sigma" and "center" for the clusters?
In this example, what exactly do "Cluster" and "Sigma" mean? (They chose random coordinates for the three centroids of the groups)
Centers: Cluster centers, returned as a ...
- 129
0
votes
1
answer
79
views
Why using negative integers (as dimensions?) in tensor shapes rather than natural numbers?
Consider the following paragraph from A.1 MULTI-MNIST AND CLEVR of A IMPLEMENTATION DETAILS from the research paper titled ...
- 3,251
0
votes
1
answer
549
views
What, exactly, do mlp(64,64) and mlp(64,128,1024) mean in PointNet, and how many input neurons does 1 (x,y,z) point have?
I couldn't find out how to interpret the multilayer perceptron notation given in PointNet. Specifically, I am looking to find out what the numbers inside the parentheses of ...
- 31
0
votes
0
answers
44
views
Is there any difference between 'input' and 'conditional input' in the case of neural networks?
In the research paper titled Conditional Generative Adversarial Nets by Mehdi Mirza and Simon Osindero, there is a notion of conditioning a neural network on a class label.
It is mentioned in the ...
- 3,251
0
votes
1
answer
60
views
Does generator in conditonal GAN obey probability laws?
In probability, we have two types of probability functions: unconditional probability $p(x)$ and conditional probability $p(x | y)$. Both are fundamentally different and the latter can be obtained by ...
- 3,251
1
vote
1
answer
187
views
Is it abuse of notation to use tilde operator in this context?
The following is a way to use tilde (∼) in context of random variables or random vectors.
In statistics, the tilde is frequently used to mean "has the
distribution (of)," for instance, $X∼N(...
- 3,251
-1
votes
1
answer
148
views
Why is noise vector represented by letter $z$? [closed]
Most of the notations in Artificial Intelligence are borrowed from the mathematics.
$x$ stands for input (vector), $y$ stands for output (vector) etc., and the list is long.
But, I am not sure whether ...
- 3,251
3
votes
1
answer
281
views
Is the Bandit Problem an MDP?
I've read Sutton and Barto's introductory RL book. They define a policy as a mapping from states to probabilities of selecting each possible action. If the agent is following policy $\pi$ at time $t$, ...
- 203
0
votes
0
answers
17
views
To denote a training example should I use row vector or column vector?
This code accesses the first 3 examples in the iris data set,
from sklearn.datasets import load_iris
iris = load_iris()
print(iris.data[:3])
and gives
...
- 201
2
votes
2
answers
85
views
What is the correct notation for an operation that applies to each element of an array independently?
I am looking for the standard notation to define element-wise / Hadamard-style functions, if there is one.
That is to say, if the operator I am looking for were represented by a hexagon ⬡, I could use ...
- 83
1
vote
2
answers
131
views
In variational autoencoders, what does p(x|z) mean?
If $x \sim \mathcal{N}(\mu,\,\sigma^{2})$, then it is a continuous variable, and therefore $P(x) = 0$ for any x. One can only consider things like $P(x<X)$ to get a probability greater than 0.
So ...
- 139
1
vote
1
answer
53
views
Does a trajectory in reinforcement learning contain the last action?
From what I learn from CS285 and OpenAI's spinning up, a trajectory in RL is a sequence of state-action pairs:
$$\tau = \{s_0, a_0, ..., s_t, a_t\}$$
And the resulting trajectory probability is:
$$ P(\...
- 11
1
vote
1
answer
97
views
Why is the behaviour policy denoted by $\beta$ and the exploration policy by $ \mu'$ in the DDPG paper?
I am learning about the deep deterministic policy gradient (DDPG) (Lillicrap et al, 2016) and got confused about the notation of the behavior policy.
Lillicrap et al. denote the policy gradient by
$$\...
- 35
2
votes
1
answer
77
views
What do the variables in the cross-correlation formula mean?
I understand what cross-correlation does given a kernel and an input image, but the formula confuses me a little. Given here in Goodfellow's Deep Learning (page 329), I can't quite understand what $m$ ...
2
votes
1
answer
86
views
What is $ \nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?
I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML. I came across this blog: https://lilianweng.github.io/lil-log/2018/11/30/meta-...
- 23
3
votes
2
answers
83
views
What does the parameter $y$ stand for in function $g(y,\mu,\sigma)$ related to REINFORCE algorithm?
I am wondering what the parameter $y$ in the function $g(y,\mu,\sigma)=\frac{1}{(2\pi)^{1/2}\sigma}e^{-(y-\mu)^{2/2\sigma^2}}$ stands for in Section 6 (page 14) of the paper introducing the REINFORCE ...
- 715
2
votes
1
answer
95
views
In AlphaZero, do we need to store the data of terminal states?
I have a question about the training data used during the update/back-propagation step of the neural network in AlphaZero.
From the paper:
The data for each time-step $t$ is stored as ($s_t, \pi_t, ...
- 117
0
votes
1
answer
197
views
What is the meaning of these equations in Noise2Noise paper?
I am trying to understand what is meant by following equations in the Noise2Noise paper by Nvidia.
What is meant by the equation in this image? What is $\mathbb{E}_y\{y\}$? And how should I try to ...
- 31
1
vote
1
answer
167
views
In the definition of the state-action value function, what is the random variable we take the expectation of?
I know that
$$\mathbb{E}[g(X) \mid A] = \sum\limits_{x} g(x) p_{X \mid A}(x)$$
for any random variable $X$.
Now, consider the following expression.
$$\mathbb{E}_{\pi} \left[ \sum \limits_{k=0}^{\infty}...
- 3,251
0
votes
1
answer
103
views
Do the rows of the design matrix refer to the observations or predictors?
I attempt to understand the formulation of dictionary learning for this paper:
Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution
Multimodal Task-Driven ...
- 103
5
votes
1
answer
358
views
What does the notation $\mathcal{N}(z; \mu, \sigma)$ stand for in statistics?
I know that the notation $\mathcal{N}(\mu, \sigma)$ stands for a normal distribution.
But I'm reading the book "An Introduction to Variational Autoencoders" and in it, there is this notation:...
- 153
4
votes
2
answers
115
views
Why do we use $X_{I_t,t}$ and $v_{I_t}$ to denote the reward received and the at time step $t$ and the distribution of the chosen arm $I_t$?
I'm doing some introductory research on classical (stochastic) MABs. However, I'm a little confused about the common notation (e.g. in the popular paper of Auer (2002) or Bubeck and Cesa-Bianchi (2012)...
- 41
4
votes
1
answer
124
views
What does the term $|\mathcal{A}(s)|$ mean in the $\epsilon$-greedy policy?
I've been looking online for a while for a source that explains these computations but I can't find anywhere what does the $|A(s)|$ mean. I guess $A$ is the action set but I'm not sure about that ...
- 95
0
votes
1
answer
77
views
What does the notation "for t=T to 1,−1 do" in terms of time steps, in deep recurrent q network?
In looking at an algorithm in the paper Learning to Communicate with Deep Multi-Agent Reinforcement Learning.
Here is the full algorithm:
What does the notation ...
- 43
1
vote
1
answer
54
views
What do the notations $\sim$ and $\Delta (A) $ mean in the paper "Fairness Through Awareness"?
In this paper Fairness Through Awareness, the notation $\mathbb{E}_{x \sim V} \mathbb{E}_{a \sim \mu_x} L(x,a)$ is being used (page 5 top line), where $V$ denotes the set of individuals (so I guess ...
- 175
4
votes
2
answers
363
views
Why are the value functions sometimes written with capital letters and other times with lower-case letters?
Why are the state-value and action-value functions are sometimes written in small letters and other times in capitals? For instance, why in the Q-learning algorithm (page 131 of Barto and Sutton's ...
- 223
1
vote
1
answer
215
views
What does the notation $\partial \theta_{\pi}$ mean in this actor-critic update rule?
One of the steps in the actor-critic algorithm is $$\partial \theta_{\pi} \gets \partial \theta_{\pi} + \nabla_{\theta}\log\pi_{\theta} (a_i | s_i) (R - V_{\theta}(s_i))$$
For me, $\theta$ are just ...
- 161
1
vote
1
answer
78
views
What does equation in the "related work" section of the GAN paper mean?
I was going through the paper on GAN by Ian Goodfellow. Under the related work section, there is an equation. I cannot decipher the equation. Can anyone help me understand the meaning of the equation? ...
- 384
2
votes
1
answer
45
views
What does the notation ${s'\sim T(s,a,\cdot)}$ mean?
I have been seeing notations on Expectations with their respective subscripts such as $E_{s_0 \sim D}[V^\pi (s_0)] = \Sigma_{t=0}^\infty[\gamma^t\phi(s_t)]$. This equation is taken from https://ai....
- 1,241
2
votes
0
answers
55
views
How is the gradient with respect to weights derived in batch normalization?
At the bottom of page 2 of the paper L2 Regularization versus Batch and Weight Normalization, the equation for the gradient of the output with respect to the weights is given as:
$$
\triangledown y_{...
- 21
1
vote
1
answer
40
views
What is the purpose of the arrow $\leftarrow$ in this formula?
What is the purpose of the arrow $\leftarrow$ in the formula below?
$$V(S_t) \leftarrow V(S_t) + \alpha \left[ G_t - V(S_t) \right]$$
I presume it's not the same as 'equals'.
- 583
1
vote
1
answer
222
views
Why is exp used in encoder of VAE instead of using the value of standard deviation alone?
There's one VAE example here:
https://towardsdatascience.com/teaching-a-variational-autoencoder-vae-to-draw-mnist-characters-978675c95776.
And the source code of encoder can be found at the ...
- 1,223
0
votes
1
answer
111
views
How to understand the average l2 loss?
In the snippet below, the highlighted part is the average norm, but since $1/|p_i|$ is outside the summation, it is very confusing to understand.
is $|p_i|$ l2-norm(as per wolfram) or l1-norm or ...
- 317
3
votes
1
answer
97
views
What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$?
What is the difference between the notations $\|x\|_1, \|x\|_2$ and $|x|$? I think $|x|$ is the magnitude of $x$.
- 317
3
votes
1
answer
92
views
What does the notation sup dist mean in distributional RL?
I'm trying to understand distributional RL, based on this article. In one of the equations, there is a symbol $\operatorname{sup dist}$.
\begin{align}
\operatorname{sup dist}_{s, a} (R(s, a) + \...
- 43