Questions tagged [notation]

For questions related to notation (in general).

Filter by
Sorted by
Tagged with
4 votes
1 answer
136 views

Notation used in paper on Continuous Time Reinforcement Learning

I am working on implementing the learning shown in this paper: https://homes.cs.washington.edu/~todorov/courses/amath579/reading/Continuous.pdf In the paper, the authors devise a continuous time ...
Derick Diana's user avatar
1 vote
1 answer
71 views

What does the notation $\hat{A}_t\left(s_{0: \infty}, a_{0: \infty}\right)$ appearing in Generalized Advantage Estimation mean?

In general, the advantage function is defined as: $A^\pi\left(s_t, a_t\right):=Q^\pi\left(s_t, a_t\right)-V^\pi\left(s_t\right)$ So far, I understand this formular like this: With the advantage ...
kklaw's user avatar
  • 175
0 votes
0 answers
19 views

Is there a standard nomenclature for model names suffixed by strings such as "Q4_K_S"?

Many models specify details using prefixes such as Q6_K or Q4_K_S. It seems obvious that the ...
Mark Harrison's user avatar
0 votes
1 answer
42 views

What is the difference between an input and observed data in a Bayesian neural network?

I'm new to the Bayesian perspective and would appreciate clarity on this. In a few resources concerning Bayesian deep learning (such as this one), I see this notation: $p(y|x, D) = \int p(y|x, \theta)...
Seo's user avatar
  • 3
1 vote
0 answers
35 views

Meaning of a symbol in trace

I am reading this paper Not All Samples Are Created Equal: Deep Learning with Importance Sampling. In the paper there is a deviation shown below. I can understand everything except the $V$ symbol in ...
onexpeters's user avatar
3 votes
1 answer
125 views

How to interpret the policy notation $\pi_{\theta}(a_{t}|s_{t})$ in Reinforcement Learning?

In the context of Reinforcement Learning, I have seen that the policy $\pi$ (for some algorithms) is nothing but a Neural Network architecture (for example a Feedforward Neural Network). This policy ...
moth123's user avatar
  • 31
1 vote
1 answer
124 views

In the DQN paper, why do we have both $\max_{a'}$ and $\max_{a}$ in the pseudocode?

I was reading this article https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf and in it there is an algorithm of deep q learning with experience replay as follows: On line 12, when the algorithm is ...
Ness's user avatar
  • 206
1 vote
1 answer
41 views

What does this bracket notation $\langle\phi(x),v\rangle$ mean?

I found it at the bottom of page 2 of the paper Intriguing properties of neural networks (2014), in the form of $$\underset{x\in\mathcal{I}}{\mathrm{arg\,max}}\langle\phi(x),v\rangle$$
Nils André's user avatar
1 vote
0 answers
14 views

Where exactly is permutation happening in equation 5 of the paper "Learning with Sets in Multiple Instance Regression Applied to Remote Sensing"?

I am reading the article Learning with Sets in Multiple Instance Regression Applied to Remote Sensing about creating an embedding which is order-invariant to inputs ($m_{l}$). They referred to order-...
Oculu 's user avatar
  • 43
0 votes
1 answer
315 views

What is the equation for $\pi_*$ in terms of $q_*(s,a)$?

I am trying to solve the following exercise from Sutton and Barto: Sutton and Barto Exercise 3.27 Give an equation for $\pi_*$ in terms of $q_*(s,a)$ However, I am struggling to do so. I know that $\...
user's user avatar
  • 145
0 votes
1 answer
59 views

What do we mean by the notation $\mathbf{x}_{p} \in \mathbb{R}^{N \times\left(P^{2} \cdot C\right)}$?

I was going through this VIT paper, what will it look like in torch , if we are trying to write this expression.
TheExorcist's user avatar
0 votes
1 answer
50 views

What is the meaning of $p_{\text {data }}(y)$ in the CycleGAN?

In the original CycleGAN paper, on the second page, there is a sentence that I didn't quite understand In theory, this objective can induce an output distribution over $\hat{y}$ that matches the ...
Lukas Pezzei's user avatar
1 vote
1 answer
365 views

What are the steps to derive the original GAN loss function from the generalized version?

I am trying to understand how the loss function from the original GAN paper $$\min_{G} \max_{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[\log D(\boldsymbol{x})]+\...
Paul's user avatar
  • 13
0 votes
1 answer
54 views

Does $R_{s}=E[R_{t}|S_{t}=s]$ indicate the reward we might expect on getting on average moving from any other state to $s$?

I'm trying to understand correctly what each "variable" in RL is and I'm not sure about $R_{s}$ the reward function. I used to think that it's the reward we may expect on average after ...
Daviiid's user avatar
  • 575
0 votes
1 answer
218 views

Why $ t=τ+n-1$ instead of $t=τ+n$ in n-step TD?

If $\tau$ is the time, whose state’s estimate is being updated, and $t$ is the current time, then, in n-step TD method, we have $t=\tau+n$ (because we have to wait n-steps, before we can update). ...
DSPinfinity's user avatar
2 votes
1 answer
71 views

Why do we have $t$ as subscript in $V$ instead of $t+1$ in the expression of $G_{t:t+1}$?

In one-step TD updates, the target is the first reward plus the discounted estimated value of the next state, which we call the one-step return (page 143 of Sutton & Barto): $$ G_{t:t+1} \...
DSPinfinity's user avatar
1 vote
1 answer
300 views

What do the square brackets $[ ]$ and $\mid$ mean in $[G_t \mid S_t=s]$?

Here is the formula of state-value function in Reinforcement Learning. What do the square brackets $[ ]$ and $\mid$ mean in $[G_t \mid S_t=s]$? Why use square brackets? Why use $\mid$? Why do ...
huang's user avatar
  • 295
2 votes
1 answer
288 views

Where are the parentheses in the Bellman update rule?

I'm not having a lot of intuition about the equation. I have this Bellman update rule: $$v_{\pi}(s) =\sum_a \pi(a|s)\sum_{s',r} p(s',r|s,a)[r+ \gamma v_{k}(s')]$$ But where are the parenthesis? Is the ...
nammerkage's user avatar
1 vote
1 answer
83 views

What does the complexity equation constitute exactly in “Why Should I Trust You?” LIME paper

I've recently been reading this paper on LIME, an algorithm to interpret ANY machine learning model. I encountered this equation (in red) on page 4 and have just been having a hard time deciphering ...
loose the fools juice's user avatar
1 vote
1 answer
77 views

Why do we use $q_{\phi}(z \mid x^{(i)})$ in the objective function of amortized variational inference, while sometimes we use $q(z)$?

In page 21 here, it states: General Idea of Amortization: if same inference problem needs to be solved many times, can we parameterize a neural network to solve it? Our case: for all $x^{(i)}$ we ...
a12345's user avatar
  • 233
2 votes
3 answers
213 views

Are the authors of the VAE paper writing the PDFs as a function of the random variables?

Usually, I see the conventions: discrete random variable is denoted as $X$, the pmf is written as $P(X=x)$ or $p(X=x)$ or $p_{X}(x)$ or $p(x)$, where $x$ is an instance of $X$ a continuous random ...
a12345's user avatar
  • 233
0 votes
1 answer
43 views

Which is more popular/common way of representing a gradient in AI community: as a row or column vector?

Consider the following remark about writing gradients from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning by Marc Peter Deisenroth et al. Remark (...
hanugm's user avatar
  • 3,850
1 vote
1 answer
150 views

What is the name of this letter $\mathcal{J}$?

What is the name of this letter $\mathcal{J}$ in the following deep learning equation? And what alphabet it is from? $$\mathcal{J} = \frac{1}{m} \sum_{i=1}^m \mathcal{L}^{(i)}$$
Paul Reiners's user avatar
1 vote
1 answer
313 views

What is a filter in the context of graph convolutional networks?

In Section 2.1 of the research paper titled Semi-Supervised Classification with Graph Convolutional Networks by Thomas N. Kipf et al., Spectral convolution on graphs defined as The multiplication of ...
JAEMTO's user avatar
  • 125
0 votes
1 answer
119 views

In this example of fuzzy c-means, what is the difference between "sigma" and "center" for the clusters?

In this example, what exactly do "Cluster" and "Sigma" mean? (They chose random coordinates for the three centroids of the groups) Centers: Cluster centers, returned as a ...
user5520049's user avatar
0 votes
1 answer
313 views

Why using negative integers (as dimensions?) in tensor shapes rather than natural numbers?

Consider the following paragraph from A.1 MULTI-MNIST AND CLEVR of A IMPLEMENTATION DETAILS from the research paper titled ...
hanugm's user avatar
  • 3,850
0 votes
1 answer
2k views

What, exactly, do mlp(64,64) and mlp(64,128,1024) mean in PointNet, and how many input neurons does 1 (x,y,z) point have?

I couldn't find out how to interpret the multilayer perceptron notation given in PointNet. Specifically, I am looking to find out what the numbers inside the parentheses of ...
Justin's user avatar
  • 41
0 votes
0 answers
79 views

Is there any difference between 'input' and 'conditional input' in the case of neural networks?

In the research paper titled Conditional Generative Adversarial Nets by Mehdi Mirza and Simon Osindero, there is a notion of conditioning a neural network on a class label. It is mentioned in the ...
hanugm's user avatar
  • 3,850
0 votes
1 answer
96 views

Does generator in conditonal GAN obey probability laws?

In probability, we have two types of probability functions: unconditional probability $p(x)$ and conditional probability $p(x | y)$. Both are fundamentally different and the latter can be obtained by ...
hanugm's user avatar
  • 3,850
1 vote
1 answer
337 views

Is it abuse of notation to use tilde operator in this context?

The following is a way to use tilde (∼) in context of random variables or random vectors. In statistics, the tilde is frequently used to mean "has the distribution (of)," for instance, $X∼N(...
hanugm's user avatar
  • 3,850
-1 votes
1 answer
163 views

Why is noise vector represented by letter $z$? [closed]

Most of the notations in Artificial Intelligence are borrowed from the mathematics. $x$ stands for input (vector), $y$ stands for output (vector) etc., and the list is long. But, I am not sure whether ...
hanugm's user avatar
  • 3,850
4 votes
1 answer
711 views

Is the Bandit Problem an MDP?

I've read Sutton and Barto's introductory RL book. They define a policy as a mapping from states to probabilities of selecting each possible action. If the agent is following policy $\pi$ at time $t$, ...
Snowball's user avatar
  • 225
0 votes
0 answers
44 views

To denote a training example should I use row vector or column vector?

This code accesses the first 3 examples in the iris data set, from sklearn.datasets import load_iris iris = load_iris() print(iris.data[:3]) and gives ...
JJJohn's user avatar
  • 221
2 votes
2 answers
460 views

What is the correct notation for an operation that applies to each element of an array independently?

I am looking for the standard notation to define element-wise / Hadamard-style functions, if there is one. That is to say, if the operator I am looking for were represented by a hexagon ⬡, I could use ...
user7834's user avatar
1 vote
2 answers
417 views

In variational autoencoders, what does p(x|z) mean?

If $x \sim \mathcal{N}(\mu,\,\sigma^{2})$, then it is a continuous variable, and therefore $P(x) = 0$ for any x. One can only consider things like $P(x<X)$ to get a probability greater than 0. So ...
IttayD's user avatar
  • 219
1 vote
1 answer
107 views

Does a trajectory in reinforcement learning contain the last action?

From what I learn from CS285 and OpenAI's spinning up, a trajectory in RL is a sequence of state-action pairs: $$\tau = \{s_0, a_0, ..., s_t, a_t\}$$ And the resulting trajectory probability is: $$ P(\...
Ezekiel Chen's user avatar
1 vote
1 answer
199 views

Why is the behaviour policy denoted by $\beta$ and the exploration policy by $ \mu'$ in the DDPG paper?

I am learning about the deep deterministic policy gradient (DDPG) (Lillicrap et al, 2016) and got confused about the notation of the behavior policy. Lillicrap et al. denote the policy gradient by $$\...
Manuel's user avatar
  • 45
2 votes
1 answer
114 views

What do the variables in the cross-correlation formula mean?

I understand what cross-correlation does given a kernel and an input image, but the formula confuses me a little. Given here in Goodfellow's Deep Learning (page 329), I can't quite understand what $m$ ...
InvestingScientist's user avatar
2 votes
1 answer
109 views

What is $ \nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?

I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML. I came across this blog: https://lilianweng.github.io/lil-log/2018/11/30/meta-...
Blake Camp's user avatar
3 votes
2 answers
115 views

What does the parameter $y$ stand for in function $g(y,\mu,\sigma)$ related to REINFORCE algorithm?

I am wondering what the parameter $y$ in the function $g(y,\mu,\sigma)=\frac{1}{(2\pi)^{1/2}\sigma}e^{-(y-\mu)^{2/2\sigma^2}}$ stands for in Section 6 (page 14) of the paper introducing the REINFORCE ...
Daniel B.'s user avatar
  • 815
2 votes
1 answer
161 views

In AlphaZero, do we need to store the data of terminal states?

I have a question about the training data used during the update/back-propagation step of the neural network in AlphaZero. From the paper: The data for each time-step $t$ is stored as ($s_t, \pi_t, ...
sb3's user avatar
  • 147
0 votes
1 answer
306 views

What is the meaning of these equations in Noise2Noise paper?

I am trying to understand what is meant by following equations in the Noise2Noise paper by Nvidia. What is meant by the equation in this image? What is $\mathbb{E}_y\{y\}$? And how should I try to ...
Markov's user avatar
  • 41
1 vote
1 answer
402 views

In the definition of the state-action value function, what is the random variable we take the expectation of?

I know that $$\mathbb{E}[g(X) \mid A] = \sum\limits_{x} g(x) p_{X \mid A}(x)$$ for any random variable $X$. Now, consider the following expression. $$\mathbb{E}_{\pi} \left[ \sum \limits_{k=0}^{\infty}...
hanugm's user avatar
  • 3,850
0 votes
1 answer
267 views

Do the rows of the design matrix refer to the observations or predictors?

I attempt to understand the formulation of dictionary learning for this paper: Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution Multimodal Task-Driven ...
Angus's user avatar
  • 103
5 votes
1 answer
1k views

What does the notation $\mathcal{N}(z; \mu, \sigma)$ stand for in statistics?

I know that the notation $\mathcal{N}(\mu, \sigma)$ stands for a normal distribution. But I'm reading the book "An Introduction to Variational Autoencoders" and in it, there is this notation:...
Peyman's user avatar
  • 564
4 votes
2 answers
120 views

Why do we use $X_{I_t,t}$ and $v_{I_t}$ to denote the reward received and the at time step $t$ and the distribution of the chosen arm $I_t$?

I'm doing some introductory research on classical (stochastic) MABs. However, I'm a little confused about the common notation (e.g. in the popular paper of Auer (2002) or Bubeck and Cesa-Bianchi (2012)...
MAB_N00B's user avatar
4 votes
1 answer
155 views

What does the term $|\mathcal{A}(s)|$ mean in the $\epsilon$-greedy policy?

I've been looking online for a while for a source that explains these computations but I can't find anywhere what does the $|A(s)|$ mean. I guess $A$ is the action set but I'm not sure about that ...
Metrician's user avatar
0 votes
1 answer
119 views

What does the notation "for t=T to 1,−1 do" in terms of time steps, in deep recurrent q network?

In looking at an algorithm in the paper Learning to Communicate with Deep Multi-Agent Reinforcement Learning. Here is the full algorithm: What does the notation ...
kikram's user avatar
  • 43
1 vote
1 answer
65 views

What do the notations $\sim$ and $\Delta (A) $ mean in the paper "Fairness Through Awareness"?

In this paper Fairness Through Awareness, the notation $\mathbb{E}_{x \sim V} \mathbb{E}_{a \sim \mu_x} L(x,a)$ is being used (page 5 top line), where $V$ denotes the set of individuals (so I guess ...
MMM's user avatar
  • 185
4 votes
2 answers
611 views

Why are the value functions sometimes written with capital letters and other times with lower-case letters?

Why are the state-value and action-value functions are sometimes written in small letters and other times in capitals? For instance, why in the Q-learning algorithm (page 131 of Barto and Sutton's ...
d56's user avatar
  • 233