We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [notation]

For questions related to notation (in general).

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

How is the general return-based off-policy equation derived?

I'm wondering how is the general return-based off-policy equation in Safe and efficient off-policy reinforcement learning derived $$\mathcal{R} Q(x, a):=Q(x, a)+\mathbb{E}_{\mu}\left[\sum_{t \geq 0} \...
4
votes
1answer
55 views

What do the subscripts mean in $N_{t,n,\sigma,L}$?

A neural network can apparently be denoted as $N_{t,n,\sigma,L}$. What do these subscripts $t, n, \sigma$ and $L$ mean? Could you link me to a paper, article or webpage with an explanation for this?
3
votes
1answer
34 views

Sutton & Barto's notation $V_{t+n}$ in Chapter 7: $n$-step Bootstrapping

Until Chapter 6 of Sutton & Barto's book on Reinforcement Learning, the authors use $V$ for the current estimate of a state value. Equation (6.1), for example, shows: $$ V(S_t) \leftarrow V(S_t) +...
4
votes
1answer
52 views

What is the meaning of the square brackets in ant colony optimization?

I'm studying the paper "Minimizing Total Tardiness on a Single Machine Using Ant Colony Optimization" which has proposed to use Ant colony optimization to SMTWTP. According to this paper: Each ...
2
votes
1answer
21 views

Understanding the equation of the empirical error

The empirical error equation given in the book Understanding Machine Learning: From Theory to Algorithms is My intuition for this equation is: total wrong predictions divided by the total number of ...
5
votes
1answer
276 views

Understanding the equation of TD(0) in the paper “Learning to predict by the methods of temporal differences”

In the paper Learning to predict by the methods of temporal differences (p. 15), the weights in the temporal difference learning are updated as given by the equation $$ \Delta w_t = \alpha \left(P_{t+...
3
votes
1answer
34 views

Being confused of distribution notations in Deep Learning book

In chapter 5 of Deep Learning book of Ian Goodfellow, some notations in the loss function as below make me really confused. I tried to understand $x,y \sim p_{data}$ means a sample $(x, y)$ sampled ...
4
votes
1answer
143 views

How is the policy gradient calculated in REINFORCE?

Reading Sutton and Barto, I see the following in describing policy gradients: How is the gradient calculated with respect to an action (taken at time t)? I've read implementations of the algorithm, ...
2
votes
3answers
91 views

What does the formula $1-\sum_i(e_i-a_i)^2$ mean in this NEAT Python API?

I have looked at the documentation for the NEAT Python API found here, but it shows calculus like this: The error for each genome is $1-\sum_i(e_i-a_i)^2$ I haven't learned calculus at the moment....
5
votes
1answer
188 views

Understanding notation of Goodfellow's GAN objective function

What is the meaning of $V(D,G)$? How do we get these expectation parts? I was trying to understand it following this article: Understanding Generative Adversarial Networks (D.Seita), but, after many ...
2
votes
1answer
45 views

What is the use of the $\epsilon$ term in this back-propagation equation?

I am currently looking at different documents to understand back-propagation, mainly at this document. Now, at page 3, there is the $\epsilon$ symbol involved: While I understand the main part of the ...
2
votes
1answer
102 views

Understanding the notation in the definition of the expected reward

I am new to RL and I am trying to work through the book Reinforcement Learning: An Introduction I (Sutton & Barto, 2018). In chapter 3 on Finite Markov Decision Processes, the authors write the ...
3
votes
2answers
96 views

What does the argmax of the expectation of the log likelihood mean?

What does the following equation mean? What does each part of the formula represent or mean? $$\theta^* = \underset {\theta}{\arg \max} \Bbb E_{x \sim p_{data}} \log {p_{model}(x|\theta) }$$
1
vote
1answer
191 views

What is $I$ in the noise described in the paper “Parameter Space Noise for Exploration”?

In the paper Parameter Space Noise for Exploration, the authors describe the noise that they add to the parameter vector as: $$ \tilde{\theta} = \theta + \mathcal{N}(0, \sigma^2I) $$ is $I$ simply ...