calveeen
  • Member for 1 year, 10 months
  • Last seen this week
3 answers
5 votes
2k views
What is the target Q-value in DQNs?
4 votes

When training a Deep Q network with experienced replay, you accumulate what is known as training experiences $e_t = (s_t, a_t, r_t, s_{t+1})$. You then sample a batch of such experiences and for each ...

View answer
1 answers
1 votes
137 views
Why does Monte Carlo policy evaluation relies on action-value function rather than state-value function?
Accepted answer
3 votes

In Model Based Reinforcement learning, state and state-action values for all states can be calculated based on the bellman equations. The equations are taken from Andrew Ng's Algorithms for Inverse ...

View answer
1 answers
3 votes
50 views
Do all expert trajectories have the same starting state in apprenticeship learning?
Accepted answer
3 votes

All right, I figured it out. trajectories need not have the same starting state because the distribution of $s_0$ is drawn from a distribution D (mentioned in the paper). Had been confused because ...

View answer
2 answers
4 votes
392 views
Formula for expected rewards for state–action–next-state triples as a three-argument function
2 votes

$\frac{p(s', r \mid s, a)}{p(s' \mid s, a)}$ represents the probability of observing reward $r$ in state $s'$, given that state $s'$ is the next state transitioned to. The equation assumes a ...

View answer
1 answers
1 votes
58 views
Question on identifiability in the “Dueling Network Architectures for Deep Reinforcement Learning” paper
Accepted answer
1 votes

Regarding your first question, $$V^{\pi}(s) = \sum_{a \in A}\pi(a|s)Q^{\pi}(s,a)$$ so recovering the value function from Q really depends on what policy $\pi$ you are using. Hence, you can't really ...

View answer
1 answers
1 votes
156 views
Optimal mixed strategy in two player zero sum games
Accepted answer
1 votes

I will answer the first question question based on information I have gathered so far. The probability of each action for the $\textbf{two player zero sum game}$ need not be the same for both players. ...

View answer
1 answers
2 votes
73 views
Why do we minimise the loss between the target Q values and 'local' Q values?
1 votes

The loss function is designed in a way to approximate the bellman optimality for $Q^*(s,a)$. Given an optimal policy $\pi^*$, $Q^*(s,a)$ satisfies the equation $$Q^*(s,a) = r(s) + \gamma max_{a'}\sum_{...

View answer
1 answers
3 votes
125 views
Can recovering a reward function using IRL lead to better policies compared to reward shaping?
1 votes

Inverse Reinforcement Learning (IRL) is a technique that attempts to recover the reward function that the expert is implicitly maximising based on expert demonstrations. When solving reinforcement ...

View answer
2 answers
0 votes
151 views
When we use a neural network to approximate the Q values, is the Q target a single value?
Accepted answer
1 votes

When we use our network to approximate our Q values,is the Q target a single value? Yes, the target Q value is a single value if you are just updating a single training example. The loss function of ...

View answer
1 answers
2 votes
83 views
Why is the hypothesis function $h_{\theta}(x)$ equivalent to $E[y | x; \theta]$ in generalised linear models?
1 votes

In generalised Linear models, each output variable $y_i$ is modelled as a distribution from the exponential family, with the hypothesis function $h_{\theta}(x)$ for a given $\theta$ as the expected ...

View answer
1 answers
2 votes
54 views
Why a single trajectory can be used to update the policy network $\theta$ in A3C?
1 votes

I guess the gradient of the expectation of the Utility function, $\nabla_{\theta}J(\theta)$ in policy gradient methods where $\nabla_{\theta}J(\theta) = E_{\tau \sim p(\tau ; \theta)}[r(\tau)\nabla_{\...

View answer
2 answers
1 votes
80 views
In variational autoencoders, what does p(x|z) mean?
0 votes

In VAE's, we want to model the distribution of images $x$ with some latent variable $z$. Because $x$ is a random variable, You can think of $P(x|z)$ as the distribution of images $x$ conditioned on ...

View answer
3 answers
1 votes
148 views
How does batch normalisation actually work?
Accepted answer
0 votes

I'm not sure how just training the batch normalisation layer, you can get an accuracy of 83%. The batch normalisation layer parameters $\gamma^{(k)}$ and $\beta^{(k)}$, are used to scale and shift the ...

View answer
2 answers
1 votes
147 views
How does the gradient increase the probabilities of the path with a positive reward in policy gradient?
0 votes

The grad log probability of the trajectory parameterised by $\theta$ tells us the direction $\theta$ should move to increase the probability of that trajectory $P(\tau;\theta)$ the most. If the ...

View answer
1 answers
1 votes
45 views
Does the substituted variable/constant have to appear in the unified term?
0 votes

Hey I am currently studying First Order Logic right now and I think I can answer your question. Others please correct me if I am wrong. For the first case, you can generally substitute variables ...

View answer
2 answers
2 votes
140 views
How can I use autoencoders to analyze patterns and classify them?
0 votes

When using an autoencoder, I believe the data u feed in has to be correlated in one way or another. For example, If i want to learn a latent representation of an image of a cat, The training data that ...

View answer