Search Results
| Search type | Search syntax |
|---|---|
| Tags | [tag] |
| Exact | "words here" |
| Author |
user:1234 user:me (yours) |
| Score |
score:3 (3+) score:0 (none) |
| Answers |
answers:3 (3+) answers:0 (none) isaccepted:yes hasaccepted:no inquestion:1234 |
| Views | views:250 |
| Code | code:"if (foo != bar)" |
| Sections |
title:apples body:"apples oranges" |
| URL | url:"*.example.com" |
| Saves | in:saves |
| Status |
closed:yes duplicate:no migrated:no wiki:no |
| Types |
is:question is:answer |
| Exclude |
-[tag] -apples |
| For more details on advanced search visit our help page | |
Results tagged with q-learning
Search options not deleted
user 1641
For questions related to the Q-learning algorithm, which is a model-free and temporal-difference reinforcement learning algorithm that attempts to approximate the Q function, which is a function that, given a state s and an action a, returns a real number that represents the return (or value) of state s when action a is taken from s. Q-learning was introduced in the PhD thesis "Learning from Delayed Rewards" (1989) by Watkins.
3
votes
Accepted
Will Q-learning converge to the optimal state-action function when the reward periodically c...
No, it will not converge in the general case (maybe it might in extremely convenient special cases, not sure, didn't think hard enough about that...).
Practically everything in Reinforcement Learnin …
- 10.5k
5
votes
Accepted
Why are there two different q-learning formulas?
The first equation (the one from the video) looks very wrong, for a few different reasons:
It doesn't involve state-action values $Q(s, a)$, but only something that looks like "state values" $Q(s)$. …
- 10.5k
7
votes
Accepted
Is the discount not needed in a deterministic environment for Reinforcement Learning?
The motivation for adding the discount factor $\gamma$ is generally, at least initially, based simply in "theoretical convenience". Ideally, we'd like to define the "objective" of an RL agent as maxim …
- 10.5k
3
votes
Accepted
Can exogenous variables be state features in reinforcement learning?
Including exogenous variables in your state representation certainly can be useful, as long as you expect them to be relevant information for determining the action to pick. So, state features are not …
- 10.5k
2
votes
Accepted
Do we have two Q-learning update formulas?
The first one is the update rule that we use in the $Q$-learning algorithm.
The second one is the "definition" of $Q(s, a)$ values, although I would personally write it as follows, with an expectatio …
- 10.5k
7
votes
How does Q-learning work in stochastic environments?
How does Q learning handle this? Is the Q function only used during the training process, where the future states are known? And is the Q function still used afterwards, if that is the case?
The …
- 10.5k
1
vote
Should we feed a greater fraction of terminal states to the value network so that their valu...
If you have enough domain knowledge to be able to reliably, intentionally reach those terminal states often when generating experience, yeah, that could help.
Generally, the assumption in Reinforceme …
- 10.5k
1
vote
How to apply or extend the $Q(\lambda)$ algorithm to semi-MDPs?
If you really just want an SMDP-version of the algorithm, which only needs to be capable of operating on the "high-level" time scale of macro-actions, you can relatively safely take the original pseud …
- 10.5k
1
vote
Questions on the identifiability issue and equations 8 and 9 in the D3QN paper
Yes, you're correct, if Equation 8 is used it will only be possible to get estimates $\leq 0$ out of the term
$$\left( A(s, a; \theta, \alpha) - \max_{a' \in \vert \mathcal{A} \vert} A(s, a'; \theta, …
- 10.5k
9
votes
Is Q-learning a type of model-based RL?
Tabular Q-Learning does not explicitly create a model of the transition function. It does not generate any output that you can afterwards use as a function to predict what the next state s' will be gi …
- 10.5k
14
votes
Accepted
What does the symbol $\mathbb E$ mean in these equations?
That's the Expected Value operator. Intuitively, it gives you the value that you would "expect" ("on average") the expression after it (often in square or other brackets) to have. Typically that expre …
- 10.5k
1
vote
Updating action-value functions in Semi-Markov Decision Process and Reinforcement Learning
Personally, I find the best way to think of SMDPs intuitively by just imagining that you just discretise the time into such small steps (infinitesimally small steps if necessary) that you can treat it …
- 10.5k
4
votes
Accepted
What is a high dimensional state in reinforcement learning?
Usually when people write about having a high-dimensional state space, they are referring to the state space actually used by the algorithm.
Suppose my state is a high dimensional vector of $N$ le …
- 10.5k
3
votes
Accepted
How can my Q-learning agent trained to solve a specific maze generalize to other mazes?
I'm going to assume here that you're using the standard, basic, simple variant of $Q$-learning that can be described as tabular $Q$-learning, where all of your state-action pairs for which you're lear …
- 10.5k
7
votes
Accepted
Does AlphaZero use Q-Learning?
Note: you mentioned in the comments that you are reading the old, pre-print version of the paper describing AlphaZero on arXiv. My answer will be for the "official", peer-reviewed, more recent publica …
- 10.5k