New answers tagged reinforcement-learning
1
vote
Can we solve the environment with only the linear and angular position through Q-Learning?
This isn't possible with a basic tabular approach, or with any agent that has no internal memory. The velocities are a necessary part of the state. Without them, the state is only partially observable,...
1
vote
Accepted
Methods of constructing input and ouput vectors in Reinforcement Learning with approximation function learning?
If you build a function like $Q(s,a)$ using DQN, you have the problem that given 100 actions, you'll need 100 forward pass of your network
Now, since neural networks can handle multiple outputs, we ...
1
vote
Why are these two implementations of the $\epsilon$-greedy policy different?
The two implementations you posted are different, but they do represent the same $\epsilon$-greedy policy.
The first function returns an array A which contains the ...
3
votes
What are the similarities between Q-learning and Value Iteration?
Q learning is very similar to value iteration. They are based on the same principles. A key similarity is that both assume a greedy action choice on the bootstrap next state value.
The big difference ...
1
vote
OpenAI Gym implementation of the delayed rewards
I think what you have here (with an important caveat, which I will get to later) is a common misunderstanding about how rewards should be structured for a reinforcement learning (RL) problem. It is ...
0
votes
Why is my agent stuck on the same action in my Twin Delayed Deep Deterministic Policy Gradient (TD3) program?
The primary issue I was having was that I wasn't normalizing the input data before sending it through the system. I can confidently say that it is working now.
1
vote
What happens when the probability of either one of the policies is 0 in Importance Sampling?
Very simply, one of the requirements of off-policy RL to converge, is that the behavioral policy $b$ has at least the same support of the target policy $\pi$, thus:
$$
\forall s \in S \forall a \in A \...
2
votes
What happens when the probability of either one of the policies is 0 in Importance Sampling?
Background: Importance sampling is used in many off-policy RL algorithms when the data is generated with one policy, yet it is being used to update another policy. The policy generating the data is ...
0
votes
Why do we discount the state distribution?
If you have three possible next states from the current state, by adding the discount factor you are introducing a fourth state. It can be a terminal state or some other state that is hidden. The ...
4
votes
What is the relation between Dynamic Programming and Reinforcement Learning?
Dynamic programming is an algorithm paradigm (i.e. a way to design algorithms) that can be applied to many problem domains, not just Markov decision processes (MDPs), as long as they satisfy certain ...
0
votes
Why clip the PPO objective on only one side?
Quote from https://huggingface.co/learn/deep-rl-course/unit8/visualize :
We update our policy only if:
Our ratio is in the range
Our ratio is outside the range, but the advantage leads to getting ...
0
votes
How to properly optimize shared network between actor and critic?
A common practice involves using a shared encoder, which is updated based solely on critic loss, as implemented in DrQv2.
3
votes
Accepted
in simple words, what is the Q-learning algortimn steps?
This is taken from Sutton & Barto's RL book
...
2
votes
Accepted
Could someone give a very simple example of Q-learning in a very small environment?
one great library is called openAI gym which has a bunch of toy problems. One of these is CliffWalking based on Sutton and is a 3x12. This article steps through it pretty nicely and I am sure you can ...
1
vote
RL agent for autonomous vehicle is able to follow the road but can't avoid crashing at all (Highway-Env / Racetrack Env.)
There are a few things I think worth looking into:
Is avoiding the other car actually possible in the environment? You may inadvertently be giving the agent a choice of crashing in lane, or going off ...
Top 50 recent answers are included
Related Tags
reinforcement-learning × 2380deep-rl × 428
q-learning × 330
dqn × 290
machine-learning × 182
policy-gradients × 173
deep-learning × 165
markov-decision-process × 164
neural-networks × 148
rewards × 112
comparison × 111
actor-critic-methods × 103
value-functions × 97
proximal-policy-optimization × 88
sutton-barto × 86
reward-functions × 83
reference-request × 78
temporal-difference-methods × 76
papers × 75
policies × 67
monte-carlo-methods × 66
off-policy-methods × 66
game-ai × 64
terminology × 63
gym × 61