New answers tagged reinforcement-learning
1
vote
Can we solve the environment with only the linear and angular position through Q-Learning?
This isn't possible with a basic tabular approach, or with any agent that has no internal memory. The velocities are a necessary part of the state. Without them, the state is only partially observable,...
- 30.3k
1
vote
Accepted
Methods of constructing input and ouput vectors in Reinforcement Learning with approximation function learning?
If you build a function like $Q(s,a)$ using DQN, you have the problem that given 100 actions, you'll need 100 forward pass of your network
Now, since neural networks can handle multiple outputs, we ...
- 1,358
1
vote
Why are these two implementations of the $\epsilon$-greedy policy different?
The two implementations you posted are different, but they do represent the same $\epsilon$-greedy policy.
The first function returns an array A which contains the ...
- 30.3k
3
votes
What are the similarities between Q-learning and Value Iteration?
Q learning is very similar to value iteration. They are based on the same principles. A key similarity is that both assume a greedy action choice on the bootstrap next state value.
The big difference ...
- 30.3k
1
vote
OpenAI Gym implementation of the delayed rewards
I think what you have here (with an important caveat, which I will get to later) is a common misunderstanding about how rewards should be structured for a reinforcement learning (RL) problem. It is ...
- 30.3k
0
votes
Why is my agent stuck on the same action in my Twin Delayed Deep Deterministic Policy Gradient (TD3) program?
The primary issue I was having was that I wasn't normalizing the input data before sending it through the system. I can confidently say that it is working now.
1
vote
What happens when the probability of either one of the policies is 0 in Importance Sampling?
Very simply, one of the requirements of off-policy RL to converge, is that the behavioral policy $b$ has at least the same support of the target policy $\pi$, thus:
$$
\forall s \in S \forall a \in A \...
- 1,358
2
votes
What happens when the probability of either one of the policies is 0 in Importance Sampling?
Background: Importance sampling is used in many off-policy RL algorithms when the data is generated with one policy, yet it is being used to update another policy. The policy generating the data is ...
- 1,292
0
votes
Why do we discount the state distribution?
If you have three possible next states from the current state, by adding the discount factor you are introducing a fourth state. It can be a terminal state or some other state that is hidden. The ...
4
votes
What is the relation between Dynamic Programming and Reinforcement Learning?
Dynamic programming is an algorithm paradigm (i.e. a way to design algorithms) that can be applied to many problem domains, not just Markov decision processes (MDPs), as long as they satisfy certain ...
- 39.6k
0
votes
Why clip the PPO objective on only one side?
Quote from https://huggingface.co/learn/deep-rl-course/unit8/visualize :
We update our policy only if:
Our ratio is in the range
Our ratio is outside the range, but the advantage leads to getting ...
- 101
0
votes
How to properly optimize shared network between actor and critic?
A common practice involves using a shared encoder, which is updated based solely on critic loss, as implemented in DrQv2.
- 51
3
votes
Accepted
in simple words, what is the Q-learning algortimn steps?
This is taken from Sutton & Barto's RL book
...
- 239
2
votes
Accepted
Could someone give a very simple example of Q-learning in a very small environment?
one great library is called openAI gym which has a bunch of toy problems. One of these is CliffWalking based on Sutton and is a 3x12. This article steps through it pretty nicely and I am sure you can ...
- 239
1
vote
RL agent for autonomous vehicle is able to follow the road but can't avoid crashing at all (Highway-Env / Racetrack Env.)
There are a few things I think worth looking into:
Is avoiding the other car actually possible in the environment? You may inadvertently be giving the agent a choice of crashing in lane, or going off ...
- 30.3k
Top 50 recent answers are included