New answers tagged

1 vote

Can we solve the environment with only the linear and angular position through Q-Learning?

This isn't possible with a basic tabular approach, or with any agent that has no internal memory. The velocities are a necessary part of the state. Without them, the state is only partially observable,...
Neil Slater's user avatar
  • 30.3k
1 vote
Accepted

Methods of constructing input and ouput vectors in Reinforcement Learning with approximation function learning?

If you build a function like $Q(s,a)$ using DQN, you have the problem that given 100 actions, you'll need 100 forward pass of your network Now, since neural networks can handle multiple outputs, we ...
Alberto's user avatar
  • 1,358
1 vote

Why are these two implementations of the $\epsilon$-greedy policy different?

The two implementations you posted are different, but they do represent the same $\epsilon$-greedy policy. The first function returns an array A which contains the ...
Neil Slater's user avatar
  • 30.3k
3 votes

What are the similarities between Q-learning and Value Iteration?

Q learning is very similar to value iteration. They are based on the same principles. A key similarity is that both assume a greedy action choice on the bootstrap next state value. The big difference ...
Neil Slater's user avatar
  • 30.3k
1 vote

OpenAI Gym implementation of the delayed rewards

I think what you have here (with an important caveat, which I will get to later) is a common misunderstanding about how rewards should be structured for a reinforcement learning (RL) problem. It is ...
Neil Slater's user avatar
  • 30.3k
0 votes

Why is my agent stuck on the same action in my Twin Delayed Deep Deterministic Policy Gradient (TD3) program?

The primary issue I was having was that I wasn't normalizing the input data before sending it through the system. I can confidently say that it is working now.
CloudZero's user avatar
1 vote

What happens when the probability of either one of the policies is 0 in Importance Sampling?

Very simply, one of the requirements of off-policy RL to converge, is that the behavioral policy $b$ has at least the same support of the target policy $\pi$, thus: $$ \forall s \in S \forall a \in A \...
Alberto's user avatar
  • 1,358
2 votes

What happens when the probability of either one of the policies is 0 in Importance Sampling?

Background: Importance sampling is used in many off-policy RL algorithms when the data is generated with one policy, yet it is being used to update another policy. The policy generating the data is ...
DeepQZero's user avatar
  • 1,292
0 votes

Why do we discount the state distribution?

If you have three possible next states from the current state, by adding the discount factor you are introducing a fourth state. It can be a terminal state or some other state that is hidden. The ...
GEORGE THOMAS's user avatar
4 votes

What is the relation between Dynamic Programming and Reinforcement Learning?

Dynamic programming is an algorithm paradigm (i.e. a way to design algorithms) that can be applied to many problem domains, not just Markov decision processes (MDPs), as long as they satisfy certain ...
nbro's user avatar
  • 39.6k
0 votes

Why clip the PPO objective on only one side?

Quote from https://huggingface.co/learn/deep-rl-course/unit8/visualize : We update our policy only if: Our ratio is in the range Our ratio is outside the range, but the advantage leads to getting ...
Serhiy's user avatar
  • 101
0 votes

How to properly optimize shared network between actor and critic?

A common practice involves using a shared encoder, which is updated based solely on critic loss, as implemented in DrQv2.
XiaoBanni's user avatar
3 votes
Accepted

in simple words, what is the Q-learning algortimn steps?

This is taken from Sutton & Barto's RL book ...
A_Arnold's user avatar
  • 239
2 votes
Accepted

Could someone give a very simple example of Q-learning in a very small environment?

one great library is called openAI gym which has a bunch of toy problems. One of these is CliffWalking based on Sutton and is a 3x12. This article steps through it pretty nicely and I am sure you can ...
A_Arnold's user avatar
  • 239
1 vote

RL agent for autonomous vehicle is able to follow the road but can't avoid crashing at all (Highway-Env / Racetrack Env.)

There are a few things I think worth looking into: Is avoiding the other car actually possible in the environment? You may inadvertently be giving the agent a choice of crashing in lane, or going off ...
Neil Slater's user avatar
  • 30.3k

Top 50 recent answers are included