New answers tagged

0

It is not clear form your question, how you use your replay buffer. Basically, you have store all states/reward tuples and train your agent on the entire buffer. Moreover, you should give the agent time to explore (all) states of the world. But if you want to speed up training, you can try to implement importance sampling


1

I see some issues in your code of the environment. Firstly, and probably most importantly, you should not be incrementing the reward. In your code, every time the agent hits $t=475$ for example, the reward given by the environment increases by 1. So if the agent oscillates between $t=450$ and $t=475$, at each timestep the environment gives a greater and ...


1

It is not 100% clear, but this seems like an instance of catastrophic forgetting. This is something that often impacts reinforcement learning. I have answered a very similar question on Data Science stack exchange, and reproduce the same answer here. This is called "catastrophic forgetting" and can be a serious problem in many RL scenarios. If you ...


4

Q-learning is said to be "model-free". Given the two examples above, is it because neither the lake's topology nor that of the mountain are changed by the actions taken? No. That's not why Q-learning is model-free. Q-learning assumes that the underlying environment (FrozenLake or MountainCar, for example) can be modelled as a Markov decision ...


1

A reinforcement learning algorithm is considered model based if it uses estimates of the environments dynamics to help learn. For instance, in the Tabular Dyna-Q algorithm, every time you visit a state action tuple you store in a look-up table the reward received and the next state transitioned to, and after every execution of an action you loop $n$ times to ...


Top 50 recent answers are included