Timeline for Why isn't my implementation of DQN using TensorFlow on the FrozenWorld environment working?
Current License: CC BY-SA 4.0
26 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 27 at 9:09 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Jun 29 at 8:00 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Mar 1 at 7:08 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Nov 2, 2023 at 6:04 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Jul 5, 2023 at 6:04 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Mar 7, 2023 at 6:03 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Nov 7, 2022 at 5:06 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Jul 10, 2022 at 4:08 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Mar 12, 2022 at 4:02 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Nov 12, 2021 at 4:02 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Jul 15, 2021 at 3:03 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Mar 17, 2021 at 2:06 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Nov 17, 2020 at 2:00 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Jul 20, 2020 at 1:08 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Mar 22, 2020 at 1:01 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Feb 23, 2020 at 21:43 | comment | added | Brett Daley | @HaiNguyen "If your regular Q-learning algorithm does not work, then your DQN has no chance to work either." Technically this isn't true. For example, consider Atari 2600 games like Pong that DQN can play but Q-learning cannot. | |
| Feb 21, 2020 at 19:05 | comment | added | kosa | @HaiNguyen, I will try with this update rule. I make a wide network with a single layer. | |
| Feb 21, 2020 at 11:52 | comment | added | Hai Nguyen |
@kosa If your regular Q learning algorithm does not work, then your DQN has no chance to work either. The neural network in DQN is just a storage device to replace the table for the Q-factors if you run out of memory. My hunch is that your target in test_fun is the real culprit. Currently it's just $Q^{new}$, it should be $(1-\alpha)Q^{old}+\alpha Q^{new}$. Also, frozen lake is a very simple problem (compared to Starcraft or Go), you won't need 2 hidden layers.
|
|
| Feb 21, 2020 at 0:28 | answer | added | Brett Daley | timeline score: 0 | |
| Feb 20, 2020 at 23:11 | history | edited | nbro | CC BY-SA 4.0 |
title made more explicit
|
| Feb 20, 2020 at 22:40 | comment | added | kosa | @NeilSlater I used NN with 2-hidden layers (size = 50)+'relu' activations which outputs 4 values(one for each action). I used Adam with a constant learning rate for minimizing MSE error. I did not use any features to represent the state. I just used the state as the input to the NN. | |
| Feb 20, 2020 at 22:32 | comment | added | kosa | @Brale_ I tried normal Q learning. It did not train. In any case i will also try UCB. | |
| Feb 20, 2020 at 22:11 | comment | added | Brale | Frozen lake is somewhat harder to solve than expected since it's a very stochastic environment. You will have a hard time solving it with DQN. Try solving it with regular Q learning. Epsilon-greedy isn't ideal because you will get stuck in local optima and you won't get out since exploration will diminish. Try using UCB strategy which is generally better than e-greedy for tabular methods. Also, resetting counters completely after certain amount of episodes in UCB helps in this case. | |
| Feb 20, 2020 at 21:51 | comment | added | Neil Slater | In general it is quite hard to do code reviews and bug hunts, so you may not get any response. To improve your chances you should give more context on the site, so your question does not rely on people going off site and getting involved in your project. For instance, what diagnostics have you collected, and what are the results? Have you observed more than "the agent doesn't seem to learn" that is worth sharing? What is your state representation for input to the NN, and what its the rough NN architecture - layer sizes, output activation function, loss function etc | |
| Feb 20, 2020 at 21:10 | review | First posts | |||
| Feb 20, 2020 at 23:01 | |||||
| Feb 20, 2020 at 21:07 | history | asked | kosa | CC BY-SA 4.0 |