Timeline for Is it possible for value-based methods to learn stochastic policies?
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 20, 2020 at 2:05 | history | edited | nbro | CC BY-SA 4.0 |
added 1 character in body; edited tags; edited title
|
| Oct 24, 2019 at 11:53 | vote | accept | Krrrl | ||
| Oct 24, 2019 at 11:38 | answer | added | Neil Slater | timeline score: 2 | |
| Oct 24, 2019 at 11:22 | history | edited | Neil Slater | CC BY-SA 4.0 |
added 278 characters in body
|
| Oct 24, 2019 at 11:21 | comment | added | Neil Slater | There are multiple ways to attempt to solve POMDPs, and important sub-types of POMDP which may result in the optimal policy being stochastic. Many toy examples are "enforced" though by deliberately obfuscating state information that the agent could use and refusing to allow even simple fixes such as giving the agent a memory. These serve as examples where a stochastic policy is the best solution, but only due to restrictions on implementation. | |
| Oct 24, 2019 at 11:17 | comment | added | Krrrl | Absolutely! I'm trying to get a clear picture of the different categories for RL algorithms, and while doing so I started to think about settings where the optimal policy is stochastic(POMDP), and if it is possible to learn this policy for the "traditional" value-based methods. I have not previously heard of POMDP solvers, but looking at it now I get the impression that they are more of a heuristic for non-obervable states. Is that fair to say? Thank you for your questions! They were helpful for me when organizing my thoughts on this. | |
| Oct 24, 2019 at 9:46 | comment | added | Neil Slater | Could you clarify - are you looking for solutions where the optimal policy is stochastic, and to find it accurately? E.g. scissor/paper/stone game or some enforced POMDP where POMDP solvers are not allowed? Strict MDPs with Markov property state features always have a deterministic optimal policy. Also, epsilon-greedy is a stochastic policy and e.g. SARSA will find the optimal epsilon-greedy policy for a given epsilon (just this won't be the optimal policy for the environment). So it would help if you explained what the context is that means you want a stochastic policy. | |
| Oct 24, 2019 at 9:30 | history | asked | Krrrl | CC BY-SA 4.0 |