Is there a notion of exploration-exploitation tradeoff in dynamic programming (or model-based RL)?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Given that this is a one-line question, which may be fine, it may be a good idea to explain why you're asking this question or wondering this. $\endgroup$– nbroJul 26, 2021 at 15:13
-
$\begingroup$ Not in dynamic programming no. If you have an unknown model then yes. The issue of RL is basically that you don't know the model. $\endgroup$– FourierFluxJul 26, 2021 at 15:18
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
I think there is an implicit notion of it in dynamic programming; say, if you have to make some sort of search over a subset of a state space and you are deciding whether to use BFS, breath first search, or DFS, depth first search, you are at least implicitly thinking on the best way to explore/exploit the state space.
As for model based RL, yes. There is explicit algorithms that mediate exploration and exploitation. One of them is UCB, uper confidence bound. One of the best examples of a model based reinforcement learning algorithm is AlphaGo. The algorithm uses a variation of UCB to explore the state space.
