1
$\begingroup$

I was reading the paper How to Combine Tree-Search Methods in Reinforcement Learning published in AAAI Conference 2019. It starts with the sentence

Finite-horizon lookahead policies are abundantly used in Reinforcement Learning and demonstrate impressive empirical success.

What is meant by "finite horizon look-ahead"?

$\endgroup$
1
$\begingroup$

Per this paper a look ahead policy is a policy that will make decisions based on some 'horizon'. Here horizon means some time steps into the future, and so a finite horizon is simply a finite amount of time steps into the future. For example, as we are typically concerned with maximising returns in RL, a 10-step look ahead policy would choose an action at time $t$ that maximises the (expected) rewards at time $t+1, ... t+10$.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Arent all policies finite horizon lookahead policies as per bellman equations where the discount factor determine how far to look ahead? $\endgroup$ – gfdsal Jul 28 at 19:18
  • 1
    $\begingroup$ No, for instance you can get some bandit algorithms that assume an infinite horizon. Gittins Index is an example. $\endgroup$ – David Ireland Jul 28 at 19:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.