1
$\begingroup$

In the following pages (p.84-85) of RL and Optimal Control book by D.Bertsekas, he is talking about base heuristic" and a "rollout algorithm" based on this base heuristic. However, I am very confused because based on Figure 2.4.2, it seems that rollout algorithm and base heuristics are the same things.

enter image description here

enter image description here

$\endgroup$

1 Answer 1

1
$\begingroup$

As illustrated under your above flowchart figure 2.4.2, base heuristic/policy is just a simple heuristic that is used to help make a quick decision about what action to take in a given state $x_k$ via generating a heuristic Q-factor for every state-control pair ($x_k, u_k$) while satisfying the control system equations shown on your first screenshot. Then the on-line rollout algorithm selects the control with minimal Q-factor which is a specific heuristic search algorithm using the base heuristic (policy) to further simulate the behavior of the control system over a finite time horizon under the (suboptimal) rollout policy.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .