Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
In the following pages (p.84-85) of RL and Optimal Control book by D.Bertsekas, he is talking about base heuristic" and a "rollout algorithm" based on this base heuristic. However, I am very confused because based on Figure 2.4.2, it seems that rollout algorithm and base heuristics are the same things.
As illustrated under your above flowchart figure 2.4.2, base heuristic/policy is just a simple heuristic that is used to help make a quick decision about what action to take in a given state $x_k$ via generating a heuristic Q-factor for every state-control pair ($x_k, u_k$) while satisfying the control system equations shown on your first screenshot. Then the on-line rollout algorithm selects the control with minimal Q-factor which is a specific heuristic search algorithm using the base heuristic (policy) to further simulate the behavior of the control system over a finite time horizon under the (suboptimal) rollout policy.
