1
$\begingroup$

As the title suggests, I have a doubt about the computation of the $Q_a$ used to update the delta and the $Q_a$ used to select the next action in the exploitation phase, as shown below (source of pseudocode in Figure 8.9).

Watkins' Q(λ) algorithm with linear approximation

In both for loops enumerated in the image with 1 and 2, the $Q_a$ are calculated considering the new state, but while in 1 for each action the active features $F_a$ are calculated, in 2 they aren't. So what are the active features to consider in 2? Are they the same as those in 1? In that case, I could avoid recalculating the $Q_a$ by storing those calculated in 1.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.