2
$\begingroup$

I have been reading literature on reinforcement learning in healthcare. I am slightly confused between the policy evaluation for both SARSA and Q-learning.

To my knowledge, I believe that SARSA is used for policy evaluation, to find the Q values of following an already existing policy. This is usually the clinician's policy.

Q - learning on the other hand seeks to find another policy, different from the clinician's such that the policy learned at different states always maximise the Q - values. This leads to a better treatment policy.

Suppose the Q values are learned from both policies, if the Q values for Q - learning are higher than those of SARSA's, can we say that the policy learned from Q - learning is better than that of the clinician's ?

EDIT

From readings I have found out that computing the state - value function is usually used to compare how good policies are. I believe that new data has to be generated to apply the policy learned from Q - learning and compute the state - value function for following this learnt policy from Q - learning.

Why can't the Q values learnt from SARSA and Q - learning be used as comparison instead ? Also, for model free approaches (eg. continuous state space), how is policy evaluation usually carried out ?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.