I have been reading literature on reinforcement learning in healthcare. I am slightly confused between the policy evaluation for both SARSA and Q-learning.
To my knowledge, I believe that SARSA is used for policy evaluation, to find the Q values of following an already existing policy. This is usually the clinician's policy.
Q - learning on the other hand seeks to find another policy, different from the clinician's such that the policy learned at different states always maximise the Q - values. This leads to a better treatment policy.
Suppose the Q values are learned from both policies, if the Q values for Q - learning are higher than those of SARSA's, can we say that the policy learned from Q - learning is better than that of the clinician's ?
EDIT
From readings I have found out that computing the state - value function is usually used to compare how good policies are. I believe that new data has to be generated to apply the policy learned from Q - learning and compute the state - value function for following this learnt policy from Q - learning.
Why can't the Q values learnt from SARSA and Q - learning be used as comparison instead ? Also, for model free approaches (eg. continuous state space), how is policy evaluation usually carried out ?
