1
$\begingroup$

I have retrospective data for a sort of "behaviour policy" which I will use to train a deep q network to learn a target greedy policy. After learning the Q values for this target policy, can we make the conclusion that because the Q value for the target policy, $Q(s,\pi_e(s))$ is higher than the Q values for the behaviour policy, $Q(s,\pi_b(s))$ at all states encountered, where $\pi_e$ is the policy output by deep Q-learning and $\pi_b$ is the behaviour policy, then this target policy has better performance than the behaviour policy?

I know the proper way is to run the policy and do an empirical comparison of some sort. However, that is not possible in my case.

| improve this question | |
$\endgroup$
0
$\begingroup$

No, mainly because these are all stochastic approximations and may not represent the true values.

Almost nothing good can be said about NN approximations to value and Q functions(at least according to a professor I have had).

| improve this answer | |
$\endgroup$
  • $\begingroup$ I believe the first sentence is correct. However, I also think the answer needs to have a few more details. If nothing else, worth clarifying "nothing good can be said" - it could be read as it is bad choice of solution, but actually many novel results have been produced using deep RL that are not possible with any other technique. So perhaps you are referring to the lack of solid theory, such as a practical loss function that is minimised when agent performance is maximised. $\endgroup$ – Neil Slater May 14 at 6:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.