I'm currently studying on off-line reinforcement learning (RL) and trying to utilize it for medical data. Because it seemed hard to develop well-performing environment model, I decided to adopt model-free RL algorithms. Then I faced somehow ridiculous issues which cannot be ignored. "How to evaluate this RL model?".
My ideal model will yield the (estimated) best action at the specific state, and there must be a paired real action at that state. We know the reward of the real action, but the reward of the (estimated) best action is totally unknown. All I can describe is the difference between the real actions and best actions. How can I know this RL model outperforms real actions? (according to reward)
Of course I can change to model-based RL algorithm and get predicted rewards from trained environment model, but I'm not sure this 'predicted rewards' are reliable (Anyway, it is also predicted).
Is there any smart method for calculate estimated rewards in off-line RL model? (e.g. mean rewards of real actions -> 0.5, mean rewards of best actions -> 0.8)