I'd like to measure the difference between 2 grid-worlds to determine the generalization capacity of my agent using tabular Q-learning.
Example (OpenAI Frozen Lake) :
SFFF
FHFH
FFFH
HFFG
and :
SFFG
FHFH
FFFH
HFFH
are not very different but the tabular policy that I found on the first environement will completely fail on the new environment.
A correct distance should to measure the policy on the first environment and compute the norm between this one and the optimal policy (not found with RL) of the new environment. Is it accurate ? I think this is a bit strange because measure the difference between 2 policies is the intrinsic answer.. How can I measure accurately 2 environments, not forgetting the transition matrix of the environment ?
