2
$\begingroup$

I am looking for metrics to compare the exploration under different RL Algos/reward functions. I want to somehow quantify how big of a region of the policy space is explored. What are common measures to do so?

Thanks in advance for any help!

$\endgroup$
1
  • 1
    $\begingroup$ Welcome to AI stack exchange! Please put your question in the title of your post. Try to make it as clear as possible from the title what kind of answer you would like to receive. $\endgroup$ Commented May 2, 2023 at 13:04

1 Answer 1

1
$\begingroup$

For 2D (labyrinth-like) tasks the measure is usually percentage of the space that was covered. Another option is distance moved (in any direction).

Once you have an objective measure for exploration, there will also be some algorithm that uses it as a target. Measuring the entropy of actions leads you to epsilon-greedy exploration. Coverage of (state, action) pairs leads to count-based exploration, I guess. And so on.

MAP-Elites and Novelty Search: Task-specific behaviour criteria (BCs) are designed (in addition to the numerical reward/return). Then either create a grid with fixed bins and measure grid coverage, or maintain an archive of known solutions and measure the distance of new solutions to the k nearest neighbours.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .