3
$\begingroup$

I'm trying to understand distributional RL, based on this article. In one of the equations, there is a symbol $\operatorname{sup dist}$.

\begin{align} \operatorname{sup dist}_{s, a} (R(s, a) + \gamma Z(s', a^*), Z(s, a)) \\ s' \sim p(\cdot \mid s, a) \end{align}

What does $\operatorname{sup dist}$ mean?

$\endgroup$
3
$\begingroup$

It doesn't seem that it is a "proper" symbol.

I guess that $\sup$ simply refers to the supremum, that is, you want to select actions that maximize the quantity that comes to the right of $\sup$, while $\text{dist}$ is simply a proxy for any possible distance between distributions. For example, you can replace $\text{dist}$ with the Kullback-Leibler divergence or with the mutual information.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.