0
$\begingroup$

Consider a neural network function $f:\mathbb{R}\to distribution$. For simplicity, maybe consider that it returns a gaussian distribution.

I want to find $\arg\min_{s\in\mathbb{R}}D_{KL}(f(s),q)$ for some fixed distribution $q$.

Is there an efficient closed-form method to find such $s$? or do I need to run gradient descent with respect to $s$ and it might get stuck into local-optima?



For someone who might want more context: I'm reading a paper about cross-domain transfer in RL.

Cross-Domain Transfer via Semantic Skill Imitation, by Karl Pertsch and et.al... https://arxiv.org/pdf/2212.07407

Here, to map a state in the source domain to a state in the target domain, the author suggests finding a target domain state that minimizes below loss. Here $p_S(k|s^S)$ is a distribution of semantics for the source state $s^S$, and I think can be considered as fixed for each source state. $p_T(k|s^T)$ is a distribution of semantics for the target domain state $s^T$(trained NN function). so minimizing this loss means we consider a target domain state that has the most similar semantics to the source domain state to be the representation of the source domain state in target domain.

$\min_{s^T \in D_T} \left( D_{KL}(p_T(k|s^T), p_S(k|s^S)) + D_{KL}(p_S(k|s^S), p_T(k|s^T)) \right)$

However, as I know, state space is continuous, so I am confused how one can efficiently find such a target domain state that minimizes above KL term.

$\endgroup$
2
  • $\begingroup$ you made no assumption on $q$ thus, it's even very likely that you cannot even compute the KL (thus you can't even do GD) $\endgroup$
    – Alberto
    Commented Apr 25 at 16:15
  • $\begingroup$ @Alberto umm what if we consider q also to be a gaussian distribution? $\endgroup$ Commented Apr 25 at 17:10

1 Answer 1

0
$\begingroup$

Since you made no assumption on what $q$ is, the problem is untractable

If you assume $q$ to be Gaussian, it's easy to prove that the KL distance is minimized when you have $\mu_{f(s)}, \sigma_{f(s)}$ equal to $\mu_q, \sigma_q$

$\endgroup$
2
  • $\begingroup$ but is there a closed-form or clever way to find s that $\mu_{f(s)},\sigma_{f(s)}$ equal to $\mu_q,\sigma_q$? $\endgroup$ Commented Apr 26 at 19:50
  • $\begingroup$ @user3315463 for the question you posted, it looks like you assume to have $q$, this implies that you also have $\mu,\sigma of $q$... if not, please add further details on what's $q$ $\endgroup$
    – Alberto
    Commented Apr 26 at 21:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .