A2C loss is usually defined as
advantage * (-log(actor_predictions)) * target where
target is a one-hot vector (with some clipping/noise/etc...) with the selected target.
Does this mean that we get larger losses for smaller mistakes?
If for example the agent has predicted $\pi(a|s)=0.9$ but the advantage is negative, this would mean a larger mistake than if the agent predicted that $\pi(a|s)=0.1$, however, putting the numbers in the formula means a larger loss for the
advantage * (-log(actor_predictions)) * target would mean:
$$ -1 * (-log(0.9)) * 1 = log(0.9)=-0.045 $$ $$ -1 * (-log(0.1)) * 1 = log(0.1)=-1 $$
Is my understanding correct?