For everybody getting here from google, like me: the log might have been replaced in the loss function, but I think it is still there when taking the gradient of both functions (correct me if I am wrong):

So, the log in the PG function was replaced with r(theta) whose derivate is the same as the log of the PG function (appart from the prportionality factor r(theta)).

For everybody getting here from google, like me: the $\log$ might have been replaced in the loss function, but I think it is still there when taking the gradient of both functions (correct me, if I am wrong):

$$\begin{aligned} \nabla_{\theta} L^{P G}(\theta) &=\nabla_{\theta} \hat{E}_{t}\left[\log \pi_{\theta}\left(a_{t} \mid s_{t}\right) \hat{A}_{t}\right] \\ &=\hat{E}_{t}\left[\nabla_{\theta} \log \pi_{\theta}\left(a_{t} \mid s_{t}\right) \hat{A}_{t}\right] \end{aligned}$$

and

$$ \begin{aligned} \nabla_{\theta} L^{I S}(\theta)=& \nabla_{\theta} \hat{E}_{t} \left[\frac{\pi_{\theta}\left(a_{t} \mid s_{t}\right)}{\pi_{\theta_{\text {old}}}\left(a_{t} \mid s_{t}\right)} \hat{A}_{t}\right] \\ &=\hat{E}_{t} \left[\nabla_{\theta} \frac{\pi_{\theta}\left(a_{t} \mid s_{t}\right)}{\pi_{\theta_{\text {old}}}\left(a_{t} \mid s_{t}\right)} \hat{A}_{t}\right] \\ &=\hat{E}_{t} \left[\frac{\pi_{\theta}\left(a_{t} \mid s_{t}\right)}{\pi_{\theta_{\text {old}}}\left(a_{t} \mid s_{t}\right)} \frac{\nabla_{\theta} \pi_{\theta}\left(a_{t} \mid s_{t}\right)}{\pi_{\theta}\left(a_{t} \mid s_{t}\right)} \hat{A}_{t}\right] \\ &=\hat{E}_{t}\left[\frac{\pi_{\theta}\left(a_{t} \mid s_{t}\right)}{\pi_{\theta_{\text {old}}}\left(a_{t} \mid s_{t}\right)} \nabla_{\theta} \log \pi_{\theta}\left(a_{t} \mid s_{t}\right) \hat{A}_{t}\right] \end{aligned} $$

So, the $\pi_{\theta}\left(a_{t} \mid s_{t}\right) $ in the PG function was replaced with $\frac{\pi_{\theta}\left(a_{t} \mid s_{t}\right)}{\pi_{\theta_{\text {old}}}\left(a_{t} \mid s_{t}\right)} $ whose derivate is the same as the log of the PG function (apart from the proportionality factor).