I have PPO agent for discrete action space for LunarLander-v2
env in gym and it works well. However, when i am trying to solve continuous version of the same env - LunarLanderContinuous-v2
it is totally failing. I guess i made some mistakes in converting algorithm to continuous version. So, my steps of changing at algorithm are:
- Change network: return
mu
andvar
of shapen_actions
. I have 2 different last layers for that, formu
i returnTanh
of logits and forvar
i returnSoftplus
of logits. - Choosing action: sampling action from normal distribution with expectation
mu
and variancevar
-torch.distributions.multivariate_normal.MultivariateNormal(torch.squeeze(mu), torch.torch.diag_embed(var))
- For
log
of action probability i am usingdist.log_prob(actions)
With this small changes my algorithm totally doesn't work. Is it right steps to convert algorithm with discrete action space to algorithm with continuous action space? I really confused, because my PPO for discrete action space work very well and with only this changes it is failing. Could you please suggest what i am doing wrong here?
LunarLander
after 50 training loops. Holding space at any time will stop the training.agent.run(1)
will animate the environment and show you how the agent is doing andagent.train(50)
will train for 50 training loops. $\endgroup$