I found many tutorials and posts on how to solve RL environments with discrete action spaces using the cross entropy method (e.g., in this blog post for the OpenAI Gym frozen lake environment).
However now I have built my first custom environment, which simulates a car driving on a road with leading and following vehicles. I want to control the acceleration of my vehicle without crashing into anyone. The state consists of the velocity and distance to the leading and following vehicles. The observation and action spaces are continuous and not discrete, which is why I cannot implement my training loop like in the examples that use the cross entropy method. That is, because the method relies on modifying each tuple for training <s, a, r> (state, action, reward) so that the probability distribution in a is equal to 1 in one dimension and equal to 0 in all others (meaning, it it very confident in its action, i.e., [0, 1, 0, 0]).
How do I implement the cross entropy method for a continuous action space (in Python and Pytorch) or is that even possible? The answer to this question probably describes what I want to do in a very mathematical form, but I do not understand it.
