I am interested in learning how transition probabilities/mdps are constructed in continuous state and action space model-based learning setting. There is some literature available on this matter, but they do not explicitly construct the model to simulate the environment, through policy gradient.
The closest literature that builds a model that I found is on continuous state space and finite action space, which is slightly different than what I aim to do. Furthermore even in this case the main problem that I face is, transition probability is assumed to give you the probability of obtaning next state, which does not necessarily have to be non-zero for instance when the transition probability is non-atomic transition probability.
I will appreciate if someone atleast points me relevant literature in this direction.