Most introductions to the field of MDPs and Reinforcement learning focus exclusively on domains where space and action variables are integers (and finite). This way we are introduced quickly to Value Iteration, Q-Learning, and the like.
However, the most interesting applications (say, flying helicopters) of RL and MDPs involve continuous state space and action spaces. I'd like to go beyond basic introductions and focus on these cases but I am not sure how to get there.
What are some resources on continuous state and action spaces MDPs for reinforcement learning? And what areas do I need to know or study to understand these cases in-depth?