Q-Learning for continuous state space

Reinforcement learning algorithms (e.g Q-Learning) can be applied to both discrete and continuous spaces. If you understand how it works in discrete mode then you can easily move to continuous mode. That's why in the literature all the introductory material focus on discrete mode as it's easier to model (table, grid ...)

Supposing you have a discrete number of actions, the only difference in a continuous space is that you will be modelling the state each X amount of time (X being a number you can choose depending on your use case). So basically you end up with a discrete space, but probably with an infinite number of states. You apply then the same approach you learned for discrete mode.

Let's take the example of self-driving cars, at each X ms (e.g X=1) you'll be computing the state of the car which are your input features (e.g direction, orientation, rotation, distance to the pavement, relative position on the lane...) and take a decision of the action to take as in discrete mode. The approach is the same in other use cases like playing games, walking robot...

Note (continuous action space):

If you have continuous actions then in almost all use cases the best approach is to discretise your actions. I can't think of an example where discretising your actions will lead to a considerable deficiency.

Q-Learning for continuous state space

Reinforcement learning algorithms (e.g Q-Learning) can be applied to both discrete and continuous spaces. If you understand how it works in discrete mode, then you can easily move to continuous mode. That's why in the literature all the introductory material focuses on discrete mode, as it's easier to model (table, grid, etc.)

Supposing you have a discrete number of actions, the only difference in a continuous space is that you will be modeling the state each $X$ amount of time ($X$ being a number you can choose depending on your use case). So, basically, you end up with a discrete space, but probably with an infinite number of states. You apply then the same approach you learned for discrete mode.

Let's take the example of self-driving cars, at each $X$ms (e.g $X=1$), you'll be computing the state of the car which are your input features (e.g direction, orientation, rotation, distance to the pavement, relative position on the lane, etc.) and take a decision of the action to take as in discrete mode. The approach is the same in other use cases, like playing games, walking robots, and so on.

Note (continuous action space):

If you have continuous actions, then in almost all use cases the best approach is to discretize your actions. I can't think of an example where discretizing your actions will lead to a considerable deficiency.

Reinforcement learning algorithms (e.g Q-Learning) can be applied to both discrete and continuous spaces. If you understand how it works in discrete mode then you can easily move to continuous mode. That's why in the literature all the introductory material focus on discrete mode as it's easier to model (table, grid ...)

In a continuous space the only difference is that you will be modelling the state each X amount of time (X being a number you can choose depending on your use case). So basically you end up with a discrete space, but probably with an infinite number of states. You apply then the same approach you learned for discrete mode.

Let's take the example of self-driving cars, at each X ms (e.g X=1) you'll be computing the state of the car which are your input features (e.g direction, orientation, rotation, distance to the pavement, relative position on the lane...) and take a decision of the action to take as in discrete mode. The approach is the same in other use cases like playing games, walking robot...

Q-Learning for continuous state space

Reinforcement learning algorithms (e.g Q-Learning) can be applied to both discrete and continuous spaces. If you understand how it works in discrete mode then you can easily move to continuous mode. That's why in the literature all the introductory material focus on discrete mode as it's easier to model (table, grid ...)

Supposing you have a discrete number of actions, the only difference in a continuous space is that you will be modelling the state each X amount of time (X being a number you can choose depending on your use case). So basically you end up with a discrete space, but probably with an infinite number of states. You apply then the same approach you learned for discrete mode.

Let's take the example of self-driving cars, at each X ms (e.g X=1) you'll be computing the state of the car which are your input features (e.g direction, orientation, rotation, distance to the pavement, relative position on the lane...) and take a decision of the action to take as in discrete mode. The approach is the same in other use cases like playing games, walking robot...

Note (continuous action space):

If you have continuous actions then in almost all use cases the best approach is to discretise your actions. I can't think of an example where discretising your actions will lead to a considerable deficiency.