I'm having difficulty picturing how models and policies are represented. Could someone describe how they would look in the context/environment of a game of Tic Tac Toe? For example, "In Tic Tac Toe, the policy would be how the agent decides in which position to place its token"(?) Equally, how model-free and off-policy would look in a TTT environment.

Using the same environment of TTT, could you also give examples of the differences between discrete and continuous Action Space, as well as State Space.

  • $\begingroup$ In TTT, you have a discrete number of states and actions (because you have 9 distinct cells and, in each of them, you have a discrete number of possible actions). You have continuous action space e.g. when your action is a value in a range (e.g. acceleration or velocity). Similarly, for example, in the pendulum example, you have an infinite number of positions where the pendulum can be, so, in theory, that will be a continuous state space. (In practice, in a digital computer, you will only have discrete values anyway.) $\endgroup$ – nbro Feb 3 '20 at 13:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.