I am learning with the OpenAI gym's cart pole environment.

I want to make the observation states discrete (with small stepsize) and for that purpose, I need to change two of the observations from [$ -\infty, \infty$] to some finite upper and lower limits. (By the way, these states are velocity and pole velocity at the tip).

How can I change these limits in the actual gym's environment? Any other suggestions are also welcome.


I don't recommend changing the rules of the environment.

What you could do:

Perform a method called bucketing i.e. take a value from a continuous state space see which discrete bucket it should go into and then let your agent use the bucket number as the observation.

e.g. Say I do have a continuous state space with one variable in range $[-\infty,\infty]$

The buckets can be as follows:

0). x < -1000

1). -1000 $\le$ x $<$ -500

2). -500 $\le$ x $<$ -100

3). -100 $\le$ x $<$ -50

4). -50 $\le$ x $<$ 0

5). 0 $\le$ x $<$ 50

6). 50 $\le$ x $<$ 100

7). 100 $\le$ x $<$ 500

8). 500 $\le$ x $<$ 1000

9). x > 1000

Therefore in this example scenario there are 9 buckets. Hence, the observations can be in range [0, 9] discretely.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.