I'm looking for some general advice here before I dive in.

I'm interested in creating a new environment for OpenAI gym to provide some slightly more challenging continuous control problems than the ones currently in their set of Classic Control environments. The intended users of this new environment are just me and members of a meetup group I am in but obviously I want everyone to be able to access, install and potentially modify the same environment.

What's the easiest way to do this?

  • Can I simply import and sub-class the OpenAI gym.Env module similar to the way cartpole.py does.

  • Or do I need to create a whole new PIP package as this article suggests?

Also, before I invest a lot of time on this, has anyone already created a cart-pole system environment where the goal is to stabilize the full state (not just the vertical position of the pendulum)? (I tried googling and couldn't find any variants on the original cart-pole-v1 but I'm suspicious as I can't be the first person to make modified versions of some of these classic control environments).

Thanks, (I realize this question is a bit open-ended) but hoping for some good advice that will point me in the right direction.


This stackoverflow post provides a good answer. It recommends creating the environment as a new package.

Note the updated link to the instructions on OpenAI.

So that's what I did and it's honestly not that much work and then you can import the environment into your script like this:

import gym
import gym_foo
env = gym.make('foo-v0')

This blog post also describes the process in a little bit more detail.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.