I just started working on a DRL project from scratch. The state of each episode can be expressed as a state set $S=(S^A, S^B, S^C, S^D)$. Each subset is a feature set of a constituent component of the environment, say, $S^A=(a_1, a_2, a_3)$. To model components, I decided to create four pythonic classes with attributes as features. For example, class A is like:

class A:
    def __init__(self, a1, a2, a3): 
        self.a1 = a1
        self.a2 = a2
        self.a3 = a3

Each class has some methods that help in the interaction with other components (classes) and is used in the environment's step function to generate actions.

I am going to create one instance of class A, 10 instances of class B, 20 instances of class C, and a random number of between 1-10 instances of class D at the beginning of each episode. So, my observation includes 33-42 states of entities.

As far as I know, the observation space is usually encoded in n-dimensional arrays as it is in OpenAI Gym. Is it possible, feasible, or considered good practice to store instances as a sub-state of the whole observation space? In my case, it would be like storing 33-42 instances in an array (list) of 33-42 elements.

Thanks for your time and suggestions!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.