1
$\begingroup$

I just started working on a DRL project from scratch. The state of each episode can be expressed as a state set $S=(S^A, S^B, S^C, S^D)$. Each subset is a feature set of a constituent component of the environment, say, $S^A=(a_1, a_2, a_3)$. To model components, I decided to create four pythonic classes with attributes as features. For example, class A is like:

class A:
    def __init__(self, a1, a2, a3): 
        self.a1 = a1
        self.a2 = a2
        self.a3 = a3

Each class has some methods that help in the interaction with other components (classes) and is used in the environment's step function to generate actions.

I am going to create one instance of class A, 10 instances of class B, 20 instances of class C, and a random number of between 1-10 instances of class D at the beginning of each episode. So, my observation includes 33-42 states of entities.

As far as I know, the observation space is usually encoded in n-dimensional arrays as it is in OpenAI Gym. Is it possible, feasible, or considered good practice to store instances as a sub-state of the whole observation space? In my case, it would be like storing 33-42 instances in an array (list) of 33-42 elements.

Thanks for your time and suggestions!

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.