I just started working on a DRL project from scratch. The state of each episode can be expressed as a state set $S=(S^A, S^B, S^C, S^D)$. Each subset is a feature set of a constituent component of the environment, say, $S^A=(a_1, a_2, a_3)$. To model components, I decided to create four pythonic classes with attributes as features. For example,
class A is like:
class A: def __init__(self, a1, a2, a3): self.a1 = a1 self.a2 = a2 self.a3 = a3
Each class has some methods that help in the interaction with other components (classes) and is used in the environment's
step function to generate actions.
I am going to create one instance of
class A, 10 instances of
class B, 20 instances of
class C, and a random number of between 1-10 instances of
class D at the beginning of each episode. So, my observation includes 33-42 states of entities.
As far as I know, the observation space is usually encoded in n-dimensional arrays as it is in OpenAI Gym. Is it possible, feasible, or considered good practice to store instances as a sub-state of the whole observation space? In my case, it would be like storing 33-42 instances in an array (list) of 33-42 elements.
Thanks for your time and suggestions!