When designing an environment, what should one look out for when designing the observation space to make the environment as easy to be learnable for an agent as possible?
E.g. make sure the markov property is fulfilled if possible, but I mean also more specific details like the coding of observation channels: Try avoiding continuous values, try to keep the number of categories for categorical observations small, or try to represent continuous values in smaller/larger intervals or consider discretizing continuous values into ordinal categorical values.
I read a paper presenting recommendations for the design of the action space, regarding several video games RL studies (Action Space Shaping in Deep Reinforcement Learning, 2020). I wondered whether there are recommendations concerning the observation space as well.
