I'm experimenting with an RL agent that interacts with the following environment. The learning algorithm is double DQN. The neural network represents the function from state to action. It's build with Keras sequential model and has two dense layers. The observation in the environment consists of the following features

  1. the agent's position in an N-dimensional grid,
  2. metrics that represent the hazards of adjacent cells (temperatures, toxicity, radiation, etc.) of adjacent cells, and
  3. some parameters that represent the agent's current characteristics (health, mood, etc.).

There are patterns to the distribution of hazards and the agent's goal is to learn to navigate safely through space.

I am concatenating these features, in the aforementioned order, into a tensor, which is fed into the double DQN.

Does the order in which the features are concatenated to create the state (or observation) matter? Is it possible to group the features in some way to increase the learning speed? If I mix up the features randomly, would that have any effect or it doesn't matter to the agent?

  • $\begingroup$ If states are represented as a 1D vector, the order in which features are concatenated doesn't matter at all. $\endgroup$ – Durathror Nov 20 '20 at 6:14
  • $\begingroup$ Can you explain why this would be the case? Provide some evidence for what you state. Ideally, a link to a research article or reliable resource that shows this. $\endgroup$ – nbro Nov 20 '20 at 12:31
  • $\begingroup$ What do you mean by 1D vector? $\endgroup$ – mark mark Nov 20 '20 at 15:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.