I'm currently writing code for a reward predictor function r=f(s,a) in reinforcement learning, where 's' is the state with 256 dimensions (the embedding dimension after visual input is processed by an encoder) and 'a' is the action with 6 dimensions. I could use a Fully Connected Neural Network (FCNN) for this function.
I'm wondering if it's necessary or beneficial to adjust the dimensions of the state or action inputs given the significant difference in their dimensions? For instance, should I consider directly increasing the dimensionality of the action to match the state dimensionality using a linear layer? Or are there other methods to handle these large dimensionality differences?
I welcome any related discussions, or references to similar concept code or code repository links.