1
$\begingroup$

I'm currently writing code for a reward predictor function r=f(s,a) in reinforcement learning, where 's' is the state with 256 dimensions (the embedding dimension after visual input is processed by an encoder) and 'a' is the action with 6 dimensions. I could use a Fully Connected Neural Network (FCNN) for this function.

I'm wondering if it's necessary or beneficial to adjust the dimensions of the state or action inputs given the significant difference in their dimensions? For instance, should I consider directly increasing the dimensionality of the action to match the state dimensionality using a linear layer? Or are there other methods to handle these large dimensionality differences?

I welcome any related discussions, or references to similar concept code or code repository links.

$\endgroup$

1 Answer 1

2
$\begingroup$

No the dimensionality size difference it's not a problem when defining a reward function using neural networks, but if you really want to do so, you can easily do that using two different mappings as first layer: $$ r(a,s) = \phi(Wa+b, Us+c)\\ |W| = 256\times N\\ |U| = 6\times N $$

However, I don't really see a point to add this step instead of going for a fully connected layer since the beginning (as this ^^^ method can be seen as a masked fully connected layer, thus it's less powerful than it)

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .