I'm replicating a particular network architecture which is very sparse with its details.
One part of said architecture is shown in the image, where
h is a 1x1024 or 1024x1 output from a transformer and needs to be combined with the output of an
MLP to produce a 1x55 or 55x1 vector, with the order of the dimensions not mattering. As such the matrix multiplication step should involve either multiplying (55 x k) and (k x 1), or (1 x k) and (k x 55) matrices. The input to the
MLP is a (55 x 32) matrix flattened to a (1760 x 1) vector. For context, this input encodes item information from a game about 55 items. h contains information about the current game state and we want to produce a policy
pi for selecting an item.
The constraints of the matrix multiplication mean that I need to perform some kind of dimensionality increase or reduction on either the elements of
h or the output of the
MLP. However, I'm not really sure what would be best in this case as I don't have much experience with building neural networks in general. Any advice would be appreciated.