I'm easing my way into a toy reinforcement learning problem where my objects can move and take different actions on a simple grid, but I'm having trouble understanding what constraints might exist in how I build my output layer and map it to the environent. I understand that, since vanilla backpropagation doesn't make sense in this context, we use methods like Policy Updating or Q-Learning to be able to differentiate with respect to the gradient. However, it is unclear to me if these methods impose constraints on the "form" of the output layer and its mappings to actions?
For instance, in the examples I read, typically there is one output node for each of the available actions in the environment. However, in a complex environment I could see one wishing for a variety of output forms. Perhaps, in my case, I desire to have one series of nodes determine the action to take and another node (i.e. a sin output for the angle) or two (i.e. determining the x/y) to determine the direction (i.e. "grab" action at "angle/position"). Is this viable, and indeed are any arbitrary mappings from the output layer to the environment valid? Or does current reinforcement learning techniques constrain the output and mapping (such as each node must be a possible action choice)?