I am trying to train a neural network using reinforcement learning / policy gradient methods. The states, i.e. the inputs, as well as the actions I am trying to sample are vectors with each element being a real number like in this question: https://math.stackexchange.com/questions/3179912/policy-gradient-reinforcement-learning-for-continuous-state-and-action-space
The answer that was given there already helped me a lot. Also, I have been trying to understand Chapter 13: Policy Gradient Methods from "Reinforcement Learning: An Introduction" by Sutton et al. and in particular Section 13.7: Policy Parametrization for Continous Actions.
My current level of understanding that I can use the weights in the network to calculate the mean/means and the standard deviation/covariance matrix. I can then use them to define a multivariate Gaussian distribution and randomly sample an action from there.
For now, I have one main question: In the book it says, that I have to split the weights, i.e. the policy's parameter vector, into two parts: $\theta = [\theta_{\mu}, \theta_{\sigma}]$. I can then use each part together with a feature vector to calculate the means and the covariance matrix. However, I was wondering, how this is usually done? Do I train two separate networks? I am not sure how an architecture of this will look like. Also, I am not sure what the output nodes will be in this case. Do they have any meaning like for supervised learning?
So far, I have only found papers that talk about this issue rather theoretically like it is presented in the book. I would be very happy to understand how this is actually implemented. Thank you very much!
