2
$\begingroup$

I am trying to train a neural network using reinforcement learning / policy gradient methods. The states, i.e. the inputs, as well as the actions I am trying to sample are vectors with each element being a real number like in this question: https://math.stackexchange.com/questions/3179912/policy-gradient-reinforcement-learning-for-continuous-state-and-action-space

The answer that was given there already helped me a lot. Also, I have been trying to understand Chapter 13: Policy Gradient Methods from "Reinforcement Learning: An Introduction" by Sutton et al. and in particular Section 13.7: Policy Parametrization for Continous Actions.

My current level of understanding that I can use the weights in the network to calculate the mean/means and the standard deviation/covariance matrix. I can then use them to define a multivariate Gaussian distribution and randomly sample an action from there.

For now, I have one main question: In the book it says, that I have to split the weights, i.e. the policy's parameter vector, into two parts: $\theta = [\theta_{\mu}, \theta_{\sigma}]$. I can then use each part together with a feature vector to calculate the means and the covariance matrix. However, I was wondering, how this is usually done? Do I train two separate networks? I am not sure how an architecture of this will look like. Also, I am not sure what the output nodes will be in this case. Do they have any meaning like for supervised learning?

So far, I have only found papers that talk about this issue rather theoretically like it is presented in the book. I would be very happy to understand how this is actually implemented. Thank you very much!

$\endgroup$
  • 2
    $\begingroup$ Welcome to SE:AI! $\endgroup$ – DukeZhou Nov 17 '20 at 3:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.