In an environment with image observations, if we use an actor-critic method to find a good policy, commonly, we will use a feature extraction neural network, such as ResNet, to extract the information from the images.

There are two ways that we can use the feature extraction neural network in the actor-critic architecture.

  1. We can make the actor and critic share a common feature extraction neural network.

  2. We can make the actor and critic use separate feature extraction neural network

Which one is better?

By the way, if you choose the first option, is there a good way to optimize the feature extraction neural network?

  • $\begingroup$ I am currently not familiar with the details of Actor-Critic methods, but when you say "share a common feature extraction neural network", do you mean a "layer" (and not a "neural network")? Moreover, what do you mean by "better" in "Which one is better?"? Which one is better in terms of what? So, you may want to reformulate your question so that it makes more sense and you should also clarify your second question, in the sense that it's not clear what you mean by "good way to optimize". $\endgroup$ – nbro May 4 at 10:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.