5
votes
Accepted
How does being on-policy prevent us from using the replay buffer with the policy gradients?
Let's say your old policy is $\pi_b$ and your current one is $\pi_a$. If you collected trajectory by using policy $\pi_b$ you would get return $G$ whose expected value is
\begin{align}
E_{\pi_b}[G_t|...
- 2,286
3
votes
Accepted
What is the pros and cons of increasing and decreasing the number of worker processes in A3C?
The correct number of child processes will depend on the hardware available to you.
Simplifying a bit, child processes can be in one of two states: waiting for memory or disk access, or running.
If ...
- 9,037
2
votes
Accepted
Why do we also need to normalize the action's values on continuous action spaces?
Notably, these two tips/tricks are useful because we are assuming the context of deep reinforcement learning here, as you pointed out. In DRL, the RL algorithm is guided in some fashion by a deep ...
- 1,122
1
vote
How do I create a custom gym environment based on an image?
The question is conceptually wrong, because of misunderstanding of area. Explanation: The idea is to replace open ai gym by something different. For example: web-site or computer game. There is no way ...
- 21
1
vote
Accepted
Why I got the same action when testing the A2C?
Disclaimer: Without the full code, we can only speculate. I encourage you to post the full code on Google Colab or something like this.
In the meanwhile, here is my point of view:
The Problem
Looks ...
- 824
Only top scored, non community-wiki answers of a minimum length are eligible