I have made a (D)DQN Model.
In this model, regardless of whether I initialize it in DDQN or DQN mode, it uses an experience replay memory. The way I gather transitions for this experience replay memory is by stepping in the environment.
As my model seems not to be learning correctly, I need to confirm whether or not I am stepping (and therefore gathering transitions) correctly. This is the process:
- First I decide whether to pursuit the greedy or exploratory action.
- If I get greedy; I forward the state to my policy network (the network accountable for calculating the greedy policy in the training step - not the one evaluating the value of the state-action pair)
- I then step using the argmax from the output in part 2.
Is this correct? Or do I somehow have to utilize the policy network, not only when training?