I have made a (D)DQN Model.

In this model, regardless of whether I initialize it in DDQN or DQN mode, it uses an experience replay memory. The way I gather transitions for this experience replay memory is by stepping in the environment.

As my model seems not to be learning correctly, I need to confirm whether or not I am stepping (and therefore gathering transitions) correctly. This is the process:

  1. First I decide whether to pursuit the greedy or exploratory action.
  2. If I get greedy; I forward the state to my policy network (the network accountable for calculating the greedy policy in the training step - not the one evaluating the value of the state-action pair)
  3. I then step using the argmax from the output in part 2.

Is this correct? Or do I somehow have to utilize the policy network, not only when training?

  • $\begingroup$ I think there's a typo in the title "How Are The Steps Done in DDQN?" does not make sense to me. Maybe you need to clarify which steps. Can you be more specific about this "As my model seems not to be learning correctly"? Maybe tell us how many training steps have you trained your model for and also things like the learning rate, which environment you're using, etc, etc. $\endgroup$
    – nbro
    Jan 6 at 21:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.