What is the idea behind double DQN?
The Bellman equation used to calculate the Q values to update the online network follows the equation:
value = reward + discount_factor * target_network.predict(next_state)[argmax(online_network.predict(next_state))]
The Bellman equation used to calculate the Q value updates in the original DQN is:
value = reward + discount_factor * max(target_network.predict(next_state))
But the target network for evaluating the action is updated using weights of the online_network and the value fed to the target value is basically the old q value of the action.
Any ideas on how or why adding another network based on weights from the first network helps? Any example?