# Why does adding another network help in double DQN?

What is the idea behind double DQN?

The Bellman equation used to calculate the Q values to update the online network follows the equation:

value = reward + discount_factor * target_network.predict(next_state)[argmax(online_network.predict(next_state))]


The Bellman equation used to calculate the Q value updates in the original DQN is:

value = reward + discount_factor * max(target_network.predict(next_state))


But the target network for evaluating the action is updated using weights of the online_network and the value fed to the target value is basically the old q value of the action.

Any ideas on how or why adding another network based on weights from the first network helps? Any example?

In $$Q$$-learning, the agent updates the value of executing an action in the current state, using the values of executing actions in a successive state. This procedure often results in an instability because the values change simultaneously on both sides of the update equation. A target network is a copy of the estimated value function that is held fixed to serve as a stable target for some number of steps.