My best guess that it's been done to reduce the computation time, otherwise we would have to find out the q value for each action and then select the best one.
It has no real impact on computation time, other than a slight increase (due to extra memory used by two networks). You could cache results of the target network I suppose, but it probably would not be worth it for most environments, and I have not seen an implementation which does that.
Am I missing something?
It is to do with stability of the Q-learning algorithm when using function approximation (i.e. the neural network). Using a separate target network, updated every so many steps with a copy of the latest learned parameters, helps keep runaway bias from bootstrapping from dominating the system numerically, causing the estimated Q values to diverge.
Imagine one of the data points (at
S, A, R, S') causes a currently poor over-estimate for
Q(S', A') to get worse. Maybe
S', A' has not even been visited yet, or the value of
R seen so far is higher than average, just by chance. If a sample of
(S,A) cropped up multiple times in experience replay, it would get worse again each time, because the update to
Q(S,A) is based on
R + max_a Q(S',a). Fixing the target network limits the damage that such over-estimates can do, giving the learning network time to converge and lose more of its initial bias.
In this respect, using a separate target network has a very similar purpose to experience replay. It stabilises an algorithm that otherwise has problems converging.
It is also possible to have DQN with "double learning" to address a separate issue: Maximisation bias. In that case you may see DQN implementations with 4 neural networks.