Projected Bellman error has shown to be stable with linear function approximation. The technique is not at all new. I can only wonder why this technique is not adopted to use with non-linear function approximation (e.g. DQN)? Instead, a less theoretical justified target network is used.
I could come up with two possible explanations:
- It doesn't readily apply to non-linear function approximation case (some work needed)
- It doesn't yield a good solution. This is the case for true Bellman error but I'm not sure about the projected one.