Context: Double Q-learning was introduced to prevent the maximization bias from q-learning. Instead of learning a single Q-network, we can learn two (or in general $K > 1$) and our Q-estimate would be the min across all these Q-networks.

Question: Does it make sense to share the layers of these Q-networks (except the last layer)?

So, instead of having 2 networks of size [64, 64, 2] (with ~8.5K parameters in total) we can have one network of size [64, 64, 4] (with ~4.3K params).

I couldn't see much of a downside to this, but all the implementations I've seen keep two completely different networks.



You must log in to answer this question.

Browse other questions tagged .