Why experience reply memory in DQN instead of a RNN memory?

Question

I was trying to implement a DQN without experience reply memory, and the agent is not learning anything at all. I know from readings that experience reply is used for stabilizing gradients. But how important is experience reply in DQN and similar RL algorithms? If the model needs to learn from memory, why don't we use a recurrent network, which has inbuilt memory to it? What is the advantage of experience reply over a recurrent memory?