Imagine that the agent receives a positive reward upon reaching a state 𝑠. Once the state 𝑠 has been reached the positive reward associated with it vanishes and appears somewhere else in the state space, say at state 𝑠′. The reward associated to 𝑠′ also vanishes when the agent visits that state once and re-appears at state 𝑠. This goes periodically forever. Will discounted Q-learning converge to the optimal policy in this setup? Is yes, is there any proof out there, I couldn't find anything.
No, it will not converge in the general case (maybe it might in extremely convenient special cases, not sure, didn't think hard enough about that...).
Practically everything in Reinforcement Learning theory (including convergence proofs) relies on the Markov property; the assumption that the current state $s_t$ includes all relevant information, that the history leading up to $s_t$ is no longer relevant. In your case, this property is violated; it is important to remember whether or not we visited $s$ more recently than $s'$.
I suppose if you "enhance" your states such that they include that piece of information, then it should converge again. This means that you'd essentially double your state-space. For every state that you have in your "normal" state space, you'd have to add a separate copy that would be used in cases where $s$ was visited more recently than $s'$.