The Q-learning does not guarantee convergence for continuous state space problems (Why doesn't Q-learning converge when using function approximation?). In that case, is there an algorithm which can guarantee convergence?
I am looking at model-based RL specifically iLQR but all the solutions I find are for the continuous action space problem.