I have to deal with a non-episodic task, where there is addittionally a continuous state space and more specifically in each time step there is always a new state that has never been seen before. I want to use DQN algorithm. As it is referred in Sutton's book (Chapter 10), the average reward setting, that is the undiscounted setting with differential function, should be preferred for non-episodic tasks with function approximation.

(a) Are there any reference papers that use DQN with the average reward setting?

(b) Why should the classic discounted setting (with no average reward) fail in such tasks, comparing to the average reward setting, taking into account that the highest reward that my agent can gain in a time step is 1.0 and thus the max $G_t = \frac{1}{1-γ}$ and not infinite ?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.