0
$\begingroup$

Let say we are in an environment where a random agent can easily explore all the states of an environment (for example: tic-tac-toe).

In those environments, using off-policy algorithm, is it a good practice to train using exclusively random actions, instead or epsilon-greedy, Boltzmann or whatever ?

For my mind, it seems logical, but I have never heard about it before.

$\endgroup$
  • $\begingroup$ What makes you think that, given that DQN uses the experience replay (which is what I assume you mean by "DQN is offline"), then you never need to exploit? Can you also confirm that by "offline" you mean that it uses the experience replay or maybe you meant something else? $\endgroup$ – nbro Nov 7 '20 at 21:41
  • $\begingroup$ By "offline" I mean training a policy with samples not collected by this policy (isn't it the literature meaning ?). When you ask "then you never need to exploit" ? this is my question too. If we are using offline algo, and if we are able to collect data from all possible states with only random actions, is there a need for standard exploration strategies like e-greedy or Boltzmann, instead of a simple random exploration ? $\endgroup$ – Loheek Nov 7 '20 at 22:56
  • $\begingroup$ That's "off-policy" and not "offline". Check this. $\endgroup$ – nbro Nov 7 '20 at 22:59
  • $\begingroup$ Thank you for this correction ! I confused, I have updated my question. $\endgroup$ – Loheek Nov 7 '20 at 23:14
  • 1
    $\begingroup$ Yes, exactly, I was referring to that part "exploration strategy" (you should have said "I removed the exploitation"). Of course, I know what you mean, but that wording doesn't make the question very clear. That's what I mean. $\endgroup$ – nbro Nov 9 '20 at 18:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.