0
$\begingroup$

Let say we are in an environment where a random agent can easily explore all the states of an environment (for example: tic-tac-toe).

In those environments, using off-policy algorithm, is it a good practice to train using exclusively random actions, instead or epsilon-greedy, Boltzmann or whatever ?

For my mind, it seems logical, but I have never heard about it before.

Improve this question
$\endgroup$
20
  • $\begingroup$ What makes you think that, given that DQN uses the experience replay (which is what I assume you mean by "DQN is offline"), then you never need to exploit? Can you also confirm that by "offline" you mean that it uses the experience replay or maybe you meant something else? $\endgroup$
    – nbro
    Commented Nov 7, 2020 at 21:41
  • $\begingroup$ By "offline" I mean training a policy with samples not collected by this policy (isn't it the literature meaning ?). When you ask "then you never need to exploit" ? this is my question too. If we are using offline algo, and if we are able to collect data from all possible states with only random actions, is there a need for standard exploration strategies like e-greedy or Boltzmann, instead of a simple random exploration ? $\endgroup$
    – Loheek
    Commented Nov 7, 2020 at 22:56
  • $\begingroup$ That's "off-policy" and not "offline". Check this. $\endgroup$
    – nbro
    Commented Nov 7, 2020 at 22:59
  • $\begingroup$ Thank you for this correction ! I confused, I have updated my question. $\endgroup$
    – Loheek
    Commented Nov 7, 2020 at 23:14
  • 1
    $\begingroup$ Yes, exactly, I was referring to that part "exploration strategy" (you should have said "I removed the exploitation"). Of course, I know what you mean, but that wording doesn't make the question very clear. That's what I mean. $\endgroup$
    – nbro
    Commented Nov 9, 2020 at 18:14

0

Reset to default

You must log in to answer this question.