0
$\begingroup$

In reinforcement learning (DQN) do I use epsilon when I am collecting examples from the environment or do I use epsilon when I am training the Q network and Target network ?

$\endgroup$
5
  • 1
    $\begingroup$ When taking actions in the environment, assuming you’re using an off-policy algorithm like Q-Learning which DQN is based on. $\endgroup$
    – David
    Jul 2, 2023 at 0:12
  • 1
    $\begingroup$ great thanks man $\endgroup$
    – Stef
    Jul 2, 2023 at 2:25
  • $\begingroup$ I can write into a proper answer if you find the comment useful? $\endgroup$
    – David
    Jul 2, 2023 at 8:41
  • $\begingroup$ yea I would find it very usefull $\endgroup$
    – Stef
    Jul 2, 2023 at 16:57
  • $\begingroup$ People can probably guess what "epsilon" you're referring to (me included), but you should have provided more context. Although there's already an answer, I would still suggest that you edit your post and describe what you mean by "epsilon". $\endgroup$
    – nbro
    Jul 3, 2023 at 10:46

1 Answer 1

1
$\begingroup$

As you asked specifically about DQN, I will talk about Q-Learning. Since it is an off-policy algorithm, we can collect data with any exploratory policy $\beta$ whilst learning about the greedy policy $\pi(s) = \arg\max_aQ(s,a)$. Typically $\beta$ will be an $\epsilon$-greedy, but you do not want to use this when updating your DQN. That is because you update your DQN using the Q-Learning update rule, by training it to approximate $r(s, a) + \gamma \max_a Q_{\bar{\theta}}(s', a)$, where $\bar{\theta}$ are the parameters of the target network.

$\endgroup$
1
  • 1
    $\begingroup$ thx man it makes sense $\endgroup$
    – Stef
    Jul 2, 2023 at 23:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .