Questions tagged [offline-reinforcement-learning]

Filter by
Sorted by
Tagged with
1 vote
1 answer
33 views

Expectile regression in Implicit Q-Learning

I am reading Kostrikov et al.'s "Offline Reinforcement Learning with Implicit Q-Learning" but got stuck understanding one particular transformation they use. They describe the loss function ...
user118967's user avatar
0 votes
0 answers
22 views

Offline CartPole on infinitely long line?

I am tentatively exploring some RL research that involves doing offline RL on a version of the Gymnasium CartPole where the cart can move on $\mathbb R$, as opposed to the standard version (see link) ...
Novice's user avatar
  • 111
1 vote
1 answer
158 views

Fit Q Evaluation in offline reinforcement learning

I am working on a PyTorch implementation of Implicit Q-Learning (IQL) (paper), given a dataset $\mathcal D = \left\{ (\mathbf s_i, \mathbf a_i, \mathbf s_i', r_i ) \right\}$ of transitions. I think I ...
Novice's user avatar
  • 111
1 vote
1 answer
96 views

DQN with experience history to learn from already saved - which reward should I take?

I want to train a DQN model in an off-policy fashion, where my behavior policy is an older agent. I have a big memory of a lot of episodes of this agent. Now I want to find a better policy using DQN. ...
PatrickSVM's user avatar
0 votes
0 answers
36 views

How to evaluate the performance of off-line & model-free reinforcement leaning?

I'm currently studying on off-line reinforcement learning (RL) and trying to utilize it for medical data. Because it seemed hard to develop well-performing environment model, I decided to adopt model-...
Maverick's user avatar
0 votes
1 answer
245 views

Action selection in Batch-Constrained Deep Q-learning (BCQ)

For simplicity, let's consider the discrete version of BCQ where the paper and the code are available. In the line 5 of Algorithm 1 we have the following: $$ a' = \text{argmax}_{a'|G_{\omega}(a', s')/\...
HenDoNR's user avatar
  • 81
2 votes
1 answer
954 views

Offline/Batch Reinforcement Learning: when to stop training and what agent to select

Context: My team and I are working on a RL problem for a specific application. We have data collected from user interactions (states, actions, rewards, etc.). It is too costly for us to emulate agents....
MetaHG's user avatar
  • 21
14 votes
1 answer
8k views

What is the relation between online (or offline) learning and on-policy (or off-policy) algorithms?

In the context of RL, there is the notion of on-policy and off-policy algorithms. I understand the difference between on-policy and off-policy algorithms. Moreover, in RL, there's also the notion of ...
nbro's user avatar
  • 40.1k