Skip to main content

Questions tagged [offline-reinforcement-learning]

Filter by
Sorted by
Tagged with
2 votes
1 answer
259 views

Export trained model offline to be used by an application

I'm trying to create a text-based game using AI. I was playing around with text classification AutoML from Vertex AI just to learn AI, and then pick the best solution for my use case. Is it possible ...
Danilo Ruziska's user avatar
1 vote
1 answer
50 views

Expectile regression in Implicit Q-Learning

I am reading Kostrikov et al.'s "Offline Reinforcement Learning with Implicit Q-Learning" but got stuck understanding one particular transformation they use. They describe the loss function ...
user118967's user avatar
0 votes
0 answers
29 views

Offline CartPole on infinitely long line?

I am tentatively exploring some RL research that involves doing offline RL on a version of the Gymnasium CartPole where the cart can move on $\mathbb R$, as opposed to the standard version (see link) ...
Novice's user avatar
  • 121
2 votes
1 answer
335 views

Fit Q Evaluation in offline reinforcement learning

I am working on a PyTorch implementation of Implicit Q-Learning (IQL) (paper), given a dataset $\mathcal D = \left\{ (\mathbf s_i, \mathbf a_i, \mathbf s_i', r_i ) \right\}$ of transitions. I think I ...
Novice's user avatar
  • 121
1 vote
1 answer
109 views

DQN with experience history to learn from already saved - which reward should I take?

I want to train a DQN model in an off-policy fashion, where my behavior policy is an older agent. I have a big memory of a lot of episodes of this agent. Now I want to find a better policy using DQN. ...
PatrickSVM's user avatar
0 votes
0 answers
37 views

How to evaluate the performance of off-line & model-free reinforcement leaning?

I'm currently studying on off-line reinforcement learning (RL) and trying to utilize it for medical data. Because it seemed hard to develop well-performing environment model, I decided to adopt model-...
Maverick's user avatar
0 votes
1 answer
252 views

Action selection in Batch-Constrained Deep Q-learning (BCQ)

For simplicity, let's consider the discrete version of BCQ where the paper and the code are available. In the line 5 of Algorithm 1 we have the following: $$ a' = \text{argmax}_{a'|G_{\omega}(a', s')/\...
HenDoNR's user avatar
  • 106
2 votes
1 answer
1k views

Offline/Batch Reinforcement Learning: when to stop training and what agent to select

Context: My team and I are working on a RL problem for a specific application. We have data collected from user interactions (states, actions, rewards, etc.). It is too costly for us to emulate agents....
MetaHG's user avatar
  • 21
14 votes
1 answer
9k views

What is the relation between online (or offline) learning and on-policy (or off-policy) algorithms?

In the context of RL, there is the notion of on-policy and off-policy algorithms. I understand the difference between on-policy and off-policy algorithms. Moreover, in RL, there's also the notion of ...
nbro's user avatar
  • 41k