Questions tagged [inverse-rl]

For questions related to inverse reinforcement learning (IRL), the problem of recovering the reward function from the observed behavior (or policy) of an agent. It's called IRL because it's the inverse problem of RL, i.e. the problem of finding optimal policies given the reward function.

6 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
2 votes
0 answers
63 views

What is the dimensionality of these derivatives in the paper "Active Learning for Reward Estimation in Inverse Reinforcement Learning"?

I'm trying to implement in code part of the following paper: Active Learning for Reward Estimation in Inverse Reinforcement Learning. I'm specifically referring to section 2.3 of the paper. Let's ...
user avatar
1 vote
0 answers
80 views

What do state features mean in the context of inverse RL?

I am reading Zeibart's Inverse RL paper, and it states - The agent is assumed to be attempting to optimize some function that linearly maps the features of each state, $f_{sj} \in \mathbb{R}^k$, to a ...
user avatar
1 vote
1 answer
185 views

Why is it that the state visitation frequency equals the sum of state visitation frequency from initial time step to the horizon?

In the maximum entropy inverse reinforcement learning paper, Ziebart et al. show that the state visitation frequency $\rho(s)$ of a state $s$ can be computed as $$ \rho_{\pi}(s) = \sum_{t}^{T} P(s_t=s|...
user avatar
0 votes
0 answers
19 views

Augmented an Image with other data when training CNN

In the typical RL/MDP framework, I have offline data of $(s,a,r,s')$ of expert Atari gameplay. I'm looking to train a CNN to predict $r$ based on $(s, a)$. The states are represented by a $4 \times 84 ...
user avatar
  • 203
0 votes
0 answers
38 views

Proving existence or non existence of reward function to make given policy "uniquely" optimal when reward function is dependent only on S or both S,A

I was going through paper titled "Algorithms for Inverse Reinforcement Learning" by Andrew Ng and Russell. It states following basics: MDP $M$ is a tuple $(S,A,\{P_{sa}\},\gamma,R)$, where ...
user avatar
  • 211
0 votes
1 answer
135 views

How to make input variable as trainable parameter in a neural network?

I am working on an optimization problem. First, I have done forward training to work the network as a surrogate model, then I freeze the output and I want to find an optimal value of input for a given ...
user avatar
  • 11