5 votes
Accepted

What is the difference between imitation learning and classification done by experts?

Imitation learning is supervised learning applied to the RL setting. In any general RL algorithm (such as Q-learning), the learning is done on the basis of the reward function. However, consider a ...
Sabyasachi Ghosh's user avatar
4 votes
Accepted

Is there a standardized method to train a reinforcement learning NN by demonstration?

Yes, this is known as imitation learning, which can be divided into inverse RL (i.e. learn a reward function from demonstrations, then apply RL), and behaviour cloning (supervised learning applied to ...
nbro's user avatar
  • 40.5k
4 votes
Accepted

What are reward networks in reinforcement learning?

In RL, reward networks (also called reward models), say $r_\theta$, have the sole role of learning (approximating) the reward function, defined as $r(s,a)$ or $r(s, a,s')$, in which the latter ...
Luca Anzalone's user avatar
3 votes

What are reward networks in reinforcement learning?

One representative paper for imitation learning is Ho & Ermon's "Generative Adversarial Imitation Learning" (2016). The goal in imitation learning is to learn a policy that can mimic the ...
cinch's user avatar
  • 2,282
3 votes

In imitation learning, do you simply inject optimal tuples of experience $(s, a, r, s')$ into your experience replay buffer?

That seems to be functional. That is a great approach, as long as you are using an off-policy algorithm (since the samples you are using to learn are not the policy currently being performed), like Q-...
BlueMoon93's user avatar
2 votes
Accepted

What is the surrogate loss function in imitation learning, and how is it different from the true cost?

A surrogate loss is a loss than you use "instead of", "in place of", "as a proxy for" or "as a substitute for" another loss, which is typically the "true&...
nbro's user avatar
  • 40.5k
2 votes

What are reward networks in reinforcement learning?

I think the reason why some refer to learning a reward policy and others learning an actor policy is that both are possible. For example, GAIL: A discriminator (the reward policy) learns to tell apart ...
Richie Bendall's user avatar
2 votes

What is the difference between imitation learning and classification done by experts?

I also had the same question, but after looking at this two links: this article and this lecture I think we can say that behavioral cloning (which is the simplest way for doing imitation learning) is ...
Abdalwhab Bakheet's user avatar
1 vote
Accepted

Initialising DQN with weights from imitation learning rather than policy gradient network

My understanding is that you are first training a policy network using imitation learning. Then you are adjusting that trained network in some way to be a value network for DQN. The most obvious ...
Neil Slater's user avatar
  • 32.1k
1 vote
Accepted

What does the number of required expert demonstrations in Imitation Learning depend on?

The answer to your question can be found in the original paper that introduced the max-margin and projection imitation learning (IL) algorithms: Apprenticeship Learning via Inverse Reinforcement ...
nbro's user avatar
  • 40.5k
1 vote

How can imitation learning data be collected?

Imitation learning data usually means data gathered from an expert, that is data from an agent proficient in the task. The agent may be: A human operator: have the operator complete the task and ...
Rexcirus's user avatar
  • 1,174
1 vote

Why could there be "information leak" if we do not use fixed horizons?

In the standard Cartpole environment, the episode ends either at 500 timesteps or when the Cartpole falls down. Expert demonstrations show how to keep the Cartpole upright for 500 timesteps. One ...
twink_ml's user avatar
  • 111
1 vote
Accepted

Is GAIL applicable if the expert's trajectories are for the same task but are in a different environment?

The authors of the paper Learning Robust Rewards with Adversarial Inverse Reinforcement Learning (2018, published in ICRL), which introduced the inverse RL technique AIRL, argue that GAIL fails to ...
nbro's user avatar
  • 40.5k

Only top scored, non community-wiki answers of a minimum length are eligible