In short, imitation learning means learning from the experts. Suppose I have a dataset with labels based on the actions of experts. I use a simple binary classifier algorithm to assess whether it is good expert action or bad expert action.

How is this binary classification different from imitation learning?

Imitation learning is associated with reinforcement learning, but, in this case, it looks more like a basic classification problem to me.

What is the difference between imitation learning and classification done by experts?

I am getting confused because imitation learning relates to reinforcement learning while classification relates to supervised learning.


Imitation learning is supervised learning applied to the RL setting.

In any general RL algorithm (such as Q-learning), the learning is done on the basis of the reward function. However, consider a scenario where you have available the optimal policy in the form of a table, mapping each state to each action. In this scenario you will not care about the rewards received - rather, you'd simply do a table lookup to decide the optimal action.

This scenario is impractical in most settings because the table for the optimal policy will be too big. However, if you have enough entries from the table, you can use a general function approximator such as a neural network to find the optimal action. Again, you do not need to look at the rewards, but only at the state $\rightarrow$ action mappings. I do not know imitation learning in detail beyond this, but I suspect in the case of discrete actions (such as in Chess, Go), it would be trained with a cross-entropy objective as is typical of classification tasks.

I suspect that the reason this has a different name in the RL setting is because this is different than how the conventional RL algorithms work. Also, much of RL thinking is inspired from everyday interaction / biology. Think of how we learn how to drive, or play sports such as soccer. Typically there is a coach who tells you what actions to take under different conditions, and you imitate those actions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.