3
$\begingroup$

In Reinforcement Learning, when I train a model, it comes up with its own set of solutions. For example, if I am training a robot to walk, it will come up with its own walking gait, such as this Deep Mind robot that has learned to walk in a bizarre gait. It can surely walk/run although the movements does not quite look like a human.

I was wondering how can I train a model by providing it some kind of reference motion data? For example, if I collect motion data from a walking human and then provide it to the training, can the training be made learn the walking movements that looks similar to the reference motion data?

Searching online I did find some links that shows this is possible. For example, here is a research where the researchers did exactly what I am trying to do, they fed motion data captured from humans to a simulation and made it learn the movements.

So, my question is: How can I give some kind of hints or reference data to a reinforcement learning model instead of just leaving it all by itself? how exactly is this done? What is it even called? What are some terms and keywords that I can search for to learn more?

Many thanks in advance

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

You can look into the techniques used in GANs (genereative adverserial networks). These networks work by having 2 learning agents. 1 to create images and 1 to learn the difference in a human made image and a computer generated image. This works because the 2 agents drive each other to be better and ultimately make the generator create images which can't be distinguished from real life images.

In your case you can make a agent trying to tell if the data is human or computer generated. The agent learning to move will then get negative rewards when the other agent can identify it as computer movement. This way the mover will learn to move like your reference data.

UPDATE:

I just found this video and paper which does exactly the same as you asked. Instead of using a GAN like structure they use a task specific reward and a imitation reward, which is based on reference motion data they have.

https://www.youtube.com/watch?v=vppFvq2quQ0

Improve this answer
$\endgroup$
2
  • $\begingroup$ Nice answer but is it implementable? I mean did anyone implement it or is it hard to implement. $\endgroup$
    – user9947
    Commented Sep 16, 2019 at 11:35
  • 1
    $\begingroup$ To my knowledge it hasn't been done yet. Is it implementable? sure, if you look at the successes of GANs it should be able to work. to only real hard problem is getting enough reference data and figuring the right balance between moving fast and moving like a human I would guess. This is from someone who hasn't implemented a GAN before though, so I don't know how valuable my opinion on this is. $\endgroup$
    – Lustwelpintje
    Commented Sep 16, 2019 at 12:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .