3
$\begingroup$

In a recent paper about progress in computer animation a so called motion graph is used to describe the transition between keyframes of facial animation. Easy Generation of Facial Animation Using Motion Graphs, 2018 As far as i understand from the paper, they used a motion capture device to record faces of real people and extract keyframes. Then a transition matrix was created to ensure that a walk from keyframe #10 to #24 is possible but a transition from keyframe #22 to #99 is forbidden.

The idea itself sounds reasonable good, because now a solver can search in the motion graph to bring the system from a laughing face to a bored face without interruption or unnatural in-between-keyframes. But wouldn't it be great if the transition matrix can be stored inside a neural network? As far as i understand the backpropagation algorithm , the neural network can learn input-output relations. So the neural network has to learn the transition probability between two keyframes. And a second neural network can then produce the motion plan which is also be trained by a large corpus. Is that idea possible or is it the wrong direction?

$\endgroup$
1
$\begingroup$

... the neural network has to learn the transition probability between two keyframes. And a second neural network can then produce the motion plan which is also be trained by a large corpus. Is that idea possible or is it the wrong direction?

That would be trivial.

More difficult, but not hard, would be to take a 14 hour video of someone and create a new video with them saying something they have never said previously. Here is a demonstration, from the BBC YouTube channel, titled: "Fake Obama created using AI video tool - BBC News".

The University of Washington demonstrates, using a variety of clips and only audio from an impersonator, that they can create realistic videos of the former president saying anything they want.

It's possible to sketch out an idea and let AI create a finished product of excellent quality.

The YouTube channel ACMSIGGRAPH has a video titled: "Technical Papers Preview: SIGGRAPH 2018", which is discussed on the webpage: "Could These Be The Next High-Tech Tools That Animators Use Daily?".

What you have asked about was possible long ago and not particularly difficult to do.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.