3
$\begingroup$

I have a variational convolutional autoencoder that has trained on 2 images and outputs a linear interpolation (inserted at the bottleneck stage) between those 2 input images.

enter image description here

However, the result looks (rather dissapointingly) like some Powerpoint cross-fade effect.

What would be needed to obtain a more novel transition effect, for instance limbs moving from '1' and seamlessly reshaping into '7'?

$\endgroup$

1 Answer 1

3
$\begingroup$

You need more training images. Far more, at least a few hundred, with variations. The latent space has no meaningful form to it when you train with just two end points. The decoder will have no examples of shifting forms that it could arrange on the path between typical 1s and typical 7s.

Also, if there are separate classes within the encoded space, representations in the middle are not guaranteed to be coherent transformations between them. More training examples will help, but some parts of the latent space will be odd. In human face examples you can often observe this when latent space crosses between glasses and no glasses, the transition can fall apart.

$\endgroup$
2
  • $\begingroup$ thank you, I think I understand how morphing works now. Although, I still can't wrap my head around how smile replacement works (where a non-smiling face is replaced by a smiling face, but the rest of the face is unchanged). Is this also done through latent space pathfinding? But how to keep the rest of the face unchanged while changing only the smile around the mouth? $\endgroup$
    – James
    Commented Mar 23 at 17:51
  • 2
    $\begingroup$ @James that would be a different question, but the principle is to try and extract the "smile vector" by using all the labelled examples. It's rarely perfect, and often other image features change too $\endgroup$ Commented Mar 23 at 18:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .