0
$\begingroup$

When a single image is assigned for training, an auto-encoder should be able to gradient-descend and find the full set of satisfactory weights that will reconstruct this image.

enter image description here

Suppose a second image is now assigned to the autoencoder:

enter image description here

Can the previously trained autoencoder weights (for image 1) now be reused immediately (without further training) to reconstruct the second image as well?

$\endgroup$
3
  • 1
    $\begingroup$ If you do not pose constraints on the dimensionality of the encoded representation, you can cheat by imposing $d= |x|$ and learn an identity mapping of the input quite easily. Of course, this is a perfectly useless network since your encoded representation has the same dimensionality of the input and replicates the input exactly :) $\endgroup$
    – Ciodar
    Mar 22 at 8:56
  • $\begingroup$ @Ciodar thank you, yes I believe the identity mapping is what I was really after... Say suppose this auto-encoder is implemented as a fully-convolutional U-net type network (all weights are kernel values or bias), skipped connections, and relu slapped onto each stage's output, then is it certain that a specific set of weights that perform the identity mapping (that maps all images into itself pixel-to-pixel) can still be found after all those convolution steps and relu information truncation? $\endgroup$
    – James
    Mar 22 at 9:18
  • 1
    $\begingroup$ Look at a single residual block: if you feed $x$, all you need to do is make the network zero out everything (i.e all weights to zero), to obtain $x$ as output. Maybe this question is answering exactly to that. $\endgroup$
    – Ciodar
    Mar 22 at 17:36

1 Answer 1

1
$\begingroup$

The latent space representation learned by the VAE may not be meaningful or informative with only one input image. Reconstruction of the input image may not be accurate or meaningful since the VAE may simply learn to memorize the input image rather than learning useful latent features for reconstruction.

Even assuming it can be trained successfully with only one image as yours, it's quite impossible to achieve your goal since these 2 images have obviously very different appearances and features. For example, your trained image has a big hat covering a large portion of space which your target output image lacks. There're many other dissimilarities. By randomly sampling the learned latent space with only one image, you can only expect similarly generated image at best.

$\endgroup$
2
  • $\begingroup$ thank you! The reason I ask is I guess because while I'm convinced that a good set of weights can be found that reconstruct a single image, I have serious fundamental doubts whether/why there must exist a single set of weights that can perform the dual purpose of reconstructing exactly both image 1 and image 2 when given each image as input... Is it certain that 1 large enough model having sufficient number of weights can always reconstruct any number of input images exactly (auto-encode many images at once), assuming we can find the correct weights that can perform all this multitasking? $\endgroup$
    – James
    Mar 22 at 7:08
  • 1
    $\begingroup$ While in theory a sufficiently large AE may have the capacity to memorize individual training samples (reconstruct the input images exactly), the goal of ML is not memorization but generalization, that's why DAE/CAE exist. And VAE unlike AE cannot generate exactly same image since there's sampling uncertainty inherent. For your OP since the 2nd image is not in the training set, AE won't be able to reconstruct it exactly or even closely, as it hasn't learned the specific patterns and features present in that image which is very different from your trained image. You may try fine tuning though. $\endgroup$
    – cinch
    Mar 22 at 18:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .