I have a CNN with the regression task of a single scalar. I was wondering if an additional task of reconstructing the image (used for learning visual concepts), seen in a DeepMind presentation with the loss and re-parametrization trick of Variational Autoencoder, might help the principal task of regression.

So you can imagine some convolutions with the role of feature extraction with some output X (let's say a vector of 256 values), that X goes into the VAE which computes Z and then the reconstructed image. And then the original regression task would take either X or Z in order to compute that scalar value.

Has anyone tried such an approach, is it worth the work? Thank you


1 Answer 1


I have not worked on this but I think I can give you a theoretical perspective of using VAE's. Regression is a Supervised Learning task and is basically a mapping from Input to Output where the Neural Net will approximate the function $f(input) = output$.

VAE's on the other hand are good for finding how a latent variable affects the output. For example, if you have a task of training on a persons facial emotions, and if your latent space contains 2 variables $z_1$ and $z_2$ then you might find varying $z_1$ varies the amount of smile on the face, while varying $z_2$ might give the amount of drooping of eyes. I suggest you check this video from Stanford at ~44:00 to see this actually happens or check this blog. So VAE's might have been useful if your output contained more features which would vary according to variations in latent variables, but a single scalar output can only tell you about the rate of effect on varying a latent variable.

But if your job is only for better regression, Auto-Encoders are the better alternative, since it has an inherent de-noising ability and sufficient training might help in de-noising the input, and thus provide better results if classified on the basis of latent variables.

An approach, which I think is kind of similar (to your thinking) has been proposed by Kingma, et al. for Semi Supervised learning in this paper. The paper has very poor description of the method so I would suggest you check out this blog. They have used an additional classifier for reconstruction of the original input and trained the classifier when labels are present.

  • 1
    $\begingroup$ Nice, thank you for your references. I will definitely read them. Actually my neural network has multiple outputs, but I just presented the problem in a simpler manner. By doing this, you really made a good point with "a single scalar output can only tell you about the rate of effect on varying a latent variable". $\endgroup$ Apr 18, 2019 at 8:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .