Questions tagged [autoencoders]
For questions about autoencoders, a type of unsupervised artificial network for learning efficient data codings.
131
questions
0
votes
0
answers
7
views
Drastic Change in MSE when Scaling Factor Changes
I am training an autoencoder meant to detect anomalies. Initially I scaled my data using a min-max scaler. I realized that this scalar isn't the best because anomalies can cause bias in the scalar. ...
- 113
0
votes
1
answer
31
views
Why is the variational lower bound is easier to compute than the original marginal distribution?
Why is the ELBO of $p(x)=\int p(x|z)p(z)\mathrm{d}z$ easier to compute/estimate than the expression itself? Can we compute this quantity itself through sampling in the same way? I understanding that ...
- 101
0
votes
0
answers
16
views
Mean and std in vaerational autoencoder
Are mean and standard deviation in variational autoencoders equal?
If not, then why are both calculated in the same way?
1
vote
0
answers
63
views
Latent Diffusion Model Can't Learn the Latent Space of a VAE for the MNIST-Fashion Dataset
I'm currently playing around with LDMs on the MNIST-Fashion dataset. I thought the VQVAEs used in the original paper were a bit overkill for what I'm doing (and I don't fully understand how they ...
- 127
0
votes
1
answer
22
views
How does mixing and matching encoders and decoders work in image segmentation?
I had a conceptual questions regarding architectures. I am using this git hub repository that allows one to quickly put together a segmentation pipeline. In reading the readme one thing that has me ...
- 103
0
votes
1
answer
64
views
How to generate new data using VAE?
I have built the following function which takes as input some data and runs a VAE on them:
...
- 101
0
votes
0
answers
39
views
Are autoencoders computationally cheaper than MLPs with the same number of neurons?
Are autoencoders computationally cheaper than other neural networks such as MLP with the same number of neurons?
I have read in some papers that autoencoders train the network faster, and I could ...
- 1
0
votes
0
answers
17
views
Should I pre-compute embeddings from a frozen network or are the gradients important?
I'm training an AutoEncoder-like network to take a face embedding, encode it, decode it, and then I calculate the loss between the input and output embedding. The input embedding is calculated by ...
- 121
1
vote
0
answers
21
views
Can a convolution learn to generate fine details? [closed]
I'm trying to get a convolutional autoencoder to reconstruct images of a dataset with crisp details.
I've read in a couple places that convolutional autoencoders "naturally produce blurry images&...
- 191
1
vote
0
answers
288
views
Why VQ-VAE instead of VAE?
From the paper on VQ-VAE, it said that the vector quantized variational autoencoder (VQ-VAE), differs from VAEs in two key ways:
the encoder network output discrete, rather than continuous, codecs
...
- 125
2
votes
2
answers
688
views
What is an appropriate size for a latent space of (variational) autoencoders and how it varies with the features of the images?
I am training an autoencoder and a variational autoencoder using satellite and streetview images. I have tested my program on standard datasets such as MNIST and CelebA. It seems that the latent space ...
- 23
0
votes
0
answers
39
views
is Flipout an upgrade of the local reparameterization trick or a completely different technique?
I was reading the Flipout paper and I am confused about 1 thing:
when the author samples the perturbation matrix $\hat{\Delta W}$ does he do directly from the real variational distribution $q_{\...
- 101
1
vote
0
answers
28
views
Pseudo Label Generation for Generative Cooperative Learning
I am trying to implement this paper for unsupervised video anomaly detection.
The gist of the paper seems to be:
Create a dataset for an unsupervised setting, by mixing up the train and anomalous ...
- 141
0
votes
0
answers
46
views
Autoencoder make spectrogram important parts more pronounced with a "log loss"
Hi I want to create a neural network that essentially picks out the most pronounced parts of a spectrogram.
Assume this is the True spectrogram:
...
- 101
1
vote
0
answers
10
views
Is it possible for PixelCNN to tell us what it generates?
I coded PixelCNN with the help of Keras official website. Also, I read the paper. I can use PixelCNN, similar to a decoder or generator (to generate samples). My question is, "is it possible to ...
- 29
0
votes
0
answers
10
views
Extracting behavior (switch On/Off) of an electric load from unlabeled time series data
Following are the details of my dataset:
sampling frequency: 1 Hz
No. of useful features: 10
The time series dataset is from household wherein I'm required to find ...
- 71
0
votes
1
answer
79
views
Reconstructing 3D models from 2D images using autoencoders
I went through a research paper ("Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders") and tried to implement the approach following this diagram:
0
votes
1
answer
106
views
What is the distribution of autoencoder embeddings?
Is there any result on the distribution of autoencoder embeddings?
For example, the following image (taken from this article) visualizes the latent space with t-SNE. As you can see, images from the ...
- 181
0
votes
1
answer
83
views
Autoencoders: Where does the encoder end and the decoder begin?
Consider a simple Autoencoder neural net:
...
- 101
8
votes
2
answers
796
views
Is plain autoencoder a generative model?
I am wondering how a plain auto encoder is a generative model though its version might be but how can a plain auto encoder can be generative. I know that Vaes which is a version of the autoencoder is ...
- 125
1
vote
0
answers
41
views
Is there a name for this model?
I have an image autoencoder model trained as follows:
Step 1) train a GAN to obtain a generator capable of drawing from the data manifold by sampling a normal distribution in latent space
Step 2) ...
- 11
-1
votes
1
answer
119
views
Denoise autoencoder not training properly [closed]
I'm trying to make a denoise autoencoder wherein the encoder part is vgg16 and decoder is opposite of vgg16(encoder) network. My dataset consists of 5K images in grayscale.
Now while training, the ...
0
votes
0
answers
49
views
Why does the feature space of an autoencoder typically contain more info than a teacher-student model?
This is a question our Prof gave us as exam preparation, but I don't know why the Autoencoder should contain more info than the Teacher Student model.
Teacher Student Models are a class of models in ...
- 348
0
votes
0
answers
76
views
Do channels on the encoder's output layer really matter in convolutional autoencoders?
I viewed the example in Keras for a convolutional autoencoder. I noticed that the number of units in each layer of both the encoder and decoder never falls below that of the input. In other words, ...
- 53
1
vote
2
answers
426
views
Strange artifacts in autoencoder outputs
I'm training an autoencoder, that does not downsample images but processes them in the same size. For example, a 256x256 input will always be processed at 256x256 resolution, only the channels ...
- 125
0
votes
0
answers
177
views
VAEs vs Autoencoders with BatchNorm and Dropout?
It struck me that regular auto-encoders with batch-norm and dropout have quite similar properties to VAEs which made me wonder whether VAEs where really much better than this simpler alternative. Let ...
- 306
0
votes
1
answer
23
views
Do Quo et al (2013) perform backpropagation between layers?
Le et al. 2013's non-weight sharing CNN has inspired me to ask two questions on this site previously.
When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) ...
- 151
0
votes
1
answer
532
views
What is the state of the art concerning autoencoder that connect 2 images that are not similar but are physicaly related?
I am currently working on an autoencoder that connect two images. The first one can be seen as the electron flow and the second one is the electrostatic potential seen by the electrons. Long story ...
1
vote
1
answer
52
views
Why is training all layers at a time effective for a multi-layer autoencoder?
This training of all layers of a CNN simultaneously is standard practice today. It is found in every CNN (AlexNet (2012), VGG, Inception, GANs, etc) and even pre-CNN networks such as Le et al. 2012.
...
- 151
4
votes
2
answers
573
views
Can I apply reparametrization trick on "any" deep neural network?
I came across the "reparametrization trick" for the first time in the following paragraph from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning ...
- 3,481
0
votes
0
answers
37
views
Masked Autoencoder Structure
In the following structure when we use MADE due to the constraints for making a masked autoencoder, it seems some inputs do not have any connection to the next layer, and also there is the output that ...
- 1
3
votes
1
answer
238
views
How to determine the quality of synthetic data?
I'm working on a VAE model to produce synthetic data of X-Ray diffraction spectrums.
I try to figure out how I can measure the quality of the spectrums. The goal would be to produce synthetic data ...
- 33
1
vote
1
answer
162
views
Why is the prior on the latent variable standard gaussian in VAE?
While training a standard VAE, we assume that the prior on the latent variable Z is the standard gaussian and we use KL divergence to push the posterior as close as possible to the standard gaussian. ...
1
vote
2
answers
861
views
Are mean and standard deviation in variational autoencoders unique?
In general, if I have a collection of data then mean(Expectation) and standard deviation are calculated as follows
$$\text{mean } = \mu = \mathbb{E}[X] = \sum\limits_{i = 1}^n p_ix_i $$
$$\text{...
- 3,481
0
votes
0
answers
68
views
Train separate AutoEncoder's on each class or one AE for all classes to learn features?
I'm working on a project where the dataset contains time series of three classes, depending on the shape of the series. I want to learn the representations of these series as vectors, so naturally I ...
- 1
1
vote
0
answers
40
views
What is the conceptual difference between convolutional neural networks and auto-encoders?
I'm familiar with Auto-Encoders and I'm about to dive into CNNs. By having a look at the most important component of a CNN, the filter:
I wonder how it is different from Auto-Encoders:
For me, it ...
- 205
0
votes
0
answers
42
views
Weird KL divergence behaviour
I'm training a complex model for motion prediction using a VAE, however the KL divergence has a very strange behavior.
A scheleton of the network is the following:
At the end my network compute the ...
- 179
0
votes
0
answers
235
views
Why would an auto-encoder produce latent vectors with many zeros?
My autoencoder give latent vectors with many zeroes components like:
...
- 103
1
vote
0
answers
38
views
How do I select the number of neurons for each layer in an auto-encoder for dimensionality reduction?
I am trying to apply an auto-encoder for dimensionality reduction. I wonder how it will be applied on a large dataset.
I have tried this code below. I have total of 8 features in my data and I want to ...
- 111
0
votes
1
answer
48
views
How to train a model for 1 image class to detect anomaly?
I want to train a model with python over the images, and these images are for a metal product.
my aim is to detect the defects, to notice if a product is a failure.
what kind of architecture do you ...
- 1
0
votes
2
answers
3k
views
In variational autoencoders, why do people use MSE for the loss?
In VAEs, we try to maximize the ELBO = $\mathbb{E}_q [\log\ p(x|z)] + D_{KL}(q(z \mid x), p(z))$, but I see that many implement the first term as the MSE of the image and its reconstruction. Here's a ...
- 149
0
votes
0
answers
148
views
What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?
Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently.
However, do they follow the same process during the inference phase, i.e., do they ...
- 73
0
votes
1
answer
119
views
Convert LSTM univariate Autoencoder to multivariate Autoencoder
I have the following code snippet which takes in a single column of value i.e. 1 feature. How do I modify the LSTM model such that it accepts 3 features?
...
- 21
0
votes
1
answer
105
views
Why Autoencoder Weights Are Not Always Tied
To me, tying weights in an autoencoder makes sense if we think of the auto encoder as doing PCA. Why in any situation would it make sense to not tie the weights? If we don't tie the weights, would it ...
- 116
1
vote
0
answers
22
views
Dealing with bias in multi-channel auto encoders
The problem
I have a multi-channel 1D signal I want to auto-encode.
I am unable to resonstruct the input when the number of channels increases.
Code
I am using a convolutional encoder, and a ...
- 689
1
vote
0
answers
76
views
Underfitting a single batch: Can't cause autoencoder to overfit multi-sample batches of 1d data. How to debug?
TL;DR
I am unable to overfit batches with multiple samples using autoencoder.
Fully connected decoder seems to handle more samples per batch than conv decoder, but then also fails when number of ...
- 689
2
votes
1
answer
101
views
How does replacing states with latent representations help RL agents?
I have seen many papers using autoencoders to replace images (states) with latent representations. Some of those methods have shown higher rewards using such techniques. However, I do not understand ...
- 478
1
vote
0
answers
147
views
variational auto encoder loss goes down but does not reconstruct input. out of debugging ideas
My variational autoencoder seems to work for MNIST, but fails on slightly "harder" data.
By "fails" I mean there are at least two apparent problems:
Very poor reconstruction, for ...
- 689
2
votes
1
answer
418
views
VAE giving near zero output when latent space dimension is large
I'm training a VAE to reconstruct some input (channels picked up by some MIMO BS for context) and I ran an experiment on the training set to see how the performance improves with the latent space ...
- 21
2
votes
0
answers
32
views
Literature on the advantages of using an auto-encoder for classification
Given a supervised problem with X, y input pairs, one can do two things for obtaining the function f that maps X with y with Neural Networks (and in general in machine learning):
Deploy directly a ...