Questions tagged [autoencoders]

For questions about autoencoders, a type of unsupervised artificial network for learning efficient data codings.

Filter by
Sorted by
Tagged with
1 vote
1 answer
13 views

Is plain autoencoder a generative model?

I am wondering how a plain auto encoder is a generative model though its version might be but how can a plain auto encoder can be generative. I know that Vaes which is a version of the autoencoder is ...
user avatar
1 vote
0 answers
37 views

Is there a name for this model?

I have an image autoencoder model trained as follows: Step 1) train a GAN to obtain a generator capable of drawing from the data manifold by sampling a normal distribution in latent space Step 2) ...
user avatar
-1 votes
1 answer
69 views

Denoise autoencoder not training properly [closed]

I'm trying to make a denoise autoencoder wherein the encoder part is vgg16 and decoder is opposite of vgg16(encoder) network. My dataset consists of 5K images in grayscale. Now while training, the ...
user avatar
0 votes
0 answers
22 views

Why does the feature space of an autoencoder typically contain more info than a teacher-student model?

This is a question our Prof gave us as exam preparation, but I don't know why the Autoencoder should contain more info than the Teacher Student model. Teacher Student Models are a class of models in ...
user avatar
  • 348
0 votes
0 answers
15 views

Do channels on the encoder's output layer really matter in convolutional autoencoders?

I viewed the example in Keras for a convolutional autoencoder. I noticed that the number of units in each layer of both the encoder and decoder never falls below that of the input. In other words, ...
user avatar
0 votes
2 answers
95 views

Strange artifacts in autoencoder outputs

I'm training an autoencoder, that does not downsample images but processes them in the same size. For example, a 256x256 input will always be processed at 256x256 resolution, only the channels ...
user avatar
0 votes
0 answers
38 views

VAEs vs Autoencoders with BatchNorm and Dropout?

It struck me that regular auto-encoders with batch-norm and dropout have quite similar properties to VAEs which made me wonder whether VAEs where really much better than this simpler alternative. Let ...
user avatar
  • 276
0 votes
1 answer
14 views

Do Quo et al (2013) perform backpropagation between layers?

Le et al. 2013's non-weight sharing CNN has inspired me to ask two questions on this site previously. When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) ...
user avatar
0 votes
1 answer
148 views

What is the state of the art concerning autoencoder that connect 2 images that are not similar but are physicaly related?

I am currently working on an autoencoder that connect two images. The first one can be seen as the electron flow and the second one is the electrostatic potential seen by the electrons. Long story ...
user avatar
1 vote
1 answer
34 views

Why is training all layers at a time effective for a multi-layer autoencoder?

This training of all layers of a CNN simultaneously is standard practice today. It is found in every CNN (AlexNet (2012), VGG, Inception, GANs, etc) and even pre-CNN networks such as Le et al. 2012. ...
user avatar
4 votes
2 answers
115 views

Can I apply reparametrization trick on "any" deep neural network?

I came across the "reparametrization trick" for the first time in the following paragraph from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning ...
user avatar
  • 3,099
0 votes
0 answers
28 views

Masked Autoencoder Structure

In the following structure when we use MADE due to the constraints for making a masked autoencoder, it seems some inputs do not have any connection to the next layer, and also there is the output that ...
user avatar
  • 1
0 votes
0 answers
28 views

Adversarial Autoencoder is not working and not learning properly

I am trying to get an Adversarial AutoEncoder going using keras Fit method on a keras.model class but for some reason it is not working. Keep in mind that I tried updating encoder and decoder at the ...
user avatar
  • 1
3 votes
1 answer
125 views

How to determine the quality of synthetic data?

I'm working on a VAE model to produce synthetic data of X-Ray diffraction spectrums. I try to figure out how I can measure the quality of the spectrums. The goal would be to produce synthetic data ...
user avatar
  • 33
1 vote
1 answer
35 views

Why is the prior on the latent variable standard gaussian in VAE?

While training a standard VAE, we assume that the prior on the latent variable Z is the standard gaussian and we use KL divergence to push the posterior as close as possible to the standard gaussian. ...
user avatar
1 vote
2 answers
245 views

Are mean and standard deviation in variational autoencoders unique?

In general, if I have a collection of data then mean(Expectation) and standard deviation are calculated as follows $$\text{mean } = \mu = \mathbb{E}[X] = \sum\limits_{i = 1}^n p_ix_i $$ $$\text{...
user avatar
  • 3,099
0 votes
0 answers
27 views

Train separate AutoEncoder's on each class or one AE for all classes to learn features?

I'm working on a project where the dataset contains time series of three classes, depending on the shape of the series. I want to learn the representations of these series as vectors, so naturally I ...
user avatar
1 vote
0 answers
32 views

What is the conceptual difference between convolutional neural networks and auto-encoders?

I'm familiar with Auto-Encoders and I'm about to dive into CNNs. By having a look at the most important component of a CNN, the filter: I wonder how it is different from Auto-Encoders: For me, it ...
user avatar
  • 205
0 votes
0 answers
33 views

Weird KL divergence behaviour

I'm training a complex model for motion prediction using a VAE, however the KL divergence has a very strange behavior. A scheleton of the network is the following: At the end my network compute the ...
user avatar
  • 149
0 votes
0 answers
105 views

Why would an auto-encoder produce latent vectors with many zeros?

My autoencoder give latent vectors with many zeroes components like: ...
user avatar
1 vote
0 answers
32 views

How do I select the number of neurons for each layer in an auto-encoder for dimensionality reduction?

I am trying to apply an auto-encoder for dimensionality reduction. I wonder how it will be applied on a large dataset. I have tried this code below. I have total of 8 features in my data and I want to ...
user avatar
  • 111
0 votes
1 answer
30 views

How to train a model for 1 image class to detect anomaly?

I want to train a model with python over the images, and these images are for a metal product. my aim is to detect the defects, to notice if a product is a failure. what kind of architecture do you ...
user avatar
  • 1
0 votes
2 answers
818 views

In variational autoencoders, why do people use MSE for the loss?

In VAEs, we try to maximize the ELBO = $\mathbb{E}_q [\log\ p(x|z)] + D_{KL}(q(z \mid x), p(z))$, but I see that many implement the first term as the MSE of the image and its reconstruction. Here's a ...
user avatar
  • 129
0 votes
0 answers
128 views

What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?

Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently. However, do they follow the same process during the inference phase, i.e., do they ...
user avatar
  • 73
0 votes
1 answer
72 views

Convert LSTM univariate Autoencoder to multivariate Autoencoder

I have the following code snippet which takes in a single column of value i.e. 1 feature. How do I modify the LSTM model such that it accepts 3 features? ...
user avatar
0 votes
1 answer
79 views

Why Autoencoder Weights Are Not Always Tied

To me, tying weights in an autoencoder makes sense if we think of the auto encoder as doing PCA. Why in any situation would it make sense to not tie the weights? If we don't tie the weights, would it ...
user avatar
  • 116
1 vote
0 answers
20 views

Dealing with bias in multi-channel auto encoders

The problem I have a multi-channel 1D signal I want to auto-encode. I am unable to resonstruct the input when the number of channels increases. Code I am using a convolutional encoder, and a ...
user avatar
  • 669
1 vote
0 answers
44 views

Underfitting a single batch: Can't cause autoencoder to overfit multi-sample batches of 1d data. How to debug?

TL;DR I am unable to overfit batches with multiple samples using autoencoder. Fully connected decoder seems to handle more samples per batch than conv decoder, but then also fails when number of ...
user avatar
  • 669
2 votes
1 answer
88 views

How does replacing states with latent representations help RL agents?

I have seen many papers using autoencoders to replace images (states) with latent representations. Some of those methods have shown higher rewards using such techniques. However, I do not understand ...
user avatar
1 vote
0 answers
53 views

variational auto encoder loss goes down but does not reconstruct input. out of debugging ideas

My variational autoencoder seems to work for MNIST, but fails on slightly "harder" data. By "fails" I mean there are at least two apparent problems: Very poor reconstruction, for ...
user avatar
  • 669
1 vote
0 answers
172 views

VAE giving near zero output when latent space dimension is large

I'm training a VAE to reconstruct some input (channels picked up by some MIMO BS for context) and I ran an experiment on the training set to see how the performance improves with the latent space ...
user avatar
2 votes
0 answers
31 views

Literature on the advantages of using an auto-encoder for classification

Given a supervised problem with X, y input pairs, one can do two things for obtaining the function f that maps X with y with Neural Networks (and in general in machine learning): Deploy directly a ...
user avatar
0 votes
1 answer
117 views

Autoencoder: predictions missing for nodes in the bottleneck layer

I'm using tf.Keras to build a deep-fully connected autoencoder. My input dataset is a dataframe with shape (19947,), and the purpose of the autoencoder is to predict normalized gene expression values. ...
user avatar
  • 11
2 votes
0 answers
31 views

Compressing Parameters of an Response System

I have an input-output system, which is fully determined by 256 parameters, of which I know a significant amount are of less importance to the input-output pattern. The data I have is some (64k in ...
user avatar
  • 121
0 votes
2 answers
71 views

Which models can I use for supervised learning with images?

I have to do a project that detects fabric surface errors and I will use machine learning methods to deal with it. I have a dataset that includes around six thousand fabric surface images with the ...
user avatar
1 vote
0 answers
35 views

How can I improve the performance on unseen data for semantic segmentation using an auto-encoder?

I am using simple autoencoders for the task of semantic segmentation on the VOC2012 dataset. I am currently using a simple autoencoder based model. It is trained on adam optimizer with cross-entropy ...
user avatar
1 vote
1 answer
43 views

Why do we add additional axis in CNN autoencoder while denoising?

I am currently learning about autoencoders and I follow https://www.tensorflow.org/tutorials/generative/autoencoder When denoising images, authors of tutorial add an additional axis to the data and I ...
user avatar
1 vote
0 answers
48 views

how to handle highly imbalanced multilabel classification?

I am working on a multilabel classification in which I am having 206 labels. When I saw the percentage of the number of 1's in each label they are way less than 0.1% for each label. The maximum ...
user avatar
1 vote
2 answers
377 views

Is it possible to have a variable-length latent vector in an autoencoder?

I'm trying to have a simple autoencoder but with variable latent length (the network can produce variable latent lengths with respect to the complexity of the input), but I've not seen any related ...
user avatar
  • 430
1 vote
1 answer
99 views

Why does the training time of SVMs dramatically decrease after applying dimensionality reduction to the features?

Training an SVM with an RBF kernel model with c = 5.5 and gamma = 1.06, for a 5-class classification problem on the NSL-KDD ...
user avatar
1 vote
0 answers
67 views

Role of autoencoder in Hierarchical Extreme Learning Machine

I want to build HELM neural network that consists of autoencoder (AE) and one class classification (OC). HELM with AE and OC have following shape: That is, hidden layer output of AE is input of OC. ...
user avatar
  • 11
2 votes
1 answer
117 views

Are Autoencoders for noise-reduction only suited to deal with salt-and-pepper kind of noise?

I'm currently looking at NN to deal with noisy data. I like the Autoencoder approach https://medium.com/@aliaksei.mikhailiuk/unsupervised-learning-for-data-interpolation-e259cf5dc957 because it seems ...
user avatar
3 votes
0 answers
29 views

How to quantify the amount of information lost by the decoder NN in an AE?

Is there a way to quantify the amount of information lost in the lossy part of an autoencoder where the original input is compressed to a representation with less degrees of freedom? I was thinking ...
user avatar
  • 261
0 votes
0 answers
171 views

What is the time complexity for training a single-hidden layer auto-encoder?

What is the time complexity for training a single-hidden layer auto-encoder, for 1 epoch? You can assume that there are $n$ training examples, $m$ features, and $k$ neurons in the hidden layer, and ...
user avatar
3 votes
0 answers
46 views

Enforcing sparsity constraints that make use of spatial contiguity

I have a deep learning network that outputs grayscale image reconstructions. In addition to good reconstruction performance (measured through mean squared error or some other measure like psnr), I ...
user avatar
6 votes
1 answer
140 views

Are there transformer-based architectures that can produce fixed-length vector encodings given arbitrary-length text documents?

BERT encodes a piece of text such that each token (usually words) in the input text map to a vector in the encoding of the text. However, this makes the length of the encoding vary as a function of ...
user avatar
0 votes
1 answer
27 views

Which type of feature extractor do you suggest to classify sensor data?

I have IMU (Inertial Measurment Unit- 6 axis) sensor data. The sensor attached on a car and 7 different drivers wipe on same path. I want to extract features and classify drivers. Which type of ...
user avatar
  • 247
6 votes
2 answers
233 views

Why don't we use auto-encoders instead of GANs?

I have watched Stanford's lectures about artificial intelligence, I currently have one question: why don't we use autoencoders instead of GANs? Basically, what GAN does is it receives a random vector ...
user avatar
1 vote
1 answer
923 views

What is the best activation function for the embedding layer in a deep auto-encoder?

I am designing a deep autoencoder for graph embedding (exactly node embedding) following this paper SDNE. In the original paper, they used the sigmoid activation for all hidden layers in the ...
user avatar
1 vote
1 answer
405 views

How to determine the number of hidden layers and units of a deep auto-encoder?

I am using a deep autoencoder for my problem. However, the way I choose the number of hidden layers and hidden units in a hidden layer is still based on my feeling. The size of the model that ...
user avatar