Questions tagged [autoencoders]
For questions about autoencoders, a type of unsupervised artificial network for learning efficient data codings.
51
questions with no upvoted or accepted answers
7
votes
1
answer
175
views
Are there transformer-based architectures that can produce fixed-length vector encodings given arbitrary-length text documents?
BERT encodes a piece of text such that each token (usually words) in the input text map to a vector in the encoding of the text. However, this makes the length of the encoding vary as a function of ...
- 241
6
votes
0
answers
4k
views
Does it make sense to use batch normalization in deep (stacked) or sparse auto-encoders?
Does it make sense to use batch normalization in deep (stacked) or sparse auto-encoders?
I cannot find any resources for that. Is it safe to assume that, since it works for other DNNs, it will also ...
- 211
4
votes
0
answers
658
views
Is there any way and any reason why one would introduce a sparsity constraint on a deep auto-encoder?
Is there any way and any reason why one would introduce a sparsity constraint on a deep autoencoder?
In particular, in deep autoencoders, the first layer often has more units than the dimensionality ...
- 577
3
votes
0
answers
29
views
How to quantify the amount of information lost by the decoder NN in an AE?
Is there a way to quantify the amount of information lost in the lossy part of an autoencoder where the original input is compressed to a representation with less degrees of freedom?
I was thinking ...
- 261
3
votes
0
answers
47
views
Enforcing sparsity constraints that make use of spatial contiguity
I have a deep learning network that outputs grayscale image reconstructions. In addition to good reconstruction performance (measured through mean squared error or some other measure like psnr), I ...
- 143
3
votes
0
answers
56
views
Looking for the proper algorithm to compress many lowres images of nearby locations
I have an optimization problem that I'm looking for the right algorithm to solve.
What I have: A large set of low-res 360 images that were taken on a regular grid within a certain area. each of these ...
3
votes
0
answers
358
views
Are there any general tips for troubleshooting a VAE when apparently it is not learning?
I am trying to train a VAE for anomaly detection. I chose one architecture from this Github repository and I adjusted the input and output to match what I need. In my case, the input (and hence the ...
- 31
2
votes
1
answer
209
views
VAE giving near zero output when latent space dimension is large
I'm training a VAE to reconstruct some input (channels picked up by some MIMO BS for context) and I ran an experiment on the training set to see how the performance improves with the latent space ...
- 21
2
votes
0
answers
32
views
Literature on the advantages of using an auto-encoder for classification
Given a supervised problem with X, y input pairs, one can do two things for obtaining the function f that maps X with y with Neural Networks (and in general in machine learning):
Deploy directly a ...
2
votes
0
answers
31
views
Compressing Parameters of an Response System
I have an input-output system, which is fully determined by 256 parameters, of which I know a significant amount are of less importance to the input-output pattern.
The data I have is some (64k in ...
- 121
2
votes
1
answer
123
views
Are Autoencoders for noise-reduction only suited to deal with salt-and-pepper kind of noise?
I'm currently looking at NN to deal with noisy data. I like the Autoencoder approach https://medium.com/@aliaksei.mikhailiuk/unsupervised-learning-for-data-interpolation-e259cf5dc957 because it seems ...
2
votes
2
answers
215
views
How can I use autoencoders to analyze patterns and classify them?
I generated a bunch of simulation data from a complex physical simulation that spits out patterns. I am trying to apply unsupervised learning to analyze the patterns and ideally classify them into ...
- 71
2
votes
0
answers
613
views
How does deepfake technology work with multiple people in a single frame?
I was watching this video from corridor crew, according to them, they have used deepfake technology to create this video. I myself have never made a deepfake videos, but I have enough knowledge in the ...
- 946
2
votes
1
answer
121
views
What are some limitations of using Collaborative Deep learning for Recommender systems?
Recently I worked on a paper by Hao Wang, Collaborative Deep learning for Recommender Systems; which uses a two way tightly coupled method, Collaborative filtering for Item correlation and Stacked ...
- 303
1
vote
0
answers
40
views
Is there a name for this model?
I have an image autoencoder model trained as follows:
Step 1) train a GAN to obtain a generator capable of drawing from the data manifold by sampling a normal distribution in latent space
Step 2) ...
- 11
1
vote
1
answer
35
views
Why is training all layers at a time effective for a multi-layer autoencoder?
This training of all layers of a CNN simultaneously is standard practice today. It is found in every CNN (AlexNet (2012), VGG, Inception, GANs, etc) and even pre-CNN networks such as Le et al. 2012.
...
- 151
1
vote
0
answers
35
views
What is the conceptual difference between convolutional neural networks and auto-encoders?
I'm familiar with Auto-Encoders and I'm about to dive into CNNs. By having a look at the most important component of a CNN, the filter:
I wonder how it is different from Auto-Encoders:
For me, it ...
- 205
1
vote
0
answers
35
views
How do I select the number of neurons for each layer in an auto-encoder for dimensionality reduction?
I am trying to apply an auto-encoder for dimensionality reduction. I wonder how it will be applied on a large dataset.
I have tried this code below. I have total of 8 features in my data and I want to ...
- 111
1
vote
0
answers
20
views
Dealing with bias in multi-channel auto encoders
The problem
I have a multi-channel 1D signal I want to auto-encode.
I am unable to resonstruct the input when the number of channels increases.
Code
I am using a convolutional encoder, and a ...
- 669
1
vote
0
answers
53
views
Underfitting a single batch: Can't cause autoencoder to overfit multi-sample batches of 1d data. How to debug?
TL;DR
I am unable to overfit batches with multiple samples using autoencoder.
Fully connected decoder seems to handle more samples per batch than conv decoder, but then also fails when number of ...
- 669
1
vote
0
answers
57
views
variational auto encoder loss goes down but does not reconstruct input. out of debugging ideas
My variational autoencoder seems to work for MNIST, but fails on slightly "harder" data.
By "fails" I mean there are at least two apparent problems:
Very poor reconstruction, for ...
- 669
1
vote
0
answers
35
views
How can I improve the performance on unseen data for semantic segmentation using an auto-encoder?
I am using simple autoencoders for the task of semantic segmentation on the VOC2012 dataset. I am currently using a simple autoencoder based model. It is trained on adam optimizer with cross-entropy ...
- 121
1
vote
0
answers
51
views
how to handle highly imbalanced multilabel classification?
I am working on a multilabel classification in which I am having 206 labels. When I saw the percentage of the number of 1's in each label they are way less than 0.1% for each label. The maximum ...
- 47
1
vote
0
answers
73
views
Role of autoencoder in Hierarchical Extreme Learning Machine
I want to build HELM neural network that consists of autoencoder (AE) and one class classification (OC).
HELM with AE and OC have following shape:
That is, hidden layer output of AE is input of OC.
...
- 11
1
vote
0
answers
81
views
How can a de-noising auto-encoder act as an anomaly detection model?
In some research papers, I have seen that, for training the autoencoders, instead of giving the non-anomalous input images, they add some anomalies to the normal input images, and train the auto-...
- 11
1
vote
0
answers
39
views
How much data do we need for making a successful de-noising auto-encoder?
Is there a guide how much data do you need for making successful denoising model using autoencoders?
Or the rule is, the more data, the better it is?
I tried with small dataset 350 samples, to see ...
- 161
1
vote
0
answers
58
views
Does the reduction of the dimensions over multiple layers allow more details to be stored within the final representation?
From : https://debuggercafe.com/implementing-deep-autoencoder-in-pytorch/
the following autoencoder is defined
...
- 295
1
vote
0
answers
43
views
How estimate the minimum size of an autoencoder to overfit the training data?
Given e.g. $1$M vectors of $1000$ floating points each, where every point in vectors is sampled from a uniform distribution between $-1$ to $1$, how to estimate the minimum network size required ...
- 21
1
vote
0
answers
26
views
Can denoising auto-encoders be convolutional and fully connected?
I have been reading lately on autoencoders a lot. I just wanted to summarize my understanding of denoising autoencoders. As far as I understand they can be
Fully connected (in which case, they will ...
- 111
1
vote
0
answers
103
views
Why is my variational auto-encoder generating random noise?
This is my first variational autoencoder. Background info: I am using the MNIST digits dataset. The model is created and trained in PyTorch. The model is able to get a reasonably low loss, but the ...
- 11
1
vote
0
answers
31
views
Reduce same sample distance in VAE encodings
I'm working on a beta VAE model learning a latent representation used as a similarity metric for image registration.
One of the main problems I'm facing is that the encoder + sampler output doesn't ...
- 11
1
vote
0
answers
17
views
Camera pose to environment Mapping
I would like to teach a model the environment of a room. I'm doing so by mapping a camera pose (x, y, z, q0, q1, q2, q3) to its corresponding image; where x, y, z represent location in Cartesian ...
- 11
1
vote
0
answers
19
views
Limits for a bottleneck
I have some 64x64 pixels frames from a (simulated) video, with a spaceship moving on a fixed background. The spaceship moves in a straight line with constant velocity from left to right (along the x-...
- 225
1
vote
0
answers
41
views
How to learn to sample?
Imagine you have access to a dataset of pairs $(s, \hat{\pi}(s))$ where s is a state in a high dimension continuous space $S$, $\pi(s)$ is a probabilistic distribution on a large discrete space $D$ (...
- 111
1
vote
0
answers
95
views
Can we use Autoencoders for unsupervised CNN feature learning?
I searched through the internet but couldn't find a reliable article that answers this question.
Can we use Autoencoders for unsupervised CNN feature learning of unlabeled images like the below
and ...
0
votes
0
answers
9
views
Extracting behavior (switch On/Off) of an electric load from unlabeled time series data
Following are the details of my dataset:
sampling frequency: 1 Hz
No. of useful features: 10
The time series dataset is from household wherein I'm required to find ...
- 71
0
votes
1
answer
45
views
Reconstructing 3D models from 2D images using autoencoders
I went through a research paper ("Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders") and tried to implement the approach following this diagram:
0
votes
1
answer
31
views
Autoencoders: Where does the encoder end and the decoder begin?
Consider a simple Autoencoder neural net:
...
- 101
0
votes
0
answers
30
views
Why does the feature space of an autoencoder typically contain more info than a teacher-student model?
This is a question our Prof gave us as exam preparation, but I don't know why the Autoencoder should contain more info than the Teacher Student model.
Teacher Student Models are a class of models in ...
- 348
0
votes
0
answers
19
views
Do channels on the encoder's output layer really matter in convolutional autoencoders?
I viewed the example in Keras for a convolutional autoencoder. I noticed that the number of units in each layer of both the encoder and decoder never falls below that of the input. In other words, ...
- 53
0
votes
0
answers
59
views
VAEs vs Autoencoders with BatchNorm and Dropout?
It struck me that regular auto-encoders with batch-norm and dropout have quite similar properties to VAEs which made me wonder whether VAEs where really much better than this simpler alternative. Let ...
- 296
0
votes
1
answer
15
views
Do Quo et al (2013) perform backpropagation between layers?
Le et al. 2013's non-weight sharing CNN has inspired me to ask two questions on this site previously.
When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) ...
- 151
0
votes
1
answer
205
views
What is the state of the art concerning autoencoder that connect 2 images that are not similar but are physicaly related?
I am currently working on an autoencoder that connect two images. The first one can be seen as the electron flow and the second one is the electrostatic potential seen by the electrons. Long story ...
0
votes
0
answers
30
views
Masked Autoencoder Structure
In the following structure when we use MADE due to the constraints for making a masked autoencoder, it seems some inputs do not have any connection to the next layer, and also there is the output that ...
- 1
0
votes
0
answers
29
views
Adversarial Autoencoder is not working and not learning properly
I am trying to get an Adversarial AutoEncoder going using keras Fit method on a keras.model class but for some reason it is not working.
Keep in mind that I tried updating encoder and decoder at the ...
- 1
0
votes
0
answers
31
views
Train separate AutoEncoder's on each class or one AE for all classes to learn features?
I'm working on a project where the dataset contains time series of three classes, depending on the shape of the series. I want to learn the representations of these series as vectors, so naturally I ...
- 1
0
votes
0
answers
34
views
Weird KL divergence behaviour
I'm training a complex model for motion prediction using a VAE, however the KL divergence has a very strange behavior.
A scheleton of the network is the following:
At the end my network compute the ...
- 159
0
votes
0
answers
130
views
Why would an auto-encoder produce latent vectors with many zeros?
My autoencoder give latent vectors with many zeroes components like:
...
- 103
0
votes
0
answers
130
views
What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?
Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently.
However, do they follow the same process during the inference phase, i.e., do they ...
- 73
0
votes
0
answers
186
views
What is the time complexity for training a single-hidden layer auto-encoder?
What is the time complexity for training a single-hidden layer auto-encoder, for 1 epoch?
You can assume that there are $n$ training examples, $m$ features, and $k$ neurons in the hidden layer, and ...
