Questions tagged [autoencoders]
For questions about autoencoders, a type of unsupervised artificial network for learning efficient data codings.
144
questions
0
votes
0
answers
13
views
Classical architecture of autoencoder neural networks
I want to try Autoencoders(AE) (not variational) with a PINNs. The AE will be to encode the points of the domain. Lets say the variable domain (time-space + parameter space) dimension is n where n is ...
- 1
0
votes
1
answer
36
views
Is it possible to build a convolutional autoencoder with fully connected bottleneck with low dimension?
I want to do a project with a small size image dataset (the size is about 50*50). There's another similar dataset, and I want to prove that the datasets are different. I built a convolutional ...
- 3
1
vote
0
answers
28
views
Replicating conv autoencoder for anomaly detection, very blurry reconstructions
I’m trying to train an autoencoder on the hazelnut dataset of MVTec AD for reconstruction to detect anomalies.
I’m am trying to replicate the results of this study:
https://arxiv.org/pdf/2008.12977....
- 165
0
votes
0
answers
18
views
Is synthetic data just a placebo for immature models?
I apologize for the provocative question, but let me elaborate.
I am trying to wrap my head around the logic of synthetic data.
When you train a model what you are trying to do is to teach the ground ...
- 295
0
votes
2
answers
205
views
From where do the Encoders in Transformers gets Input Embedding from?
In Transformers Encoders, from where do the Encoders get Input Embedding from?
So when a sentence is given to a transformer-based model it first tokenises the sentence and each token is mapped with ...
- 1
0
votes
0
answers
24
views
Seeking methods to incorporate arbitrary actuator faults for Control Optimization
I am working on a problem where a control method, backed by a Neural Network (NN), dictates the movement of a 1D actuator to influence a specific process. This actuator can move linearly within a set ...
0
votes
0
answers
6
views
autoencoders for anomaly detection, training individual models for different users or roles, how?
Do I first train a generic model for all of my users on a network, say for a network anomaly detection example, then fine tune for each user on their own subset of the training data? But I'd be using ...
- 115
0
votes
0
answers
60
views
Why does each row of data have the same bottleneck features in the Autoencoder after training?
I was training an autoencoder for anomaly detection and I wish to extract the bottleneck features of the encoder for K-NN. The model architecture is as such:
...
- 1
2
votes
1
answer
80
views
Why can Variational Autoencoders (VAEs) approximate arbitrary distributions?
I am trying to reason to myself why is it that VAEs can approximate arbitrary probability distributions even though 𝑞𝜙(𝑧|𝑥)
and 𝑝𝜃(𝑥|𝑧)
are Gaussian.
I understand that the parameters are ...
- 23
0
votes
0
answers
55
views
How do i approach creating a masked auto-encoder for feature extraction
I trained Masked Autoencoder-based models in order to use the encoder as a backbone for another task. Pretraining has been done in a Self-Supervised manner on image data. Now that it comes to my ...
- 1
0
votes
0
answers
17
views
Analysis of the output samples from an autoencoder
I am conducting some experiments on an autoencoder as part of my research project. For our first experiment, we have a feedforward neural network (using pytorch), which is being given an input of ...
0
votes
1
answer
23
views
How to learn Categorial Embeddings in Unsupervised Learning?
I want to cluster mixed-type tabular data, for the categorial columns I want to use Categorial Embeddings and then an Autoencoder Network before clustering with KMeans or similar.
Now, when I want to ...
- 1
0
votes
0
answers
39
views
Is this a valid application of Autoencodeers/VAE?
I am trying to predict a spectrum (1D vector) from various scalar inputs which are known to be correlated. As the spectrum vector is very long (4000 points) it was suggested that I use dimensionality ...
0
votes
0
answers
10
views
What would be the best approach to resume longer texts in one word?
I am trying to create a model capable of resuming longer texts (my dataset has up to 140 words for each instance) in a single word (or multiple separate words). The idea is to synthetize positive or ...
- 101
0
votes
1
answer
89
views
For a transformer decoder, how exactly are K, Q, and V for each decoding step?
For a transformer decoder, how exactly are K, Q, and V for each decoding step?
Assume my input prompt is "today is a" (good day).
At t= 0 (generation step 0):
K, Q, and V are the projections ...
- 183
0
votes
0
answers
23
views
How do I make an autoencoder and make it work on extracting the feature of a stationary wave?
I have a project to complete in a day, and I know that doing it in a day is a bit far-fetched. The problem is this -
"Design an autoencoder with two neurons as the constriction, multiple hidden ...
- 101
0
votes
1
answer
112
views
Can I implement a sklearn model inside a Pytorch nn.Module? [closed]
I am making a custom Pytorch model that at some point, clusters a latent space that was created by another, previous routine of the model (Autoencoder).
In a bit more detail, my model is a regular ...
0
votes
0
answers
13
views
Drastic Change in MSE when Scaling Factor Changes
I am training an autoencoder meant to detect anomalies. Initially I scaled my data using a min-max scaler. I realized that this scalar isn't the best because anomalies can cause bias in the scalar. ...
- 113
0
votes
1
answer
48
views
Why is the variational lower bound is easier to compute than the original marginal distribution?
Why is the ELBO of $p(x)=\int p(x|z)p(z)\mathrm{d}z$ easier to compute/estimate than the expression itself? Can we compute this quantity itself through sampling in the same way? I understanding that ...
- 101
0
votes
0
answers
36
views
Mean and std in vaerational autoencoder
Are mean and standard deviation in variational autoencoders equal?
If not, then why are both calculated in the same way?
1
vote
0
answers
395
views
Latent Diffusion Model Can't Learn the Latent Space of a VAE for the MNIST-Fashion Dataset
I'm currently playing around with LDMs on the MNIST-Fashion dataset. I thought the VQVAEs used in the original paper were a bit overkill for what I'm doing (and I don't fully understand how they ...
- 137
0
votes
1
answer
107
views
How does mixing and matching encoders and decoders work in image segmentation?
I had a conceptual questions regarding architectures. I am using this git hub repository that allows one to quickly put together a segmentation pipeline. In reading the readme one thing that has me ...
- 103
0
votes
1
answer
193
views
How to generate new data using VAE?
I have built the following function which takes as input some data and runs a VAE on them:
...
- 101
0
votes
0
answers
43
views
Are autoencoders computationally cheaper than MLPs with the same number of neurons?
Are autoencoders computationally cheaper than other neural networks such as MLP with the same number of neurons?
I have read in some papers that autoencoders train the network faster, and I could ...
- 1
0
votes
0
answers
20
views
Should I pre-compute embeddings from a frozen network or are the gradients important?
I'm training an AutoEncoder-like network to take a face embedding, encode it, decode it, and then I calculate the loss between the input and output embedding. The input embedding is calculated by ...
- 125
1
vote
0
answers
38
views
Can a convolution learn to generate fine details? [closed]
I'm trying to get a convolutional autoencoder to reconstruct images of a dataset with crisp details.
I've read in a couple places that convolutional autoencoders "naturally produce blurry images&...
- 221
3
votes
0
answers
911
views
Why VQ-VAE instead of VAE?
From the paper on VQ-VAE, it said that the vector quantized variational autoencoder (VQ-VAE), differs from VAEs in two key ways:
the encoder network output discrete, rather than continuous, codecs
...
- 155
2
votes
2
answers
3k
views
What is an appropriate size for a latent space of (variational) autoencoders and how it varies with the features of the images?
I am training an autoencoder and a variational autoencoder using satellite and streetview images. I have tested my program on standard datasets such as MNIST and CelebA. It seems that the latent space ...
- 23
1
vote
0
answers
36
views
Pseudo Label Generation for Generative Cooperative Learning
I am trying to implement this paper for unsupervised video anomaly detection.
The gist of the paper seems to be:
Create a dataset for an unsupervised setting, by mixing up the train and anomalous ...
- 141
0
votes
0
answers
77
views
Autoencoder make spectrogram important parts more pronounced with a "log loss"
Hi I want to create a neural network that essentially picks out the most pronounced parts of a spectrogram.
Assume this is the True spectrogram:
...
- 101
1
vote
0
answers
11
views
Is it possible for PixelCNN to tell us what it generates?
I coded PixelCNN with the help of Keras official website. Also, I read the paper. I can use PixelCNN, similar to a decoder or generator (to generate samples). My question is, "is it possible to ...
- 29
0
votes
1
answer
102
views
Reconstructing 3D models from 2D images using autoencoders
I went through a research paper ("Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders") and tried to implement the approach following this diagram:
0
votes
1
answer
211
views
What is the distribution of autoencoder embeddings?
Is there any result on the distribution of autoencoder embeddings?
For example, the following image (taken from this article) visualizes the latent space with t-SNE. As you can see, images from the ...
- 271
0
votes
1
answer
134
views
Autoencoders: Where does the encoder end and the decoder begin?
Consider a simple Autoencoder neural net:
...
- 101
9
votes
2
answers
2k
views
Is plain autoencoder a generative model?
I am wondering how a plain auto encoder is a generative model though its version might be but how can a plain auto encoder can be generative. I know that Vaes which is a version of the autoencoder is ...
- 155
1
vote
0
answers
42
views
Is there a name for this model?
I have an image autoencoder model trained as follows:
Step 1) train a GAN to obtain a generator capable of drawing from the data manifold by sampling a normal distribution in latent space
Step 2) ...
- 11
-1
votes
1
answer
151
views
Denoise autoencoder not training properly [closed]
I'm trying to make a denoise autoencoder wherein the encoder part is vgg16 and decoder is opposite of vgg16(encoder) network. My dataset consists of 5K images in grayscale.
Now while training, the ...
1
vote
2
answers
778
views
Strange artifacts in autoencoder outputs
I'm training an autoencoder, that does not downsample images but processes them in the same size. For example, a 256x256 input will always be processed at 256x256 resolution, only the channels ...
- 125
0
votes
0
answers
404
views
VAEs vs Autoencoders with BatchNorm and Dropout?
It struck me that regular auto-encoders with batch-norm and dropout have quite similar properties to VAEs which made me wonder whether VAEs where really much better than this simpler alternative. Let ...
- 370
0
votes
1
answer
27
views
Do Quo et al (2013) perform backpropagation between layers?
Le et al. 2013's non-weight sharing CNN has inspired me to ask two questions on this site previously.
When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) ...
- 161
0
votes
1
answer
774
views
What is the state of the art concerning autoencoder that connect 2 images that are not similar but are physicaly related?
I am currently working on an autoencoder that connect two images. The first one can be seen as the electron flow and the second one is the electrostatic potential seen by the electrons. Long story ...
1
vote
1
answer
86
views
Why is training all layers at a time effective for a multi-layer autoencoder?
This training of all layers of a CNN simultaneously is standard practice today. It is found in every CNN (AlexNet (2012), VGG, Inception, GANs, etc) and even pre-CNN networks such as Le et al. 2012.
...
- 161
4
votes
2
answers
900
views
Can I apply reparametrization trick on "any" deep neural network?
I came across the "reparametrization trick" for the first time in the following paragraph from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning ...
- 3,622
0
votes
0
answers
48
views
Masked Autoencoder Structure
In the following structure when we use MADE due to the constraints for making a masked autoencoder, it seems some inputs do not have any connection to the next layer, and also there is the output that ...
- 1
3
votes
1
answer
324
views
How to determine the quality of synthetic data?
I'm working on a VAE model to produce synthetic data of X-Ray diffraction spectrums.
I try to figure out how I can measure the quality of the spectrums. The goal would be to produce synthetic data ...
- 33
1
vote
1
answer
249
views
Why is the prior on the latent variable standard gaussian in VAE?
While training a standard VAE, we assume that the prior on the latent variable Z is the standard gaussian and we use KL divergence to push the posterior as close as possible to the standard gaussian. ...
1
vote
2
answers
1k
views
Are mean and standard deviation in variational autoencoders unique?
In general, if I have a collection of data then mean(Expectation) and standard deviation are calculated as follows
$$\text{mean } = \mu = \mathbb{E}[X] = \sum\limits_{i = 1}^n p_ix_i $$
$$\text{...
- 3,622
0
votes
0
answers
122
views
Train separate AutoEncoder's on each class or one AE for all classes to learn features?
I'm working on a project where the dataset contains time series of three classes, depending on the shape of the series. I want to learn the representations of these series as vectors, so naturally I ...
- 1
1
vote
0
answers
47
views
What is the conceptual difference between convolutional neural networks and auto-encoders?
I'm familiar with Auto-Encoders and I'm about to dive into CNNs. By having a look at the most important component of a CNN, the filter:
I wonder how it is different from Auto-Encoders:
For me, it ...
- 205
0
votes
0
answers
50
views
Weird KL divergence behaviour
I'm training a complex model for motion prediction using a VAE, however the KL divergence has a very strange behavior.
A scheleton of the network is the following:
At the end my network compute the ...
- 199