Questions tagged [autoencoders]

For questions about autoencoders, a type of unsupervised artificial network for learning efficient data codings.

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

Classical architecture of autoencoder neural networks

I want to try Autoencoders(AE) (not variational) with a PINNs. The AE will be to encode the points of the domain. Lets say the variable domain (time-space + parameter space) dimension is n where n is ...
L Maxime's user avatar
0 votes
1 answer
36 views

Is it possible to build a convolutional autoencoder with fully connected bottleneck with low dimension?

I want to do a project with a small size image dataset (the size is about 50*50). There's another similar dataset, and I want to prove that the datasets are different. I built a convolutional ...
Kekai's user avatar
  • 3
1 vote
0 answers
28 views

Replicating conv autoencoder for anomaly detection, very blurry reconstructions

I’m trying to train an autoencoder on the hazelnut dataset of MVTec AD for reconstruction to detect anomalies. I’m am trying to replicate the results of this study: https://arxiv.org/pdf/2008.12977....
JeanMi's user avatar
  • 165
0 votes
0 answers
18 views

Is synthetic data just a placebo for immature models?

I apologize for the provocative question, but let me elaborate. I am trying to wrap my head around the logic of synthetic data. When you train a model what you are trying to do is to teach the ground ...
Pigna's user avatar
  • 295
0 votes
2 answers
205 views

From where do the Encoders in Transformers gets Input Embedding from?

In Transformers Encoders, from where do the Encoders get Input Embedding from? So when a sentence is given to a transformer-based model it first tokenises the sentence and each token is mapped with ...
Swastik's user avatar
0 votes
0 answers
24 views

Seeking methods to incorporate arbitrary actuator faults for Control Optimization

I am working on a problem where a control method, backed by a Neural Network (NN), dictates the movement of a 1D actuator to influence a specific process. This actuator can move linearly within a set ...
IsolatedSushi's user avatar
0 votes
0 answers
6 views

autoencoders for anomaly detection, training individual models for different users or roles, how?

Do I first train a generic model for all of my users on a network, say for a network anomaly detection example, then fine tune for each user on their own subset of the training data? But I'd be using ...
mLstudent33's user avatar
0 votes
0 answers
60 views

Why does each row of data have the same bottleneck features in the Autoencoder after training?

I was training an autoencoder for anomaly detection and I wish to extract the bottleneck features of the encoder for K-NN. The model architecture is as such: ...
Aengus's user avatar
  • 1
2 votes
1 answer
80 views

Why can Variational Autoencoders (VAEs) approximate arbitrary distributions?

I am trying to reason to myself why is it that VAEs can approximate arbitrary probability distributions even though 𝑞𝜙(𝑧|𝑥) and 𝑝𝜃(𝑥|𝑧) are Gaussian. I understand that the parameters are ...
Joel's user avatar
  • 23
0 votes
0 answers
55 views

How do i approach creating a masked auto-encoder for feature extraction

I trained Masked Autoencoder-based models in order to use the encoder as a backbone for another task. Pretraining has been done in a Self-Supervised manner on image data. Now that it comes to my ...
Mitch's user avatar
  • 1
0 votes
0 answers
17 views

Analysis of the output samples from an autoencoder

I am conducting some experiments on an autoencoder as part of my research project. For our first experiment, we have a feedforward neural network (using pytorch), which is being given an input of ...
Darth_Vader's user avatar
0 votes
1 answer
23 views

How to learn Categorial Embeddings in Unsupervised Learning?

I want to cluster mixed-type tabular data, for the categorial columns I want to use Categorial Embeddings and then an Autoencoder Network before clustering with KMeans or similar. Now, when I want to ...
Jaanis's user avatar
  • 1
0 votes
0 answers
39 views

Is this a valid application of Autoencodeers/VAE?

I am trying to predict a spectrum (1D vector) from various scalar inputs which are known to be correlated. As the spectrum vector is very long (4000 points) it was suggested that I use dimensionality ...
Christopher McQueen's user avatar
0 votes
0 answers
10 views

What would be the best approach to resume longer texts in one word?

I am trying to create a model capable of resuming longer texts (my dataset has up to 140 words for each instance) in a single word (or multiple separate words). The idea is to synthetize positive or ...
Victor Ferreira's user avatar
0 votes
1 answer
89 views

For a transformer decoder, how exactly are K, Q, and V for each decoding step?

For a transformer decoder, how exactly are K, Q, and V for each decoding step? Assume my input prompt is "today is a" (good day). At t= 0 (generation step 0): K, Q, and V are the projections ...
wrek's user avatar
  • 183
0 votes
0 answers
23 views

How do I make an autoencoder and make it work on extracting the feature of a stationary wave?

I have a project to complete in a day, and I know that doing it in a day is a bit far-fetched. The problem is this - "Design an autoencoder with two neurons as the constriction, multiple hidden ...
Neeladri Reddy's user avatar
0 votes
1 answer
112 views

Can I implement a sklearn model inside a Pytorch nn.Module? [closed]

I am making a custom Pytorch model that at some point, clusters a latent space that was created by another, previous routine of the model (Autoencoder). In a bit more detail, my model is a regular ...
puradrogasincortar's user avatar
0 votes
0 answers
13 views

Drastic Change in MSE when Scaling Factor Changes

I am training an autoencoder meant to detect anomalies. Initially I scaled my data using a min-max scaler. I realized that this scalar isn't the best because anomalies can cause bias in the scalar. ...
theastronomist's user avatar
0 votes
1 answer
48 views

Why is the variational lower bound is easier to compute than the original marginal distribution?

Why is the ELBO of $p(x)=\int p(x|z)p(z)\mathrm{d}z$ easier to compute/estimate than the expression itself? Can we compute this quantity itself through sampling in the same way? I understanding that ...
Hanhan Li's user avatar
  • 101
0 votes
0 answers
36 views

Mean and std in vaerational autoencoder

Are mean and standard deviation in variational autoencoders equal? If not, then why are both calculated in the same way?
Hossein Goodarzi's user avatar
1 vote
0 answers
395 views

Latent Diffusion Model Can't Learn the Latent Space of a VAE for the MNIST-Fashion Dataset

I'm currently playing around with LDMs on the MNIST-Fashion dataset. I thought the VQVAEs used in the original paper were a bit overkill for what I'm doing (and I don't fully understand how they ...
sb3's user avatar
  • 137
0 votes
1 answer
107 views

How does mixing and matching encoders and decoders work in image segmentation?

I had a conceptual questions regarding architectures. I am using this git hub repository that allows one to quickly put together a segmentation pipeline. In reading the readme one thing that has me ...
TheCodeNovice's user avatar
0 votes
1 answer
193 views

How to generate new data using VAE?

I have built the following function which takes as input some data and runs a VAE on them: ...
quant's user avatar
  • 101
0 votes
0 answers
43 views

Are autoencoders computationally cheaper than MLPs with the same number of neurons?

Are autoencoders computationally cheaper than other neural networks such as MLP with the same number of neurons? I have read in some papers that autoencoders train the network faster, and I could ...
Jesus M.'s user avatar
0 votes
0 answers
20 views

Should I pre-compute embeddings from a frozen network or are the gradients important?

I'm training an AutoEncoder-like network to take a face embedding, encode it, decode it, and then I calculate the loss between the input and output embedding. The input embedding is calculated by ...
Richie Bendall's user avatar
1 vote
0 answers
38 views

Can a convolution learn to generate fine details? [closed]

I'm trying to get a convolutional autoencoder to reconstruct images of a dataset with crisp details. I've read in a couple places that convolutional autoencoders "naturally produce blurry images&...
Soltius's user avatar
  • 221
3 votes
0 answers
911 views

Why VQ-VAE instead of VAE?

From the paper on VQ-VAE, it said that the vector quantized variational autoencoder (VQ-VAE), differs from VAEs in two key ways: the encoder network output discrete, rather than continuous, codecs ...
Nervous Hero's user avatar
2 votes
2 answers
3k views

What is an appropriate size for a latent space of (variational) autoencoders and how it varies with the features of the images?

I am training an autoencoder and a variational autoencoder using satellite and streetview images. I have tested my program on standard datasets such as MNIST and CelebA. It seems that the latent space ...
Qingyi Wang's user avatar
1 vote
0 answers
36 views

Pseudo Label Generation for Generative Cooperative Learning

I am trying to implement this paper for unsupervised video anomaly detection. The gist of the paper seems to be: Create a dataset for an unsupervised setting, by mixing up the train and anomalous ...
satan 29's user avatar
  • 141
0 votes
0 answers
77 views

Autoencoder make spectrogram important parts more pronounced with a "log loss"

Hi I want to create a neural network that essentially picks out the most pronounced parts of a spectrogram. Assume this is the True spectrogram: ...
GILO's user avatar
  • 101
1 vote
0 answers
11 views

Is it possible for PixelCNN to tell us what it generates?

I coded PixelCNN with the help of Keras official website. Also, I read the paper. I can use PixelCNN, similar to a decoder or generator (to generate samples). My question is, "is it possible to ...
Pouyan's user avatar
  • 29
0 votes
1 answer
102 views

Reconstructing 3D models from 2D images using autoencoders

I went through a research paper ("Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders") and tried to implement the approach following this diagram: ![...
arizona_3's user avatar
0 votes
1 answer
211 views

What is the distribution of autoencoder embeddings?

Is there any result on the distribution of autoencoder embeddings? For example, the following image (taken from this article) visualizes the latent space with t-SNE. As you can see, images from the ...
nalzok's user avatar
  • 271
0 votes
1 answer
134 views

Autoencoders: Where does the encoder end and the decoder begin?

Consider a simple Autoencoder neural net: ...
John Titor's user avatar
9 votes
2 answers
2k views

Is plain autoencoder a generative model?

I am wondering how a plain auto encoder is a generative model though its version might be but how can a plain auto encoder can be generative. I know that Vaes which is a version of the autoencoder is ...
Nervous Hero's user avatar
1 vote
0 answers
42 views

Is there a name for this model?

I have an image autoencoder model trained as follows: Step 1) train a GAN to obtain a generator capable of drawing from the data manifold by sampling a normal distribution in latent space Step 2) ...
user11305730's user avatar
-1 votes
1 answer
151 views

Denoise autoencoder not training properly [closed]

I'm trying to make a denoise autoencoder wherein the encoder part is vgg16 and decoder is opposite of vgg16(encoder) network. My dataset consists of 5K images in grayscale. Now while training, the ...
arizona_3's user avatar
1 vote
2 answers
778 views

Strange artifacts in autoencoder outputs

I'm training an autoencoder, that does not downsample images but processes them in the same size. For example, a 256x256 input will always be processed at 256x256 resolution, only the channels ...
Susmit Agrawal's user avatar
0 votes
0 answers
404 views

VAEs vs Autoencoders with BatchNorm and Dropout?

It struck me that regular auto-encoders with batch-norm and dropout have quite similar properties to VAEs which made me wonder whether VAEs where really much better than this simpler alternative. Let ...
profPlum's user avatar
  • 370
0 votes
1 answer
27 views

Do Quo et al (2013) perform backpropagation between layers?

Le et al. 2013's non-weight sharing CNN has inspired me to ask two questions on this site previously. When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) ...
Josiah Yoder's user avatar
0 votes
1 answer
774 views

What is the state of the art concerning autoencoder that connect 2 images that are not similar but are physicaly related?

I am currently working on an autoencoder that connect two images. The first one can be seen as the electron flow and the second one is the electrostatic potential seen by the electrons. Long story ...
Gaetan Percebois's user avatar
1 vote
1 answer
86 views

Why is training all layers at a time effective for a multi-layer autoencoder?

This training of all layers of a CNN simultaneously is standard practice today. It is found in every CNN (AlexNet (2012), VGG, Inception, GANs, etc) and even pre-CNN networks such as Le et al. 2012. ...
Josiah Yoder's user avatar
4 votes
2 answers
900 views

Can I apply reparametrization trick on "any" deep neural network?

I came across the "reparametrization trick" for the first time in the following paragraph from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning ...
hanugm's user avatar
  • 3,622
0 votes
0 answers
48 views

Masked Autoencoder Structure

In the following structure when we use MADE due to the constraints for making a masked autoencoder, it seems some inputs do not have any connection to the next layer, and also there is the output that ...
Farhad's user avatar
  • 1
3 votes
1 answer
324 views

How to determine the quality of synthetic data?

I'm working on a VAE model to produce synthetic data of X-Ray diffraction spectrums. I try to figure out how I can measure the quality of the spectrums. The goal would be to produce synthetic data ...
Tekay's user avatar
  • 33
1 vote
1 answer
249 views

Why is the prior on the latent variable standard gaussian in VAE?

While training a standard VAE, we assume that the prior on the latent variable Z is the standard gaussian and we use KL divergence to push the posterior as close as possible to the standard gaussian. ...
Manish Kausik Hari Baskar's user avatar
1 vote
2 answers
1k views

Are mean and standard deviation in variational autoencoders unique?

In general, if I have a collection of data then mean(Expectation) and standard deviation are calculated as follows $$\text{mean } = \mu = \mathbb{E}[X] = \sum\limits_{i = 1}^n p_ix_i $$ $$\text{...
hanugm's user avatar
  • 3,622
0 votes
0 answers
122 views

Train separate AutoEncoder's on each class or one AE for all classes to learn features?

I'm working on a project where the dataset contains time series of three classes, depending on the shape of the series. I want to learn the representations of these series as vectors, so naturally I ...
Elise Le's user avatar
1 vote
0 answers
47 views

What is the conceptual difference between convolutional neural networks and auto-encoders?

I'm familiar with Auto-Encoders and I'm about to dive into CNNs. By having a look at the most important component of a CNN, the filter: I wonder how it is different from Auto-Encoders: For me, it ...
Ben's user avatar
  • 205
0 votes
0 answers
50 views

Weird KL divergence behaviour

I'm training a complex model for motion prediction using a VAE, however the KL divergence has a very strange behavior. A scheleton of the network is the following: At the end my network compute the ...
FraMan's user avatar
  • 199