Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [autoencoders]

For questions about autoencoders, a type of unsupervised artificial network for learning efficient data codings.

Filter by
Sorted by
Tagged with
0 votes
1 answer
39 views

How to reconstruct a new image using pre-trained autoencoder?

When a single image is assigned for training, an auto-encoder should be able to gradient-descend and find the full set of satisfactory weights that will reconstruct this image. Suppose a second image ...
James's user avatar
  • 145
0 votes
1 answer
43 views

model.fit fails using keras sequential (slice index ### of dimension 0 out of bounds) [closed]

this is the most simple model I can think of for my data yet I can't use the fit function, it gives an error. the desired procedure is to make a simple autoencoder : from 576 nodes to 64 then back to ...
Bikay's user avatar
  • 23
0 votes
0 answers
19 views

Why are various deep learning models unsupervised? [duplicate]

While studying the field of deep learning, the questions I had from the beginning have still not been resolved. In general, supervised learning is known to solve problems by comparing the results ...
장민규's user avatar
1 vote
0 answers
20 views

Is it reasonable to ask for the same time-regularity of the high and low dimensional signals?

Consider we are dealing with sequential data sampled from a continuous time signal $x(t)\in \mathbb{R}^n$, so that the dataset will look like $\{x_0,x_1,…,x_n\}$, with $x_i= x(t_i)$. Assume that we ...
user8354084's user avatar
1 vote
1 answer
41 views

Is "The Dimpled Manifold Hypothesis" correct to say this about autoencoders?

This quite famous paper states page 3 that: The (well-known) fact which underlies the new conceptual framework is that all the natural images are located on or near some low-dimensional manifold (as ...
Quersi's user avatar
  • 13
0 votes
1 answer
183 views

What are alternatives to PCA for time series data?

I have some data (20 stock price time series) and want to compare different approaches for dimensionality reduction other than PCA (I want to fit only 2 variables in my AR model). I've tried ...
J_Bake's user avatar
  • 11
1 vote
1 answer
66 views

Why aren't encoders decoders trivial?

If you have an encoder decoder with 10 input neurons for X then 3 hidden in one layer then another 10 in the output which are the same X is it not trivial to set the weights whatever you want and w1 ...
J_Bake's user avatar
  • 11
0 votes
0 answers
20 views

Does the fixed context in attention mechanism is accquired after getting the decoder hidden layer of the first hidden state?

here, the fixed context vector (ci) is used for the decoder model, why the decoder model also used by the attention weights. On the first (c1), does that mean the decoder does not have context ? (i = ...
Jeremy Kenn's user avatar
0 votes
0 answers
26 views

Is there a better way for my CNN to handle random values?

I made an autoencoder to, ideally, turn an image into seemingly random numbers(Using a loss that determines randomness) and turn those random numbers into the original image. The results were kind of ...
Nathanael Suarez's user avatar
0 votes
1 answer
58 views

Is it possible to build a convolutional autoencoder with fully connected bottleneck with low dimension?

I want to do a project with a small size image dataset (the size is about 50*50). There's another similar dataset, and I want to prove that the datasets are different. I built a convolutional ...
Kekai's user avatar
  • 3
1 vote
0 answers
34 views

Replicating conv autoencoder for anomaly detection, very blurry reconstructions

I’m trying to train an autoencoder on the hazelnut dataset of MVTec AD for reconstruction to detect anomalies. I’m am trying to replicate the results of this study: https://arxiv.org/pdf/2008.12977....
JeanMi's user avatar
  • 165
0 votes
0 answers
21 views

Is synthetic data just a placebo for immature models?

I apologize for the provocative question, but let me elaborate. I am trying to wrap my head around the logic of synthetic data. When you train a model what you are trying to do is to teach the ground ...
Pigna's user avatar
  • 295
0 votes
2 answers
401 views

From where do the Encoders in Transformers gets Input Embedding from?

In Transformers Encoders, from where do the Encoders get Input Embedding from? So when a sentence is given to a transformer-based model it first tokenises the sentence and each token is mapped with ...
Swastik's user avatar
  • 101
0 votes
0 answers
24 views

Seeking methods to incorporate arbitrary actuator faults for Control Optimization

I am working on a problem where a control method, backed by a Neural Network (NN), dictates the movement of a 1D actuator to influence a specific process. This actuator can move linearly within a set ...
IsolatedSushi's user avatar
0 votes
0 answers
7 views

autoencoders for anomaly detection, training individual models for different users or roles, how?

Do I first train a generic model for all of my users on a network, say for a network anomaly detection example, then fine tune for each user on their own subset of the training data? But I'd be using ...
mLstudent33's user avatar
0 votes
0 answers
62 views

Why does each row of data have the same bottleneck features in the Autoencoder after training?

I was training an autoencoder for anomaly detection and I wish to extract the bottleneck features of the encoder for K-NN. The model architecture is as such: ...
Aengus's user avatar
  • 1
2 votes
1 answer
106 views

Why can Variational Autoencoders (VAEs) approximate arbitrary distributions?

I am trying to reason to myself why is it that VAEs can approximate arbitrary probability distributions even though 𝑞𝜙(𝑧|𝑥) and 𝑝𝜃(𝑥|𝑧) are Gaussian. I understand that the parameters are ...
Joel's user avatar
  • 33
0 votes
0 answers
121 views

How do i approach creating a masked auto-encoder for feature extraction

I trained Masked Autoencoder-based models in order to use the encoder as a backbone for another task. Pretraining has been done in a Self-Supervised manner on image data. Now that it comes to my ...
Mitch's user avatar
  • 1
0 votes
0 answers
20 views

Analysis of the output samples from an autoencoder

I am conducting some experiments on an autoencoder as part of my research project. For our first experiment, we have a feedforward neural network (using pytorch), which is being given an input of ...
Darth_Vader's user avatar
0 votes
1 answer
23 views

How to learn Categorial Embeddings in Unsupervised Learning?

I want to cluster mixed-type tabular data, for the categorial columns I want to use Categorial Embeddings and then an Autoencoder Network before clustering with KMeans or similar. Now, when I want to ...
Jaanis's user avatar
  • 1
0 votes
0 answers
54 views

Is this a valid application of Autoencodeers/VAE?

I am trying to predict a spectrum (1D vector) from various scalar inputs which are known to be correlated. As the spectrum vector is very long (4000 points) it was suggested that I use dimensionality ...
Christopher McQueen's user avatar
0 votes
0 answers
10 views

What would be the best approach to resume longer texts in one word?

I am trying to create a model capable of resuming longer texts (my dataset has up to 140 words for each instance) in a single word (or multiple separate words). The idea is to synthetize positive or ...
Victor Ferreira's user avatar
0 votes
1 answer
141 views

For a transformer decoder, how exactly are K, Q, and V for each decoding step?

For a transformer decoder, how exactly are K, Q, and V for each decoding step? Assume my input prompt is "today is a" (good day). At t= 0 (generation step 0): K, Q, and V are the projections ...
wrek's user avatar
  • 183
0 votes
0 answers
24 views

How do I make an autoencoder and make it work on extracting the feature of a stationary wave?

I have a project to complete in a day, and I know that doing it in a day is a bit far-fetched. The problem is this - "Design an autoencoder with two neurons as the constriction, multiple hidden ...
Neeladri Reddy's user avatar
0 votes
1 answer
134 views

Can I implement a sklearn model inside a Pytorch nn.Module? [closed]

I am making a custom Pytorch model that at some point, clusters a latent space that was created by another, previous routine of the model (Autoencoder). In a bit more detail, my model is a regular ...
puradrogasincortar's user avatar
0 votes
1 answer
52 views

Why is the variational lower bound is easier to compute than the original marginal distribution?

Why is the ELBO of $p(x)=\int p(x|z)p(z)\mathrm{d}z$ easier to compute/estimate than the expression itself? Can we compute this quantity itself through sampling in the same way? I understanding that ...
Hanhan Li's user avatar
  • 101
1 vote
0 answers
585 views

Latent Diffusion Model Can't Learn the Latent Space of a VAE for the MNIST-Fashion Dataset

I'm currently playing around with LDMs on the MNIST-Fashion dataset. I thought the VQVAEs used in the original paper were a bit overkill for what I'm doing (and I don't fully understand how they ...
sb3's user avatar
  • 147
0 votes
1 answer
214 views

How does mixing and matching encoders and decoders work in image segmentation?

I had a conceptual questions regarding architectures. I am using this git hub repository that allows one to quickly put together a segmentation pipeline. In reading the readme one thing that has me ...
TheCodeNovice's user avatar
0 votes
1 answer
334 views

How to generate new data using VAE?

I have built the following function which takes as input some data and runs a VAE on them: ...
quant's user avatar
  • 101
0 votes
0 answers
46 views

Are autoencoders computationally cheaper than MLPs with the same number of neurons?

Are autoencoders computationally cheaper than other neural networks such as MLP with the same number of neurons? I have read in some papers that autoencoders train the network faster, and I could ...
Jesus M.'s user avatar
1 vote
0 answers
66 views

Can a convolution learn to generate fine details? [closed]

I'm trying to get a convolutional autoencoder to reconstruct images of a dataset with crisp details. I've read in a couple places that convolutional autoencoders "naturally produce blurry images&...
Soltius's user avatar
  • 271
3 votes
0 answers
1k views

Why VQ-VAE instead of VAE?

From the paper on VQ-VAE, it said that the vector quantized variational autoencoder (VQ-VAE), differs from VAEs in two key ways: the encoder network output discrete, rather than continuous, codecs ...
Nervous Hero's user avatar
2 votes
2 answers
6k views

What is an appropriate size for a latent space of (variational) autoencoders and how it varies with the features of the images?

I am training an autoencoder and a variational autoencoder using satellite and streetview images. I have tested my program on standard datasets such as MNIST and CelebA. It seems that the latent space ...
Qingyi Wang's user avatar
1 vote
0 answers
38 views

Pseudo Label Generation for Generative Cooperative Learning

I am trying to implement this paper for unsupervised video anomaly detection. The gist of the paper seems to be: Create a dataset for an unsupervised setting, by mixing up the train and anomalous ...
satan 29's user avatar
  • 141
0 votes
0 answers
83 views

Autoencoder make spectrogram important parts more pronounced with a "log loss"

Hi I want to create a neural network that essentially picks out the most pronounced parts of a spectrogram. Assume this is the True spectrogram: ...
GILO's user avatar
  • 101
1 vote
0 answers
13 views

Is it possible for PixelCNN to tell us what it generates?

I coded PixelCNN with the help of Keras official website. Also, I read the paper. I can use PixelCNN, similar to a decoder or generator (to generate samples). My question is, "is it possible to ...
Pouyan's user avatar
  • 39
0 votes
1 answer
116 views

Reconstructing 3D models from 2D images using autoencoders

I went through a research paper ("Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders") and tried to implement the approach following this diagram: ![...
arizona_3's user avatar
0 votes
1 answer
243 views

What is the distribution of autoencoder embeddings?

Is there any result on the distribution of autoencoder embeddings? For example, the following image (taken from this article) visualizes the latent space with t-SNE. As you can see, images from the ...
nalzok's user avatar
  • 321
0 votes
1 answer
143 views

Autoencoders: Where does the encoder end and the decoder begin?

Consider a simple Autoencoder neural net: ...
John Titor's user avatar
11 votes
2 answers
2k views

Is plain autoencoder a generative model?

I am wondering how a plain auto encoder is a generative model though its version might be but how can a plain auto encoder can be generative. I know that Vaes which is a version of the autoencoder is ...
Nervous Hero's user avatar
1 vote
0 answers
43 views

Is there a name for this model?

I have an image autoencoder model trained as follows: Step 1) train a GAN to obtain a generator capable of drawing from the data manifold by sampling a normal distribution in latent space Step 2) ...
user11305730's user avatar
-1 votes
1 answer
175 views

Denoise autoencoder not training properly [closed]

I'm trying to make a denoise autoencoder wherein the encoder part is vgg16 and decoder is opposite of vgg16(encoder) network. My dataset consists of 5K images in grayscale. Now while training, the ...
arizona_3's user avatar
1 vote
2 answers
949 views

Strange artifacts in autoencoder outputs

I'm training an autoencoder, that does not downsample images but processes them in the same size. For example, a 256x256 input will always be processed at 256x256 resolution, only the channels ...
Susmit Agrawal's user avatar
0 votes
0 answers
518 views

VAEs vs Autoencoders with BatchNorm and Dropout?

It struck me that regular auto-encoders with batch-norm and dropout have quite similar properties to VAEs which made me wonder whether VAEs where really much better than this simpler alternative. Let ...
profPlum's user avatar
  • 424
0 votes
1 answer
32 views

Do Quo et al (2013) perform backpropagation between layers?

Le et al. 2013's non-weight sharing CNN has inspired me to ask two questions on this site previously. When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) ...
Josiah Yoder's user avatar
1 vote
1 answer
816 views

What is the state of the art concerning autoencoder that connect 2 images that are not similar but are physicaly related?

I am currently working on an autoencoder that connect two images. The first one can be seen as the electron flow and the second one is the electrostatic potential seen by the electrons. Long story ...
Gaetan Percebois's user avatar
1 vote
1 answer
95 views

Why is training all layers at a time effective for a multi-layer autoencoder?

This training of all layers of a CNN simultaneously is standard practice today. It is found in every CNN (AlexNet (2012), VGG, Inception, GANs, etc) and even pre-CNN networks such as Le et al. 2012. ...
Josiah Yoder's user avatar
4 votes
2 answers
1k views

Can I apply reparametrization trick on "any" deep neural network?

I came across the "reparametrization trick" for the first time in the following paragraph from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning ...
hanugm's user avatar
  • 3,890
0 votes
0 answers
49 views

Masked Autoencoder Structure

In the following structure when we use MADE due to the constraints for making a masked autoencoder, it seems some inputs do not have any connection to the next layer, and also there is the output that ...
Farhad's user avatar
  • 1
4 votes
1 answer
376 views

How to determine the quality of synthetic data?

I'm working on a VAE model to produce synthetic data of X-Ray diffraction spectrums. I try to figure out how I can measure the quality of the spectrums. The goal would be to produce synthetic data ...
Tekay's user avatar
  • 43