Questions tagged [autoencoders]

For questions about autoencoders, a type of unsupervised artificial network for learning efficient data codings.

53 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
0answers
53 views

Are there transformer-based architectures that can produce fixed-length vector encodings given arbitrary-length text documents?

BERT encodes a piece of text such that each token (usually words) in the input text map to a vector in the encoding of the text. However, this makes the length of the encoding vary as a function of ...
4
votes
1answer
968 views

Adding a dense layer after a conv2d layer in a convolutional autoencoder

I am trying to implement a convolutional autoencoder with a dense layer at the bottleneck do to some dimensional reduction. I have seen two approaches for this which arent particularly scalable. The ...
4
votes
0answers
591 views

Sparsity constraint in a deep autoencoder

Is there any way and any reason why one would introduce a sparsity constraint on a deep autoencoder? In particular, in deep autoencoders the first layer often has more units than the dimensionality ...
3
votes
0answers
24 views

How to quantify the amount of information lost by the decoder NN in an AE?

Is there a way to quantify the amount of information lost in the lossy part of an autoencoder where the original input is compressed to a representation with less degrees of freedom? I was thinking ...
3
votes
0answers
37 views

Enforcing sparsity constraints that make use of spatial contiguity

I have a deep learning network that outputs grayscale image reconstructions. In addition to good reconstruction performance (measured through mean squared error or some other measure like psnr), I ...
3
votes
0answers
53 views

Looking for the proper algorithm to compress many lowres images of nearby locations

I have an optimization problem that I'm looking for the right algorithm to solve. What I have: A large set of low-res 360 images that were taken on a regular grid within a certain area. each of these ...
3
votes
1answer
220 views

How can auto-encoders compute the reconstruction error for the new data?

Autoencoders are used for unsupervised anomaly detection by first learning the features of the data set with mainly "normal" data points. Then new data can be considered anomalous if the new ...
2
votes
0answers
29 views

Literature on the advantages of using an auto-encoder for classification

Given a supervised problem with X, y input pairs, one can do two things for obtaining the function f that maps X with y with Neural Networks (and in general in machine learning): Deploy directly a ...
2
votes
0answers
27 views

Compressing Parameters of an Response System

I have an input-output system, which is fully determined by 256 parameters, of which I know a significant amount are of less importance to the input-output pattern. The data I have is some (64k in ...
2
votes
0answers
33 views

how to handle highly imbalanced multilabel classification?

I am working on a multilabel classification in which I am having 206 labels. When I saw the percentage of the number of 1's in each label they are way less than 0.1% for each label. The maximum ...
2
votes
1answer
60 views

Are Autoencoders for noise-reduction only suited to deal with salt-and-pepper kind of noise?

I'm currently looking at NN to deal with noisy data. I like the Autoencoder approach https://medium.com/@aliaksei.mikhailiuk/unsupervised-learning-for-data-interpolation-e259cf5dc957 because it seems ...
2
votes
2answers
103 views

How can I use autoencoders to analyze patterns and classify them?

I generated a bunch of simulation data from a complex physical simulation that spits out patterns. I am trying to apply unsupervised learning to analyze the patterns and ideally classify them into ...
2
votes
0answers
133 views

Is it possible to use entity embedding with autoencoder for anomaly detetction?

I'm trying to build autoencoder in keras in order to detect anomalies. However, most of the data is categorical and I have to encode it. When it comes to production, categorical features can take new ...
2
votes
0answers
403 views

How does deepfake technology work with multiple people in a single frame?

I was watching this video from corridor crew, according to them, they have used deepfake technology to create this video. I myself have never made a deepfake videos, but I have enough knowledge in the ...
2
votes
0answers
222 views

Are there any general tips for troubleshooting a VAE when apparently it is not learning?

I am trying to train a VAE for anomaly detection. I chose one architecture from this Github repository and I adjusted the input and output to match what I need. In my case, the input (and hence the ...
2
votes
1answer
98 views

What are some limitations of using Collaborative Deep learning for Recommender systems?

Recently I worked on a paper by Hao Wang, Collaborative Deep learning for Recommender Systems; which uses a two way tightly coupled method, Collaborative filtering for Item correlation and Stacked ...
2
votes
0answers
38 views

Autoencoder why it is special for image decoding?

I have read about auto encoder. Understood what is encoding part, and decoding part, and the latent space. Now, i tried to implement this in keras. Below is the code. ...
1
vote
0answers
19 views

What is the conceptual difference between convolutional neural networks and auto-encoders?

I'm familiar with Auto-Encoders and I'm about to dive into CNNs. By having a look at the most important component of a CNN, the filter: I wonder how it is different from Auto-Encoders: For me, it ...
1
vote
0answers
25 views

How do I select the number of neurons for each layer in an auto-encoder for dimensionality reduction?

I am trying to apply an auto-encoder for dimensionality reduction. I wonder how it will be applied on a large dataset. I have tried this code below. I have total of 8 features in my data and I want to ...
1
vote
0answers
19 views

Dealing with bias in multi-channel auto encoders

The problem I have a multi-channel 1D signal I want to auto-encode. I am unable to resonstruct the input when the number of channels increases. Code I am using a convolutional encoder, and a ...
1
vote
0answers
21 views

Underfitting a single batch: Can't cause autoencoder to overfit multi-sample batches of 1d data. How to debug?

TL;DR I am unable to overfit batches with multiple samples using autoencoder. Fully connected decoder seems to handle more samples per batch than conv decoder, but then also fails when number of ...
1
vote
0answers
32 views

variational auto encoder loss goes down but does not reconstruct input. out of debugging ideas

My variational autoencoder seems to work for MNIST, but fails on slightly "harder" data. By "fails" I mean there are at least two apparent problems: Very poor reconstruction, for ...
1
vote
0answers
64 views

VAE giving near zero output when latent space dimension is large

I'm training a VAE to reconstruct some input (channels picked up by some MIMO BS for context) and I ran an experiment on the training set to see how the performance improves with the latent space ...
1
vote
0answers
27 views

How can I improve the performance on unseen data for semantic segmentation using an auto-encoder?

I am using simple autoencoders for the task of semantic segmentation on the VOC2012 dataset. I am currently using a simple autoencoder based model. It is trained on adam optimizer with cross-entropy ...
1
vote
0answers
22 views

Why do we add additional axis in CNN autoencoder while denoising?

I am currently learning about autoencoders and I follow https://www.tensorflow.org/tutorials/generative/autoencoder When denoising images, authors of tutorial add an additional axis to the data and I ...
1
vote
0answers
36 views

Role of autoencoder in Hierarchical Extreme Learning Machine

I want to build HELM neural network that consists of autoencoder (AE) and one class classification (OC). HELM with AE and OC have following shape: That is, hidden layer output of AE is input of OC. ...
1
vote
0answers
77 views

How can a de-noising auto-encoder act as an anomaly detection model?

In some research papers, I have seen that, for training the autoencoders, instead of giving the non-anomalous input images, they add some anomalies to the normal input images, and train the auto-...
1
vote
0answers
31 views

How much data do we need for making a successful de-noising auto-encoder?

Is there a guide how much data do you need for making successful denoising model using autoencoders? Or the rule is, the more data, the better it is? I tried with small dataset 350 samples, to see ...
1
vote
0answers
48 views

Does the reduction of the dimensions over multiple layers allow more details to be stored within the final representation?

From : https://debuggercafe.com/implementing-deep-autoencoder-in-pytorch/ the following autoencoder is defined ...
1
vote
0answers
38 views

How estimate the minimum size of an autoencoder to overfit the training data?

Given e.g. $1$M vectors of $1000$ floating points each, where every point in vectors is sampled from a uniform distribution between $-1$ to $1$, how to estimate the minimum network size required ...
1
vote
0answers
26 views

Can denoising auto-encoders be convolutional and fully connected?

I have been reading lately on autoencoders a lot. I just wanted to summarize my understanding of denoising autoencoders. As far as I understand they can be Fully connected (in which case, they will ...
1
vote
0answers
76 views

Why is my variational auto-encoder generating random noise?

This is my first variational autoencoder. Background info: I am using the MNIST digits dataset. The model is created and trained in PyTorch. The model is able to get a reasonably low loss, but the ...
1
vote
0answers
29 views

Reduce same sample distance in VAE encodings

I'm working on a beta VAE model learning a latent representation used as a similarity metric for image registration. One of the main problems I'm facing is that the encoder + sampler output doesn't ...
1
vote
0answers
14 views

Camera pose to environment Mapping

I would like to teach a model the environment of a room. I'm doing so by mapping a camera pose (x, y, z, q0, q1, q2, q3) to its corresponding image; where x, y, z represent location in Cartesian ...
1
vote
0answers
17 views

Limits for a bottleneck

I have some 64x64 pixels frames from a (simulated) video, with a spaceship moving on a fixed background. The spaceship moves in a straight line with constant velocity from left to right (along the x-...
1
vote
0answers
41 views

How to learn to sample?

Imagine you have access to a dataset of pairs $(s, \hat{\pi}(s))$ where s is a state in a high dimension continuous space $S$, $\pi(s)$ is a probabilistic distribution on a large discrete space $D$ (...
1
vote
0answers
87 views

Can we use Autoencoders for unsupervised CNN feature learning?

I searched through the internet but couldn't find a reliable article that answers this question. Can we use Autoencoders for unsupervised CNN feature learning of unlabeled images like the below and ...
0
votes
0answers
31 views

Can I use the VAE for dimensionality reduction?

I'm doing a project that uses a clustering algorithm for the facial expression classification task. So, I use the output of the encoder in the VAE autoencoder for dimensionality reduction. However, I ...
0
votes
0answers
20 views

Problems while transforming a 2D Variational Autoencoder into a 1D Version

I am trying to addapt the Keras variational autoencoder (VAE) here from a 2-D input/output (matrix of a picture) to a 1-D input/output (just a vector). I thought this would be a fearly easy task, but ...
0
votes
0answers
27 views

Weird KL divergence behaviour

I'm training a complex model for motion prediction using a VAE, however the KL divergence has a very strange behavior. A scheleton of the network is the following: At the end my network compute the ...
0
votes
0answers
25 views

Why would an auto-encoder produce latent vectors with many zeros?

My autoencoder give latent vectors with many zeroes components like: ...
0
votes
0answers
27 views

Autoencoder on Sharp Images

I am training an autoencoder to reconstruct 3D images. This is going quite well apart from one slight issue. The images I wish to reconstruct are binary representations of organs. This means that they ...
0
votes
0answers
114 views

What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?

Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently. However, do they follow the same process during the inference phase, i.e., do they ...
0
votes
0answers
23 views

Compression of real time speech data

I am trying to train 2 neural networks - one to compress and the other to decompress real time speech data, and I have a few questions. First off Would this even work - trained, of course, on the same ...
0
votes
0answers
16 views

Can an Autoencoder be used to create a simple text-completion plugin (code-completion for programming)?

Can an Autoencoder neural network be used to create a simple code completion plugin for a developer using a certain programming language ? The idea is that the training data will be generated from ...
0
votes
0answers
29 views

Is the char-CNN-RNN text encoder is an encoder part of an auto-encoder?

The char-CNN-RNN encoder is relatively a popular encoder. It was proposed in this paper. Is it an encoder part of an autoencoder? Or an independent neural network?
0
votes
0answers
60 views

How to compare multiple one-class variational autoencoders?

I have trained multiple one-class vanilla variational autoencoders that each learn the distribution of one class and have the same architecture. The classes are mostly discrete, but there are several ...
0
votes
0answers
36 views

Which models can I use for supervised learning with images?

I have to do a project that detects fabric surface errors and I will use machine learning methods to deal with it. I have a dataset that includes around six thousand fabric surface images with the ...
0
votes
0answers
29 views

How did they use their dataset with VAEs?

Old Photo Restoration via Deep Latent Space Translation (https://paperswithcode.com/paper/old-photo-restoration-via-deep-latent-space) In the article, it says : "We propose to restore old photos ...
0
votes
0answers
45 views

Deep Continuous Clustering algorithm - just one output cluster

I use the DCC algorithm to cluster some data. The whole algorithm is available here, but shortly it is: construct mkNN graph of the data points (the connected components of it are the clusters). ...