Questions tagged [autoencoders]

For questions about autoencoders, a type of unsupervised artificial network for learning efficient data codings.

Filter by
Sorted by
Tagged with
2
votes
1answer
43 views

Are Autoencoders for noise-reduction only suited to deal with salt-and-pepper kind of noise?

I'm currently looking at NN to deal with noisy data. I like the Autoencoder approach https://medium.com/@aliaksei.mikhailiuk/unsupervised-learning-for-data-interpolation-e259cf5dc957 because it seems ...
0
votes
0answers
13 views

Can an Autoencoder be used to create a simple text-completion plugin (code-completion for programming)?

Can an Autoencoder neural network be used to create a simple code completion plugin for a developer using a certain programming language ? The idea is that the training data will be generated from ...
0
votes
1answer
33 views

Why Autoencoder Weights Are Not Always Tied

To me, tying weights in an autoencoder makes sense if we think of the auto encoder as doing PCA. Why in any situation would it make sense to not tie the weights? If we don't tie the weights, would it ...
3
votes
2answers
145 views

How can auto-encoders compute the reconstruction error for the new data?

Autoencoders are used for unsupervised anomaly detection by first learning the features of the data set with mainly "normal" data points. Then new data can be considered anomalous if the new ...
1
vote
0answers
16 views

Dealing with bias in multi-channel auto encoders

The problem I have a multi-channel 1D signal I want to auto-encode. I am unable to resonstruct the input when the number of channels increases. Code I am using a convolutional encoder, and a ...
-1
votes
0answers
14 views

sequential autoencoder: RNN decoding comparing to RNN with attention decoding

I've implemented an RNN-ecnoder-decoder model and the same model with attention mechanism. That is the RNN model: and that is the attentioned model: then I asked for a theoretical question: The ...
1
vote
0answers
17 views

Underfitting a single batch: Can't cause autoencoder to overfit multi-sample batches of 1d data. How to debug?

TL;DR I am unable to overfit batches with multiple samples using autoencoder. Fully connected decoder seems to handle more samples per batch than conv decoder, but then also fails when number of ...
2
votes
1answer
62 views

How does replacing states with latent representations help RL agents?

I have seen many papers using autoencoders to replace images (states) with latent representations. Some of those methods have shown higher rewards using such techniques. However, I do not understand ...
5
votes
1answer
434 views

How can genetic programming be used in the context of auto-encoders?

I am trying to understand how genetic programming can be used in the context of auto-encoders. Currently, I am going through 2 papers Training Feedforward Neural Networks Using Genetic Algorithms (a ...
1
vote
0answers
21 views

variational auto encoder loss goes down but does not reconstruct input. out of debugging ideas

My variational autoencoder seems to work for MNIST, but fails on slightly "harder" data. By "fails" I mean there are at least two apparent problems: Very poor reconstruction, for ...
1
vote
0answers
34 views

VAE giving near zero output when latent space dimension is large

I'm training a VAE to reconstruct some input (channels picked up by some MIMO BS for context) and I ran an experiment on the training set to see how the performance improves with the latent space ...
0
votes
0answers
18 views

Is the char-CNN-RNN text encoder is an encoder part of an auto-encoder?

The char-CNN-RNN encoder is relatively a popular encoder. It was proposed in this paper. Is it an encoder part of an autoencoder? Or an independent neural network?
1
vote
2answers
64 views

How can I use autoencoders to analyze patterns and classify them?

I generated a bunch of simulation data from a complex physical simulation that spits out patterns. I am trying to apply unsupervised learning to analyze the patterns and ideally classify them into ...
2
votes
1answer
94 views

What are some limitations of using Collaborative Deep learning for Recommender systems?

Recently I worked on a paper by Hao Wang, Collaborative Deep learning for Recommender Systems; which uses a two way tightly coupled method, Collaborative filtering for Item correlation and Stacked ...
0
votes
0answers
44 views

How to compare multiple one-class variational autoencoders?

I have trained multiple one-class vanilla variational autoencoders that each learn the distribution of one class and have the same architecture. The classes are mostly discrete, but there are several ...
2
votes
0answers
24 views

Literature on the advantages of using an auto-encoder for classification

Given a supervised problem with X, y input pairs, one can do two things for obtaining the function f that maps X with y with Neural Networks (and in general in machine learning): Deploy directly a ...
4
votes
1answer
474 views

Adding a dense layer after a conv2d layer in a convolutional autoencoder

I am trying to implement a convolutional autoencoder with a dense layer at the bottleneck do to some dimensional reduction. I have seen two approaches for this which arent particularly scalable. The ...
1
vote
1answer
45 views

Autoencoder: predictions missing for nodes in the bottleneck layer

I'm using tf.Keras to build a deep-fully connected autoencoder. My input dataset is a dataframe with shape (19947,), and the purpose of the autoencoder is to predict normalized gene expression values. ...
2
votes
0answers
27 views

Compressing Parameters of an Response System

I have an input-output system, which is fully determined by 256 parameters, of which I know a significant amount are of less importance to the input-output pattern. The data I have is some (64k in ...
4
votes
1answer
289 views

Is there a continuous conditional variational auto-encoder?

The Conditional Variational Autoencoder (CVAE), introduced in the paper Learning Structured Output Representation using Deep Conditional Generative Models (2015), is an extension of Variational ...
0
votes
0answers
35 views

Which models can I use for supervised learning with images?

I have to do a project that detects fabric surface errors and I will use machine learning methods to deal with it. I have a dataset that includes around six thousand fabric surface images with the ...
1
vote
0answers
22 views

How can I improve the performance on unseen data for semantic segmentation using an auto-encoder?

I am using simple autoencoders for the task of semantic segmentation on the VOC2012 dataset. I am currently using a simple autoencoder based model. It is trained on adam optimizer with cross-entropy ...
0
votes
1answer
121 views

What is the best activation function for the embedding layer in a deep auto-encoder?

I am designing a deep autoencoder for graph embedding (exactly node embedding) following this paper SDNE. In the original paper, they used the sigmoid activation for all hidden layers in the ...
1
vote
0answers
18 views

Why do we add additional axis in CNN autoencoder while denoising?

I am currently learning about autoencoders and I follow https://www.tensorflow.org/tutorials/generative/autoencoder When denoising images, authors of tutorial add an additional axis to the data and I ...
2
votes
0answers
25 views

how to handle highly imbalanced multilabel classification?

I am working on a multilabel classification in which I am having 206 labels. When I saw the percentage of the number of 1's in each label they are way less than 0.1% for each label. The maximum ...
1
vote
2answers
91 views

Is it possible to have a variable-length latent vector in an autoencoder?

I'm trying to have a simple autoencoder but with variable latent length (the network can produce variable latent lengths with respect to the complexity of the input), but I've not seen any related ...
0
votes
0answers
29 views

How did they use their dataset with VAEs?

Old Photo Restoration via Deep Latent Space Translation (https://paperswithcode.com/paper/old-photo-restoration-via-deep-latent-space) In the article, it says : "We propose to restore old photos ...
1
vote
1answer
72 views

Why does the training time of SVMs dramatically decrease after applying dimensionality reduction to the features?

Training an SVM with an RBF kernel model with c = 5.5 and gamma = 1.06, for a 5-class classification problem on the NSL-KDD ...
1
vote
0answers
19 views

Role of autoencoder in Hierarchical Extreme Learning Machine

I want to build HELM neural network that consists of autoencoder (AE) and one class classification (OC). HELM with AE and OC have following shape: That is, hidden layer output of AE is input of OC. ...
0
votes
0answers
28 views

Deep Continuous Clustering algorithm - just one output cluster

I use the DCC algorithm to cluster some data. The whole algorithm is available here, but shortly it is: construct mkNN graph of the data points (the connected components of it are the clusters). ...
1
vote
1answer
70 views

How to determine the number of hidden layers and units of a deep auto-encoder?

I am using a deep autoencoder for my problem. However, the way I choose the number of hidden layers and hidden units in a hidden layer is still based on my feeling. The size of the model that ...
6
votes
2answers
136 views

Why don't we use auto-encoders instead of GANs?

I have watched Stanford's lectures about artificial intelligence, I currently have one question: why don't we use autoencoders instead of GANs? Basically, what GAN does is it receives a random vector ...
3
votes
0answers
24 views

How to quantify the amount of information lost by the decoder NN in an AE?

Is there a way to quantify the amount of information lost in the lossy part of an autoencoder where the original input is compressed to a representation with less degrees of freedom? I was thinking ...
0
votes
0answers
50 views

What is the time complexity for training a single-hidden layer auto-encoder?

What is the time complexity for training a single-hidden layer auto-encoder, for 1 epoch? You can assume that there are $n$ training examples, $m$ features, and $k$ neurons in the hidden layer, and ...
1
vote
2answers
74 views

How fast are autoencoders?

I was exploring image/video compression using Machine Learning. In there I discovered that autoencoders are used very frequently for this sort of thing. So I wanted to enquire:- How fast are ...
4
votes
0answers
42 views

Are there transformer-based architectures that can produce fixed-length vector encodings given arbitrary-length text documents?

BERT encodes a piece of text such that each token (usually words) in the input text map to a vector in the encoding of the text. However, this makes the length of the encoding vary as a function of ...
3
votes
0answers
35 views

Enforcing sparsity constraints that make use of spatial contiguity

I have a deep learning network that outputs grayscale image reconstructions. In addition to good reconstruction performance (measured through mean squared error or some other measure like psnr), I ...
0
votes
1answer
24 views

Which type of feature extractor do you suggest to classify sensor data?

I have IMU (Inertial Measurment Unit- 6 axis) sensor data. The sensor attached on a car and 7 different drivers wipe on same path. I want to extract features and classify drivers. Which type of ...
3
votes
0answers
53 views

Looking for the proper algorithm to compress many lowres images of nearby locations

I have an optimization problem that I'm looking for the right algorithm to solve. What I have: A large set of low-res 360 images that were taken on a regular grid within a certain area. each of these ...
1
vote
0answers
77 views

How can a de-noising auto-encoder act as an anomaly detection model?

In some research papers, I have seen that, for training the autoencoders, instead of giving the non-anomalous input images, they add some anomalies to the normal input images, and train the auto-...
1
vote
1answer
67 views

What should the output of a neural network that needs to classify in an unsupervised fashion XOR data be?

XOR data, without labels: [[0,0],[0,1],[1,0],[1,1]] I'm using this network for auto-classifying XOR data: ...
3
votes
1answer
82 views

How to evaluate the performance of an autoencoder trained on image data?

I am training an autoencoder on (general) image data. I use binary crossentropy loss function, but it is not very informative when I want to evaluate the performance of my autoencoder. An obvious ...
2
votes
2answers
156 views

How can I have the same input and output shape in an auto-encoder?

I'm building a denoising autoencoder. I want to have the same input and output shape image. This is my architecture: ...
4
votes
1answer
109 views

Autoencoder produces repeated artifacts after convergence

As experiment, I have tried using an autoencoder to encode height data from the alps, however the decoded image is very pixellated after training for several hours as show in the image below. This ...
1
vote
0answers
31 views

How much data do we need for making a successful de-noising auto-encoder?

Is there a guide how much data do you need for making successful denoising model using autoencoders? Or the rule is, the more data, the better it is? I tried with small dataset 350 samples, to see ...
4
votes
1answer
102 views

Disentangled VAE doesn't reconstruct accurate grids

I am trying to implement the disentangled VAE model according to this link. I want to understand the architecture of this model in order to customize it later. As infrastructure, I have a Linux kernel ...
2
votes
1answer
44 views

What are the main differences between sparse autoencoders and convolution autoencoders?

What are the main differences and similarities between sparse autoencoders and convolution autoencoders? When should one be preferred over the other? What are their applications? (References are ...
1
vote
0answers
47 views

Does the reduction of the dimensions over multiple layers allow more details to be stored within the final representation?

From : https://debuggercafe.com/implementing-deep-autoencoder-in-pytorch/ the following autoencoder is defined ...
1
vote
0answers
38 views

How estimate the minimum size of an autoencoder to overfit the training data?

Given e.g. $1$M vectors of $1000$ floating points each, where every point in vectors is sampled from a uniform distribution between $-1$ to $1$, how to estimate the minimum network size required ...
1
vote
1answer
46 views

Is it possible to have the latent vector of an auto-encoder with size 1?

Given e.g. 1M vectors of $1000$ floating points each, where every point in vectors is sampled from a uniform distribution between $-1$ to $1$: Is it possible to have the bottleneck of the AE network ...