Questions tagged [autoencoders]

For questions about autoencoders, a type of unsupervised artificial network for learning efficient data codings.

Filter by
Sorted by
Tagged with
1
vote
1answer
84 views

Why does the training time of SVMs dramatically decrease after applying dimensionality reduction to the features?

Training an SVM with an RBF kernel model with c = 5.5 and gamma = 1.06, for a 5-class classification problem on the NSL-KDD ...
1
vote
1answer
420 views

What is the best activation function for the embedding layer in a deep auto-encoder?

I am designing a deep autoencoder for graph embedding (exactly node embedding) following this paper SDNE. In the original paper, they used the sigmoid activation for all hidden layers in the ...
1
vote
1answer
76 views

What should the output of a neural network that needs to classify in an unsupervised fashion XOR data be?

XOR data, without labels: [[0,0],[0,1],[1,0],[1,1]] I'm using this network for auto-classifying XOR data: ...
1
vote
1answer
47 views

Is it possible to have the latent vector of an auto-encoder with size 1?

Given e.g. 1M vectors of $1000$ floating points each, where every point in vectors is sampled from a uniform distribution between $-1$ to $1$: Is it possible to have the bottleneck of the AE network ...
1
vote
2answers
105 views

How fast are autoencoders?

I was exploring image/video compression using Machine Learning. In there I discovered that autoencoders are used very frequently for this sort of thing. So I wanted to enquire:- How fast are ...
1
vote
1answer
123 views

Can a variational auto-encoder learn images composed of random noise at each pixel (each drawn from the same distribution)?

Can a variational auto-encoder (VAE) learn images whose pixels have been generated from a Gaussian distribution (e.g. $N(0, 1)$), i.e. each pixel is a sample from $N(0, 1)$? My gut feeling says no, ...
1
vote
1answer
34 views

How to get top 5 movies recommendations from Auto-Encoder

I have trained a model using Auto-encoder on movielens dataset. Below is how i trained the model. ...
1
vote
1answer
42 views

What class of problem is this?

If I have a lot of input output pairs as training data <float Xi, float Yi> and I have a parametrized approximation function (I know the function algorithm, ...
1
vote
1answer
131 views

What is the best loss function for convolution neural network and autoencoder?

What is the best choice for loss function in Convolution Neural Network and in Autoencoder in particular - and why? I understand that the MSE is probably not the best choice, because little ...
1
vote
2answers
305 views

Why are VAE's useful?

I am not sure I understand what is the advantage of using a VAE's over a deterministic Auto Encoder? For example, assuming we have just 2 labels, a deterministic Auto Encoder will always map a given ...
1
vote
1answer
331 views

Autoencoder for color images in Keras backed by MXNet

I need to train an autoencoder in Keras with the JPG images I took myself. ...
1
vote
0answers
13 views

Why is the prior on the latent variable standard gaussian in VAE?

While training a standard VAE, we assume that the prior on the latent variable Z is the standard gaussian and we use KL divergence to push the posterior as close as possible to the standard gaussian. ...
1
vote
0answers
20 views

What is the conceptual difference between convolutional neural networks and auto-encoders?

I'm familiar with Auto-Encoders and I'm about to dive into CNNs. By having a look at the most important component of a CNN, the filter: I wonder how it is different from Auto-Encoders: For me, it ...
1
vote
0answers
27 views

How do I select the number of neurons for each layer in an auto-encoder for dimensionality reduction?

I am trying to apply an auto-encoder for dimensionality reduction. I wonder how it will be applied on a large dataset. I have tried this code below. I have total of 8 features in my data and I want to ...
1
vote
0answers
19 views

Dealing with bias in multi-channel auto encoders

The problem I have a multi-channel 1D signal I want to auto-encode. I am unable to resonstruct the input when the number of channels increases. Code I am using a convolutional encoder, and a ...
1
vote
0answers
22 views

Underfitting a single batch: Can't cause autoencoder to overfit multi-sample batches of 1d data. How to debug?

TL;DR I am unable to overfit batches with multiple samples using autoencoder. Fully connected decoder seems to handle more samples per batch than conv decoder, but then also fails when number of ...
1
vote
0answers
33 views

variational auto encoder loss goes down but does not reconstruct input. out of debugging ideas

My variational autoencoder seems to work for MNIST, but fails on slightly "harder" data. By "fails" I mean there are at least two apparent problems: Very poor reconstruction, for ...
1
vote
0answers
69 views

VAE giving near zero output when latent space dimension is large

I'm training a VAE to reconstruct some input (channels picked up by some MIMO BS for context) and I ran an experiment on the training set to see how the performance improves with the latent space ...
1
vote
1answer
67 views

Autoencoder: predictions missing for nodes in the bottleneck layer

I'm using tf.Keras to build a deep-fully connected autoencoder. My input dataset is a dataframe with shape (19947,), and the purpose of the autoencoder is to predict normalized gene expression values. ...
1
vote
0answers
28 views

How can I improve the performance on unseen data for semantic segmentation using an auto-encoder?

I am using simple autoencoders for the task of semantic segmentation on the VOC2012 dataset. I am currently using a simple autoencoder based model. It is trained on adam optimizer with cross-entropy ...
1
vote
0answers
22 views

Why do we add additional axis in CNN autoencoder while denoising?

I am currently learning about autoencoders and I follow https://www.tensorflow.org/tutorials/generative/autoencoder When denoising images, authors of tutorial add an additional axis to the data and I ...
1
vote
2answers
171 views

Is it possible to have a variable-length latent vector in an autoencoder?

I'm trying to have a simple autoencoder but with variable latent length (the network can produce variable latent lengths with respect to the complexity of the input), but I've not seen any related ...
1
vote
0answers
41 views

Role of autoencoder in Hierarchical Extreme Learning Machine

I want to build HELM neural network that consists of autoencoder (AE) and one class classification (OC). HELM with AE and OC have following shape: That is, hidden layer output of AE is input of OC. ...
1
vote
1answer
144 views

How to determine the number of hidden layers and units of a deep auto-encoder?

I am using a deep autoencoder for my problem. However, the way I choose the number of hidden layers and hidden units in a hidden layer is still based on my feeling. The size of the model that ...
1
vote
0answers
77 views

How can a de-noising auto-encoder act as an anomaly detection model?

In some research papers, I have seen that, for training the autoencoders, instead of giving the non-anomalous input images, they add some anomalies to the normal input images, and train the auto-...
1
vote
0answers
31 views

How much data do we need for making a successful de-noising auto-encoder?

Is there a guide how much data do you need for making successful denoising model using autoencoders? Or the rule is, the more data, the better it is? I tried with small dataset 350 samples, to see ...
1
vote
0answers
49 views

Does the reduction of the dimensions over multiple layers allow more details to be stored within the final representation?

From : https://debuggercafe.com/implementing-deep-autoencoder-in-pytorch/ the following autoencoder is defined ...
1
vote
0answers
39 views

How estimate the minimum size of an autoencoder to overfit the training data?

Given e.g. $1$M vectors of $1000$ floating points each, where every point in vectors is sampled from a uniform distribution between $-1$ to $1$, how to estimate the minimum network size required ...
1
vote
0answers
26 views

Can denoising auto-encoders be convolutional and fully connected?

I have been reading lately on autoencoders a lot. I just wanted to summarize my understanding of denoising autoencoders. As far as I understand they can be Fully connected (in which case, they will ...
1
vote
0answers
78 views

Why is my variational auto-encoder generating random noise?

This is my first variational autoencoder. Background info: I am using the MNIST digits dataset. The model is created and trained in PyTorch. The model is able to get a reasonably low loss, but the ...
1
vote
0answers
29 views

Reduce same sample distance in VAE encodings

I'm working on a beta VAE model learning a latent representation used as a similarity metric for image registration. One of the main problems I'm facing is that the encoder + sampler output doesn't ...
1
vote
0answers
14 views

Camera pose to environment Mapping

I would like to teach a model the environment of a room. I'm doing so by mapping a camera pose (x, y, z, q0, q1, q2, q3) to its corresponding image; where x, y, z represent location in Cartesian ...
1
vote
0answers
17 views

Limits for a bottleneck

I have some 64x64 pixels frames from a (simulated) video, with a spaceship moving on a fixed background. The spaceship moves in a straight line with constant velocity from left to right (along the x-...
1
vote
0answers
41 views

How to learn to sample?

Imagine you have access to a dataset of pairs $(s, \hat{\pi}(s))$ where s is a state in a high dimension continuous space $S$, $\pi(s)$ is a probabilistic distribution on a large discrete space $D$ (...
1
vote
0answers
88 views

Can we use Autoencoders for unsupervised CNN feature learning?

I searched through the internet but couldn't find a reliable article that answers this question. Can we use Autoencoders for unsupervised CNN feature learning of unlabeled images like the below and ...
1
vote
1answer
181 views

Equilateral and One-of-n encoding

I was reading AI For Humans Vol. 1 by Jeff Heaton when I came across the terms "equilateral encoding" and "one-of-n encoding." The explanations unfortunately made no sense to me and the reddit threads ...
0
votes
1answer
54 views

What are good parameters of an encoder?

I am trying to assess an encoder in my autoencoder. I can not seem to grasp which specs make an encoder better than other one in, lets say, unsupervised learning. For example, I am trying to teach my ...
0
votes
1answer
24 views

How to train a model for 1 image class to detect anomaly?

I want to train a model with python over the images, and these images are for a metal product. my aim is to detect the defects, to notice if a product is a failure. what kind of architecture do you ...
0
votes
2answers
94 views

In variational autoencoders, why do people use MSE for the loss?

In VAEs, we try to maximize the ELBO $\mathbb(E_q log\ p(x|z) + D_{KL}(q(z|x), p(z))$), but I see that many implement the first term as MSE of the image and it's reconstruction. Is this mathematically ...
0
votes
1answer
41 views

Why Autoencoder Weights Are Not Always Tied

To me, tying weights in an autoencoder makes sense if we think of the auto encoder as doing PCA. Why in any situation would it make sense to not tie the weights? If we don't tie the weights, would it ...
0
votes
1answer
25 views

Which type of feature extractor do you suggest to classify sensor data?

I have IMU (Inertial Measurment Unit- 6 axis) sensor data. The sensor attached on a car and 7 different drivers wipe on same path. I want to extract features and classify drivers. Which type of ...
0
votes
1answer
42 views

Can I use an autoencoder with high latent representational space?

I am trying to use a neural network to predict the next state output given the current state and action pairs. Both input and outputs are continuous variables. Due to the high dimensionality of each ...
0
votes
1answer
69 views

What can I do with an autoencoder? [duplicate]

I cannot find information in detail about autoencoder What can I do with an autoencoder (and how can I do this), practically speaking? What does the encoder (this part I think I understand) and a ...
0
votes
0answers
12 views

Train separate AutoEncoder's on each class or one AE for all classes to learn features?

I'm working on a project where the dataset contains time series of three classes, depending on the shape of the series. I want to learn the representations of these series as vectors, so naturally I ...
0
votes
0answers
67 views

Can I use the VAE for dimensionality reduction?

I'm doing a project that uses a clustering algorithm for the facial expression classification task. So, I use the output of the encoder in the VAE autoencoder for dimensionality reduction. However, I ...
0
votes
0answers
23 views

Problems while transforming a 2D Variational Autoencoder into a 1D Version

I am trying to addapt the Keras variational autoencoder (VAE) here from a 2-D input/output (matrix of a picture) to a 1-D input/output (just a vector). I thought this would be a fearly easy task, but ...
0
votes
0answers
27 views

Weird KL divergence behaviour

I'm training a complex model for motion prediction using a VAE, however the KL divergence has a very strange behavior. A scheleton of the network is the following: At the end my network compute the ...
0
votes
0answers
30 views

Why would an auto-encoder produce latent vectors with many zeros?

My autoencoder give latent vectors with many zeroes components like: ...
0
votes
0answers
29 views

Autoencoder on Sharp Images

I am training an autoencoder to reconstruct 3D images. This is going quite well apart from one slight issue. The images I wish to reconstruct are binary representations of organs. This means that they ...
0
votes
0answers
114 views

What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?

Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently. However, do they follow the same process during the inference phase, i.e., do they ...