# Tag Info

Accepted

### Loss jumps abruptly when I decay the learning rate with Adam optimizer in PyTorch

I see no reason why decaying learning rates should create the kinds of jumps in losses that you are observing. It should "slow down" how quickly you "move", which in the case of a loss that otherwise ...
• 9,794
Accepted

### Is plain autoencoder a generative model?

An autoencoder is not considered a generative model, because it only reconstructs the given input. You could use the decoder like a generative model by putting in different vectors. However, the ...
• 1,317
Accepted

### What is the difference between encoders and auto-encoders?

Theory Encoder In general, an Encoder is a mapping $f : X \rightarrow Y$ with $X$ Input Space and $Y$ Code Space In case of Neural Networks, it is a Generative Model hence a function which is able ...

### Why doesn't VAE suffer mode collapse?

With Generative Adversarial Networks, all the generator cares about is fooling the discriminator. There's no requirement to be clever, or exhaustive, or make efficient use of the input space. As long ...
• 278

### Is it possible for a neural network to be used to compress data?

The auto-encoder (AE) can be used to learn a compressed representation (a vectorised hash value) of each observation in the training dataset, $z$, which can then be used to later retrieve the original ...
• 37k

### What are the purposes of autoencoders?

It is important to think about what sort of patterns in the data are being represented. Suppose that you have a dataset of greyscale images, such that every image is a uniform intensity. As a human ...
• 169
Accepted

### What is an appropriate size for a latent space of (variational) autoencoders and how it varies with the features of the images?

You are asking about several things here and while related, solving one, will not necessarily "solve" your problem. Let's look at them separately: Optimal dimension of the latent space. ...
• 176

### Why is the variational auto-encoder's output blurred, while GANs output is crisp and has sharp edges?

The key is: VAE usually use a small latent dimension, the information of input is so hard to pass through this bottleneck, meanwhile it tries to minimize the loss with the batch of input data, you ...
• 49
Accepted

### Concrete example of latent variables and observables plugged into the Bayes' rule

Let's assume the probability distributions are Gaussian (or normal) distributions. In other words, in the Bayes' rule \begin{align} p(z|x)=\frac{p(x|z)p(z)}{p(x)} \tag{1}\label{1} \end{align} The ...
• 37k

### Why don't we use auto-encoders instead of GANs?

In fact, autoencoders are used for generative tasks. Have a look at Tutorial on Variational Autoencoders (VAEs). The coolest thing about VAE is that abstract features can be easily amplified or ...
Accepted

### Why don't we use auto-encoders instead of GANs?

Auto-encoders are widely used and maybe even more used than GANs (in fact, auto-encoders are older than GANs, although the main general idea behind GANs is quite old). For example, auto-encoders are ...
• 37k

### Strange artifacts in autoencoder outputs

That's a classic checkerboard artifact. I would guess you're using a CNN as encoder/decoder architectures, since is well known that convolution layers, especially at the upsampling phase, cause these ...
• 4,753

### What are the purposes of autoencoders?

A use case of autoencoders (in particular, of the decoder or generative model of the autoencoder) is to denoise the input. This type of autoencoders, called denoising autoencoders, take a partially ...
• 37k

### How should we choose the dimensions of the encoding layer in auto-encoders?

The number of dimensions is a hyperparameter of your model, and you should do a hyperparameter search, like with any other parameters. There's also a tradeoff between dimension and training speed, so ...

### Autoencoder produces repeated artifacts after convergence

Perhaps you are getting checkerboard artifacts Explained here, solutions involve changing the kernel and stride size to prevent them from being not divisible. Besides that, a solution could be to ...
Accepted

### How does replacing states with latent representations help RL agents?

In short, it is much easier for the agent to learn from a smaller dimensional state space. This is because the agent must also do representation learning; i.e. it must also infer what the state is ...
• 4,400
Accepted

### How to determine the quality of synthetic data?

Due to subjective nature, quantitative evaluation of synthetic images is difficult in general. However, there are metrics like Inception Score or FID score that are used for evaluation of generative ...
• 238

### Can autoencoders be used for supervised learning?

One such paper I know of and which I implemented is Semi-Supervised Learning using Ladder Networks . I quote here their description of the model: Our approach follows Valpola (2015), who proposed a ...
• 263
Accepted

### What are good parameters of an encoder?

I can not seem to grasp which specs make an encoder better than another one In general, in unsupervised settings, we want to learn the probability distribution of the data p(x) by some latent ...
• 364
Accepted

### Do Le et al. (2012) train all three autoencoder layers at a time, or just one?

The paper refers to layers and sub-layers, and clearly indicates that one layer includes all three sub-layers, so when they say they train all three layers simultaneously, they are talking about the ...
• 151

### Why isn't the Credit Card Fraud Detection dataset from Kaggle already balanced?

I believe that the idea is to have a similar ratio of fraud/"normal transaction" as to the ones that bank encounter on real life. If you balance it you will probably have a lot of false positive once ...
• 424

### What are the purposes of autoencoders?

PCA is a linear method that creates a transformation that is capable of changing the vectors projections (changing axis) Since PCA looks for the direction of maximum variance it usually have high ...
Accepted

### How should I detect an object in a camera image?

Is this a good approach? Will I have a lot of trouble with different backgrounds? A lot will depend on the nature of the backgrounds you have, and how well they encode/decode by themselves without ...
• 26.5k

### If I use MobileNetV2 for the encoder, can I use a different architecture for the decoder?

Other replies are commenting on the skip connections for a U-Net. I believe you want to exclude these skip connections from your auto-encoder. You say you want to use the auto-encoder for unsupervised ...
• 21

### Using ML to encypher data for production

It sounds like you are trying to compress data, and then recover the same data later. The most common tool for this task is an autoencoder. This model accepts data as input, and then learns to ...
• 9,037

### How are small scale features represented in an Inverse Graphics Network (autoencoder)?

Simply said, there is no specific "meaning" to the features generated. They are simply features that are fitted through math and calculus, and nobody knows what they represent exactly, and will never ...
• 1,725