I want to do a project with a small size image dataset (the size is about 50*50). There's another similar dataset, and I want to prove that the datasets are different. I built a convolutional autoencoder with fully connected bottleneck because I want to use the latent space which produced by encoder to get a scatter plot. I think if the scatter points are not in the same area, then it can said the similarity between the two datasets is not that high.

I tried many model architectures, and it just can’t perform well on validation data. I am wondering if there is any way to reduce the dimension to such a low level. Thanks!

Here's my model:

    self.conv1 = nn.Conv2d(1, 32, (2,2), 1,padding=1)
    self.conv2 = nn.Conv2d(32, 64, (2,2), 1,padding=1)
    self.conv3 = nn.Conv2d(64, 128, (2,2), 1,padding=1)
    self.conv4 = nn.Conv2d(128,256, (2,2), 1,padding=1)
    self.conv5 = nn.Conv2d(256, 128, (2,2), 1,padding=1)
    self.conv6 = nn.Conv2d(128, 32, (2,2), 1,padding=1)
    self.conv7 = nn.Conv2d(32, 8, (2,2), 2,padding=1)
    self.Flatten = nn.Flatten()
    self.lin1 = nn.Linear(8*30*30,512)
    self.lin2 = nn.Linear(512,128)
    self.lin3 = nn.Linear(128,10)
    self.lin4 = nn.Linear(10,128)
    self.lin5 = nn.Linear(128,512)
    self.lin6 = nn.Linear(512,8*30*30)
    #reshape here
    self.convtrans1 = nn.ConvTranspose2d(8, 32, (2,2), 2,padding=1)
    self.convtrans2 = nn.ConvTranspose2d(32, 128, (2,2), 1,padding=1)
    self.convtrans3 = nn.ConvTranspose2d(128, 256, (2,2), 1,padding=1)
    self.convtrans4 = nn.ConvTranspose2d(256, 128, (2,2), 1,padding=1)
    self.convtrans5 = nn.ConvTranspose2d(128, 64, (2,2), 1,padding=1)
    self.convtrans6 = nn.ConvTranspose2d(64, 32, (2,2), 1,padding=1)
    self.convtrans7 = nn.ConvTranspose2d(32, 1, (2,2), 1,padding=1)

1 Answer 1


In general, very simple datasets like MNIST and Fashion-MNIST can be encoded in just two dimensions, without sacrificing too much reconstruction quality.

For more complex data, this is often neither feasible nor advisable: even if you employ a very powerful encoder and decoder, the bottleneck is just too small to capture the relevant features (or hidden factors) that sufficiently describe your data. Usually, it happens that a very low bottleneck dimensionality can only capture something that is very simple and, therefore, also not very useful, which makes the reconstructions looks like random stuff or even just spurious patterns that are somewhat weakly correlated with your inputs.

I think the best way, is to apply dimensionality reduction methods (like UMAP or t-SNE) to a trained bottleneck to reduce its size to 2 or 3 dimensions such that you can inspect it nicely. As stated here, tSNE can behave quite arbitrarily due to its very random nature (and difficuly to tune the hyperparameters), and so you may end up visualizing some patters that, actually, are not in the data. For such reason I suggest using UMAP first, and recur to t-SNE only if you're not able to get good results.

I can also suggest that thinks like l2-normalization (during training) of the embeddings may help the subsequent dimensionality reduction.

Otherwise, a good way to measure similarity (or closeness) between samples is to train a Siamese network and then threshold the learned distance metric: have a look here. You can also use the triplet loss, or simply use a pre-trained VGG network in which you extract the 4096-d embedding.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .