5
$\begingroup$

Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements.

I would like to apply some augmentation techniques to reduce the feature vector dimension to speed up cosine similarity/euclidean distance matrix calculation.

I have already come up with the following two methods in my literature review:

  1. Principal Component Analysis (PCA) + Whitening
  2. Locality Search Hashing (LSH)

Are there more approaches to perform dimensionality reduction of feature vectors? If so, what are the pros/cons of each perhaps?

$\endgroup$

2 Answers 2

4
$\begingroup$

Dimensionality reduction could be achieved by using an Autoencoder Network, which learns a representation (or Encoding) for the input data. While training, the reduction side (Encoder) reduces the data to a lower-dimension and a reconstructing side (Decoder) tries to reconstruct the original input from the intermediate reduced encoding.

You could assign the encoder layer output ($L_i$) to a desired dimension (lower than that of the input). Once trained, $L_i$ could be used as a alternative representation of your input data in a lower feature-space, and can be used for further computations.

Autoencoder Architecture

$\endgroup$
3
  • 1
    $\begingroup$ Maybe this answer could be further improved if you link to a paper or implementation that shows the application of AE to reduce the dimensionality of feature vectors. If you consider images feature vectors, then, in a way, AE are commonly applied to reduce the dimensionality of images (or feature vectors), but what if the inputs are not images? $\endgroup$
    – nbro
    Jan 26, 2020 at 23:28
  • $\begingroup$ Is there any python library perform autoencoder efficiently? $\endgroup$ Feb 3, 2020 at 6:44
  • 1
    $\begingroup$ @FäridAlijani Not that I know of. However, designing one in Keras wouldn't be much of a task. The following Keras blog might help : blog.keras.io/building-autoencoders-in-keras.html $\endgroup$
    – s_bh
    Feb 3, 2020 at 21:18
2
$\begingroup$

Some examples of dimensionality reduction techniques:

Linear methods Non-linear methods Graph-based methods
("Network embedding")
PCA
CCA
ICA
SVD
LDA
NMF
Kernel PCA
GDA
Autoencoders
t-SNE
UMAP
MVU
Diffusion maps
Graph Autoencoders

Graph-based kernel PCA
(Isomap, LLE, Hessian LLE, Laplacian Eigenmaps)

Though there are many more.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .