Questions tagged [dimensionality-reduction]
For questions related to AI methods of dimensionality reduction (e.g. PCA or autoencoders).
20
questions
1
vote
1answer
22 views
Perform clustering on high dimensional data
Recently I trained a BYOL model on a set of images to learn an embedding space where similar vectors are close by. The performance was fantastic when I performed approximate K-nearest neighbours ...
0
votes
0answers
23 views
Why some encoders reduce dimensionality of data while some others increase it?
We know that the encoder part in VAE reduces dimensions of data. In some other papers related to neural radiance fields and 3d shape reconstruction like https://arxiv.org/pdf/2106.01553.pdf, the ...
1
vote
1answer
72 views
Is it generally advisable to have a low dimensional action space in Reinforcement Learning?
In supervised or unsupervised learning, it is advised to reduce the dimensionality due to the curse of dimensionality in general.
Is this also generally advisable for the action space of reinforcement ...
0
votes
0answers
111 views
Can I use the VAE for dimensionality reduction?
I'm doing a project that uses a clustering algorithm for the facial expression classification task. So, I use the output of the encoder in the VAE autoencoder for dimensionality reduction. However, I ...
1
vote
0answers
30 views
How do I select the number of neurons for each layer in an auto-encoder for dimensionality reduction?
I am trying to apply an auto-encoder for dimensionality reduction. I wonder how it will be applied on a large dataset.
I have tried this code below. I have total of 8 features in my data and I want to ...
0
votes
1answer
40 views
How does t-SNE preserves embedding orders?
According to the triplet loss Wikipedia page:
t-SNE (t-distributed Stochastic Neighbor Embedding) preserves embedding orders via probability distributions, whereas triplet loss works directly on ...
2
votes
1answer
162 views
Why does PCA of the vertices of a hexagon result in principal components of equal length?
I do PCA on the data points placed in the corners of a hexagon, and get the following principal components:
The PCA variance is $0.6$ and is the same for each component. Why is that? Shouldn't it be ...
1
vote
0answers
21 views
Does Linear Discriminant Analysis make dimensionality reduction before classification?
I'm trying to understand what LDA exactly does when used as a classifier. I've understood how the dimensionality reduction works and I've understood that the classification task is carried out with ...
3
votes
1answer
75 views
When using PCA for dimensionality reduction of the feature vectors to speed up learning, how do I know that I'm not letting the model overfit?
I'm following Andrew Ng's course for Machine Learning and I just don't quite understand the following.
Using PCA to speed up learning
Using PCA to reduce the number of features, thus lowering the ...
2
votes
0answers
49 views
Estimating dimensions to reduce input image size to in CNNs
Considering input images to a CNN that have a large dimension (e.g. 256X256), what are some possible methods to estimate the exact dimensions (e.g. 16X16 or 32X32) to which it can be condensed in the ...
2
votes
0answers
30 views
Compressing Parameters of an Response System
I have an input-output system, which is fully determined by 256 parameters, of which I know a significant amount are of less importance to the input-output pattern.
The data I have is some (64k in ...
3
votes
0answers
21 views
What is meant by subspace clustering in MFA?
The basic idea of MFA is to perform subspace clustering by assuming the covariance structure for each component of the form, $\Sigma_i = \Lambda_i \Lambda_i^T + \Psi_i$, where $\Lambda_i \in \mathbb{R}...
3
votes
1answer
140 views
How do AI researchers imagine higher dimensions?
We can visualize single, two, and three dimensions using websites or imagination.
In the context of AI and, in particular, machine learning, AI researchers often have to deal with multi-dimensional ...
1
vote
1answer
44 views
How classification neural nets are different from simple dimension reduction + clustering?
I know the training of neural nets involves some sort of dimension manipulation to separate classes of different features.
If there is no variation of features, no matter for neural nets or simple ...
1
vote
0answers
54 views
How to cluster data points such that the number of clusters is kept minimal and each cluster projects well onto a lower-dimensional subspace?
If I want to find a (linear) subspace onto which a data-set projects well, I can simply use PCA. However, often the data can project with much smaller error if I first separate it into a couple of ...
0
votes
2answers
60 views
Is it theoretically possible (or impossible) that principal component analysis worsens the performance of the model?
In case I had a prediction model and decided to add a PCA step prior to the model, is it theoretically possible/impossible that the number of output dimensions that is better for all tests may perform ...
2
votes
2answers
153 views
How can I use autoencoders to analyze patterns and classify them?
I generated a bunch of simulation data from a complex physical simulation that spits out patterns. I am trying to apply unsupervised learning to analyze the patterns and ideally classify them into ...
5
votes
2answers
87 views
What are examples of approaches to dimensionality reduction of feature vectors?
Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements.
I would like to apply some augmentation techniques to reduce the ...
2
votes
0answers
23 views
Clustering of very high dimensional data and large number of examples without losing info in dimensions
I'm trying to get a grasp on scalability of clustering algorithms, and have a toy example in mind. Let's say I have around a million or so songs from $50$ genres. Each song has characteristics - some ...
11
votes
4answers
3k views
What are the purposes of autoencoders?
Autoencoders are neural networks that learn a compressed representation of the input in order to later reconstruct it, so they can be used for dimensionality reduction. They are composed of an encoder ...
