Questions tagged [dimensionality-reduction]

For questions related to AI methods of dimensionality reduction (e.g. PCA or autoencoders).

Filter by
Sorted by
Tagged with
0 votes
0 answers
29 views

Why are rows of Attention Weights in a Hopfield Transformer the same?

I'm working on building a Hopfield Transformer using the github code from the paper (https://github.com/ml-jku/hopfield-layers/tree/master/hflayers) to forecast a timeseries dataset with 48 variables, ...
Ryan Bose-Roy's user avatar
0 votes
0 answers
9 views

Dimensions and file sizes of AI types (img/txt/sound) that work commercially and projections of future AI based on dimensionality and file simplicity?

Is it accurate that current AI breakthroughs are proportional to low dimensional complexity of datasets and small data throughput? Image and text file renders are 100's of kilobytes, and they are ...
bandybabboon's user avatar
-2 votes
1 answer
59 views

the best choice to reduce a features vector

i have 1200 features highly correlated , and i want to reduce those number of features so the best choice is use feature selection or dimensionality reduction? and which method is the best in this ...
myriamkach's user avatar
1 vote
1 answer
72 views

What is $\mathbf{S}$ (sample covariance matrix) in image compression based on PCA?

If the feature vector is $\mathbf{x} \in \mathbb{R}^{d}$, then to apply PCA we first need to construct the "sample covariance matrix) \begin{align*} \underbrace{\frac{1}{N}\sum_{i=1}^N(\mathbf{x}^...
DSPinfinity's user avatar
0 votes
0 answers
11 views

Why is k=1 in linear discriminant analysis for two classes?

With two classes, why does Linear Discriminant Analysis (LDA) consider only projecting onto one dimension (k=1)? Normally, even with 2 classes, you can consider projecting the d-dimensional original ...
DSPinfinity's user avatar
0 votes
0 answers
14 views

Why is $z$ considered a random variable in the following formulation of Locally Linear Embedding Problem?

The following is the derivation of "Locally Linear Embedding Problem" from the book Machine Learning, 4-th edt, page 152, by E.Alpaydin. Why is $z$ considered a random variable so that $E(z)...
DSPinfinity's user avatar
0 votes
0 answers
15 views

Why are projected variables in canonical correlation analysis uncorrelated?

Let $x\in R^d$ and $y\in R^e$ be two vectors with covariance and cross-covariance matrices $S_{xx}, S_{yy}, S_{xy}, S_{yx}$. The canonical correlation analysis is based on the projection of $x$ onto ...
DSPinfinity's user avatar
0 votes
0 answers
30 views

Why does in feature embedding the similarities between instances in the new space respects the original pairwise similarities?

Below is a statement from Machine Learning book, by E. Alpyadin, 4th edition, page 131: Question: Why does in feature embedding the similarities between instances in the new space respects the ...
DSPinfinity's user avatar
0 votes
0 answers
6 views

Dimensionality Reduction of a matrix preserving number of rows

Let A ($d \times k $) be a matrix such that k < d. How to reduce the dimension of the matrix A to another lower dimension matrix B ($d \times l$ ) such that $l < k$. Note that, while reducing ...
Rituraj Singh's user avatar
1 vote
0 answers
30 views

Dimensionality Limitations

I just started learning about AI and have been reading a book called "Foundations of Machine Learning" by Mehryar Mohri so that I can try to create my own. I had a question come up recently: ...
Sophia's user avatar
  • 11
0 votes
1 answer
66 views

Clustering by using Locality sensitive hashing *after* Random projection

It is well known that Random Projection (RP) is tightly linked to Locality Sensitive Hashing (LSH). My goal is to cluster a large number of points lying in a $d$-dimensional Euclidean space, where $d$ ...
Penelope Benenati's user avatar
2 votes
0 answers
114 views

How to reduce the dimensionality of the actions in RL

I have a single-agent RL model in which the dimension of the dimension of the action space is $70$. This action space is too big and the deep RL agent is not learning properly. The boundaries of the ...
Leibniz's user avatar
  • 69
2 votes
1 answer
321 views

Perform clustering on high dimensional data

Recently I trained a BYOL model on a set of images to learn an embedding space where similar vectors are close by. The performance was fantastic when I performed approximate K-nearest neighbours ...
VEDANT JOSHI's user avatar
1 vote
1 answer
457 views

Is it generally advisable to have a low dimensional action space in Reinforcement Learning?

In supervised or unsupervised learning, it is advised to reduce the dimensionality due to the curse of dimensionality in general. Is this also generally advisable for the action space of reinforcement ...
PeterBe's user avatar
  • 246
1 vote
0 answers
43 views

How do I select the number of neurons for each layer in an auto-encoder for dimensionality reduction?

I am trying to apply an auto-encoder for dimensionality reduction. I wonder how it will be applied on a large dataset. I have tried this code below. I have total of 8 features in my data and I want to ...
user12's user avatar
  • 111
0 votes
1 answer
116 views

How does t-SNE preserves embedding orders?

According to the triplet loss Wikipedia page: t-SNE (t-distributed Stochastic Neighbor Embedding) preserves embedding orders via probability distributions, whereas triplet loss works directly on ...
Revolucion for Monica's user avatar
2 votes
1 answer
243 views

Why does PCA of the vertices of a hexagon result in principal components of equal length?

I do PCA on the data points placed in the corners of a hexagon, and get the following principal components: The PCA variance is $0.6$ and is the same for each component. Why is that? Shouldn't it be ...
Vladislav Gladkikh's user avatar
1 vote
0 answers
30 views

Does Linear Discriminant Analysis make dimensionality reduction before classification?

I'm trying to understand what LDA exactly does when used as a classifier. I've understood how the dimensionality reduction works and I've understood that the classification task is carried out with ...
francesco's user avatar
3 votes
1 answer
357 views

When using PCA for dimensionality reduction of the feature vectors to speed up learning, how do I know that I'm not letting the model overfit?

I'm following Andrew Ng's course for Machine Learning and I just don't quite understand the following. Using PCA to speed up learning Using PCA to reduce the number of features, thus lowering the ...
AfiJaabb's user avatar
  • 131
2 votes
0 answers
61 views

Estimating dimensions to reduce input image size to in CNNs

Considering input images to a CNN that have a large dimension (e.g. 256X256), what are some possible methods to estimate the exact dimensions (e.g. 16X16 or 32X32) to which it can be condensed in the ...
Prishita Ray's user avatar
2 votes
0 answers
32 views

Compressing Parameters of an Response System

I have an input-output system, which is fully determined by 256 parameters, of which I know a significant amount are of less importance to the input-output pattern. The data I have is some (64k in ...
t-smart's user avatar
  • 121
3 votes
0 answers
32 views

What is meant by subspace clustering in MFA?

The basic idea of MFA is to perform subspace clustering by assuming the covariance structure for each component of the form, $\Sigma_i = \Lambda_i \Lambda_i^T + \Psi_i$, where $\Lambda_i \in \mathbb{R}...
stoic-santiago's user avatar
3 votes
1 answer
222 views

How do AI researchers imagine higher dimensions?

We can visualize single, two, and three dimensions using websites or imagination. In the context of AI and, in particular, machine learning, AI researchers often have to deal with multi-dimensional ...
hanugm's user avatar
  • 3,792
1 vote
1 answer
48 views

How classification neural nets are different from simple dimension reduction + clustering?

I know the training of neural nets involves some sort of dimension manipulation to separate classes of different features. If there is no variation of features, no matter for neural nets or simple ...
Johnny Tam's user avatar
1 vote
0 answers
62 views

How to cluster data points such that the number of clusters is kept minimal and each cluster projects well onto a lower-dimensional subspace?

If I want to find a (linear) subspace onto which a data-set projects well, I can simply use PCA. However, often the data can project with much smaller error if I first separate it into a couple of ...
matthias_buehlmann's user avatar
0 votes
2 answers
103 views

Is it theoretically possible (or impossible) that principal component analysis worsens the performance of the model?

In case I had a prediction model and decided to add a PCA step prior to the model, is it theoretically possible/impossible that the number of output dimensions that is better for all tests may perform ...
Angelo's user avatar
  • 211
2 votes
2 answers
513 views

How can I use autoencoders to analyze patterns and classify them?

I generated a bunch of simulation data from a complex physical simulation that spits out patterns. I am trying to apply unsupervised learning to analyze the patterns and ideally classify them into ...
Pavan Inguva's user avatar
5 votes
2 answers
191 views

What are examples of approaches to dimensionality reduction of feature vectors?

Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements. I would like to apply some augmentation techniques to reduce the ...
doplano's user avatar
  • 299
2 votes
0 answers
27 views

Clustering of very high dimensional data and large number of examples without losing info in dimensions

I'm trying to get a grasp on scalability of clustering algorithms, and have a toy example in mind. Let's say I have around a million or so songs from $50$ genres. Each song has characteristics - some ...
Shirish Kulhari's user avatar
12 votes
4 answers
3k views

What are the purposes of autoencoders?

Autoencoders are neural networks that learn a compressed representation of the input in order to later reconstruct it, so they can be used for dimensionality reduction. They are composed of an encoder ...
nbro's user avatar
  • 40.2k