Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [principal-component-analysis]

For questions related to principal component analysis (PCA), which is commonly used in machine learning for dimensionality reduction.

Filter by
Sorted by
Tagged with
0 votes
1 answer
185 views

What are alternatives to PCA for time series data?

I have some data (20 stock price time series) and want to compare different approaches for dimensionality reduction other than PCA (I want to fit only 2 variables in my AR model). I've tried ...
J_Bake's user avatar
  • 11
0 votes
1 answer
76 views

What is the meaning of "energy loss" in PCA?

Recently in a slide in about PCA (Principal Component Analysis) I saw a question: "How much is the data energy loss in PCA?&...
hasanghaforian's user avatar
0 votes
0 answers
23 views

Could converting features into PCs, where #PCs = #features, improve model performance?

Let's say I have a dataset with some number of features (e.g., 10) and a target variable. I create 10 PCs from the dataset excluding the target variable. Then, I run a few classification algorithms (...
BigMistake's user avatar
0 votes
0 answers
6 views

Dimensionality Reduction of a matrix preserving number of rows

Let A ($d \times k $) be a matrix such that k < d. How to reduce the dimension of the matrix A to another lower dimension matrix B ($d \times l$ ) such that $l < k$. Note that, while reducing ...
Rituraj Singh's user avatar
0 votes
0 answers
8 views

What is $D_i$ in "Common Principal Components Analysis"?

The following is from Ethem Alpaydin, "Introduction to Machine Learning", Fourth Edition, MIT Press, 2020, page 129. Here, $C$ is the orthogonal matrix found from the sample covariance of ...
DSPinfinity's user avatar
0 votes
1 answer
27 views

What is $D_i$ in "Common Principal Components Analysis"?

The following is from Ethem Alpaydin, "Introduction to Machine Learning", Fourth Edition, MIT Press, 2020, page 129. Here, $C$ is the orthogonal matrix found from the sample covariance of ...
DSPinfinity's user avatar
0 votes
0 answers
31 views

Are the dimensions in embedding vectors ordered (similar to PCA)?

I am getting started with the vector embeddings. I have a general question about the embedding vectors generated by popular algorithms. In PCA, usually, there is an implicit order of importance in the ...
JackDaniels's user avatar
0 votes
1 answer
231 views

Is there a way to select the subset of most important features using PCA?

Is there a way to select the most important features using PCA? I am not looking for the principal components with the highest scores but a subset of the original features.
Mika's user avatar
  • 341
1 vote
0 answers
43 views

How do I select the number of neurons for each layer in an auto-encoder for dimensionality reduction?

I am trying to apply an auto-encoder for dimensionality reduction. I wonder how it will be applied on a large dataset. I have tried this code below. I have total of 8 features in my data and I want to ...
user12's user avatar
  • 111
1 vote
0 answers
43 views

Under what circumstances is a fully connected layer similar to PCA?

I am reading this paper on image retrieval where the goal is to train a network that produces highly discriminative descriptors (aka embeddings) for input images. If you are familiar with facial ...
Alexander Soare's user avatar
2 votes
1 answer
243 views

Why does PCA of the vertices of a hexagon result in principal components of equal length?

I do PCA on the data points placed in the corners of a hexagon, and get the following principal components: The PCA variance is $0.6$ and is the same for each component. Why is that? Shouldn't it be ...
Vladislav Gladkikh's user avatar
0 votes
1 answer
164 views

Why Autoencoder Weights Are Not Always Tied

To me, tying weights in an autoencoder makes sense if we think of the auto encoder as doing PCA. Why in any situation would it make sense to not tie the weights? If we don't tie the weights, would it ...
tushar's user avatar
  • 116
3 votes
1 answer
382 views

When using PCA for dimensionality reduction of the feature vectors to speed up learning, how do I know that I'm not letting the model overfit?

I'm following Andrew Ng's course for Machine Learning and I just don't quite understand the following. Using PCA to speed up learning Using PCA to reduce the number of features, thus lowering the ...
AfiJaabb's user avatar
  • 131
2 votes
0 answers
61 views

Estimating dimensions to reduce input image size to in CNNs

Considering input images to a CNN that have a large dimension (e.g. 256X256), what are some possible methods to estimate the exact dimensions (e.g. 16X16 or 32X32) to which it can be condensed in the ...
Prishita Ray's user avatar
1 vote
0 answers
177 views

Is the 3d convolution associative given that it can be represented as matrix multiplication?

I'm trying to understand if a 3D convolution of the sort performed in a convolutional layer of a CNN is associative. Specifically, is the following true: $$ X \otimes(W \cdot Q)=(X \otimes W) \cdot Q, ...
HereItIs's user avatar
0 votes
0 answers
48 views

Unable to meet desired mean squared error

I wish to get MSE < 0.5 on test data (https://easyupload.io/zr7xf3) which is 20% of given data chosen randomly. But I am reaching 0.73 using both plain Ridge Regression as well as a neural network ...
Mrinmay's user avatar
2 votes
2 answers
140 views

How does PCA work when we reduce the original space to 2 or higher-dimensional space?

How does PCA work when we reduce the original space to a 2 or higher-dimensional space? I understand the case when we reduce the dimensionality to $1$, but not this case. $$\begin{array}{ll} \text{...
VN Pikachu's user avatar
3 votes
0 answers
57 views

Looking for the proper algorithm to compress many lowres images of nearby locations

I have an optimization problem that I'm looking for the right algorithm to solve. What I have: A large set of low-res 360 images that were taken on a regular grid within a certain area. each of these ...
matthias_buehlmann's user avatar
0 votes
2 answers
105 views

Is it theoretically possible (or impossible) that principal component analysis worsens the performance of the model?

In case I had a prediction model and decided to add a PCA step prior to the model, is it theoretically possible/impossible that the number of output dimensions that is better for all tests may perform ...
Angelo's user avatar
  • 211
1 vote
0 answers
45 views

Multiple-dimension scaling (MDS) objective for MDS and PCA

The following is the MDS Objective. Let's think of a senario where I apply MDS with/from the solution I obtained from PCA. Then I calculate the objective function on the initial PCA solution and MDS ...
underdisplayname's user avatar
5 votes
2 answers
193 views

What are examples of approaches to dimensionality reduction of feature vectors?

Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements. I would like to apply some augmentation techniques to reduce the ...
doplano's user avatar
  • 299
1 vote
0 answers
37 views

What is the difference between principal component analysis and singular value decomposition in image processing?

What is the difference between principal component analysis and singular value decomposition in image processing? Which one performs better, and why?
DRV's user avatar
  • 1,693
3 votes
1 answer
81 views

Do the eigenvectors represent the original features?

I've got a test dataset with 4 features and the PCA produces a set of 4 eigenvectors, e.g., ...
Crizly's user avatar
  • 131
1 vote
1 answer
154 views

Why does PCA work well while the total variance retained is small?

I'm learning machine learning by looking through other people's kernel on Kaggle, specifically this Mushroom Classification kernel. The author first applied PCA to the transformed indicator matrix. He ...
Bicheng's user avatar
  • 111
1 vote
1 answer
1k views

How to perform PCA in the validation/test set?

I was using PCA on my whole dataset (and, after that, I would split it into training, validation, and test datasets). However, after a little bit of research, I found out that this is the wrong way to ...
LVoltz's user avatar
  • 131