4 votes
Accepted

What are examples of approaches to dimensionality reduction of feature vectors?

Dimensionality reduction could be achieved by using an Autoencoder Network, which learns a representation (or Encoding) for the input data. While training, the reduction side (Encoder) reduces the ...
  • 360
3 votes
Accepted

Why does PCA of the vertices of a hexagon result in principal components of equal length?

Assuming that the $6$ vertices of the hexagon are on the unit circle, ...
2 votes

What are examples of approaches to dimensionality reduction of feature vectors?

Some examples of dimensionality reduction techniques: Linear methods Non-linear methods Graph-based methods("Network embedding") PCA CCA ICA SVD LDA NMF Kernel PCA GDA Autoencoders t-SNE ...
2 votes

When using PCA for dimensionality reduction of the feature vectors to speed up learning, how do I know that I'm not letting the model overfit?

I'm not sure if I understood your question correctly, but here's my take anyway. So, PCA is a technique that you can apply to data to reduce the number of features. In return, (i) this can speed-up ...
2 votes

How does PCA work when we reduce the original space to 2 or higher-dimensional space?

You might want to have a look at the wikipedia article of PCA, where it says: "The $k$th component can be found by subtracting the first $k − 1$ principal components from $\mathbf{X}$:" $$\...
  • 600
2 votes

Do the eigenvectors represent the original features?

The principal components (eigenvectors) correspond to the direction (in the original n-dimensional space) with the greatest variance in the data. The corresponding eigenvalue is a number that ...
  • 121
2 votes
Accepted

How to perform PCA in the validation/test set?

for the first point I'm very sorry that I cannot give you any literature on this, but I might be able to explain you, why you don't take PCA on both datasets independently. Principal components ...
1 vote

Why Autoencoder Weights Are Not Always Tied

We expect the decoder to learn anything meaningful without tying the weights because the loss function is calculated between the input and reconstructed output and training will minimize that loss. ...
1 vote
Accepted

Is it theoretically possible (or impossible) that principal component analysis worsens the performance of the model?

PCA works well where data sample space is linear. If data sample space is not linear or it is manifold data then model without PCA may perform better than model using PCA. In the given image you can ...
  • 26
1 vote

Is it theoretically possible (or impossible) that principal component analysis worsens the performance of the model?

PCA can make models worse, imagine data points scattered along two elongated parallel rectangles. The axis with the greatest variation will be parallel to the rectangles but doesn't provide any ...

Only top scored, non community-wiki answers of a minimum length are eligible