Is there a way to select the most important features using PCA? I am not looking for the principal components with the highest scores but a subset of the original features.
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
There are better methods for selecting most important features in supervised setting. Assuming they are not an option, or you're simply interested in PCA:
Say you originally had 100 features and you applied PCA and first 10 PCs explains the 95 % of ratio.
After applying PCA, you can calculate linear correlations between top 10 PCs and original features. I assume some of your features will be highly correlated with some subset of top 10 PCs. You can draw an abstract line and choose subset of original features that are at least 0.80 linearly correlated with at least one of top 10 PCs.
-
$\begingroup$ You can simplify this by looking at the resulting eigenvectors and evaluating the magnitude of each feature as a contribution to the final translation. $\endgroup$ Commented Aug 16, 2023 at 21:33
$\begingroup$
$\endgroup$
Yes, there is a way to select the most important original features using PCA, but it involves a technique called "PCA-based feature selection" or "Backward mapping" from the principal components to the original features. Here's how you can do it:
You can do it by just following these steps
- Apply PCA to Your Dataset:
=> First, you apply PCA to transform the data into its principal components. This gives you the new feature space where the data is represented in terms of principal components.
- Look at the Loadings (Eigenvectors):
=> The loadings (or eigenvectors) represent the contribution of each original feature to the principal components. These loadings can be interpreted as the importance of each original feature in forming the principal components.
=> The more a feature contributes to a principal component (i.e., the larger its coefficient in the eigenvector), the more important it is.
3. Rank Features by Contribution:
=> For each principal component, check the magnitude of the loadings (eigenvector coefficients). Features with larger loadings are considered to have a greater impact on that principal component.
=> Sum up or average the absolute values of loadings across all the principal components to rank the features by their overall contribution to explaining the variance in the data.
4. Select a Subset of Features:
Based on the ranking, you can select a subset of the original features that have the highest contributions. These are the features most responsible for explaining the variance in the data as represented by the principal components.
you can use method like recursive feature elimination also for your task
References
