i have 1200 features highly correlated , and i want to reduce those number of features so the best choice is use feature selection or dimensionality reduction? and which method is the best in this case. feature selection can work with features highly correlated ??

  • $\begingroup$ Isn't dimensionality reduction a form of feature selection? What do you mean by "feature selection" more specifically? Please, edit your post to provide more details. $\endgroup$
    – nbro
    Commented Nov 17, 2023 at 0:41

1 Answer 1

  • Feature selection -- the case in which the features are highly correlated is the prototypical case in which you want to select a subset of independent features that allows for an equal performance. However keep in mind that, exactly because they are correlated, there are in general multiple subsets of features that can achieve the same goal (i.e., if x_1 is highly correlated with x_2, you can drop either of them).

  • Dimensionality reduction -- if your features are only linearly correlated, then a linear dimensionality reduction technique (e.g., PCA) will tackle your problem. If however the dependence is more than linear (and unknown), dimensionality reduction will construct new features that are a complex combination of the original ones, hence you will get to the same problem, just in a reduced-dimensional space.

TLDR; As usual in ML, it depends on what you need to do. If it is merely a matter of reducing dimensions in order to speed up calculations, either method is valid. If you need instead to select features for further [causal?] analysis, you need to lean for the first approach. But, again, be aware of multiple equivalent sets (e.g., see this paper: https://arxiv.org/abs/1611.03227)

  • $\begingroup$ thank you for your answer , my goal is at the end i will choose a classifier and do the binary classification . this features will be the input of this classifier $\endgroup$
    – myriamkach
    Commented Nov 15, 2023 at 0:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .