I was using PCA in my whole dataset (and after split to training, validation and test), but after some researchs I found out that is wrong way to do. Then I have few questions:
-Are there some articles/references explain why is the wrong way?
-How can I transform the validation/test set?
Steps to do PCA (from https://www.sciencedirect.com/science/article/pii/S0022460X0093390X):
- zero mean
$$\mu = \frac{1}{M}\sum_{i=1}^{M} x_{i}$$
where x is my training set
- centering (variance)
$$S^{2} = \frac{1}{M}\sum_{i=1}^{M} (x_{i}-\mu)^{T}(x_{i}-\mu)$$
- use (1) and (2) to transform my original training dataset
$$x_{new} = \frac{1}{\sqrt{M}} \frac{(x_{i} - \mu)}{S}$$
- calculate covariance matrix (actually correlation matrix)
$$C= x_{new}^T x_{new}$$
- take the k-eigenvectors (/phi) from covariance matrix and defined the new space for my new dimension training set (where k are the principal components that I choose acording my variance)
$$ x_{new dim} = x_{new}\phi$$
Ok, then I have my new dimensional training dataset after PCA (till here its right according to other papers that I have read). The question is: What I have to do now for my validation/testing set?
just the equation below?
$$y_{new dim} = y\phi $$
where y is my (for exemple) validation original dataset?
Can someone explain the right thing to do?
Thanks!!! :)