Skip to main content

How to perform PCA in the validation/test set?

I was using PCA in my whole dataset (and after split to training, validation and test), but after some researchs I found out that is wrong way to do. Then I have few questions:

-Are there some articles/references explain why is the wrong way?

-How can I transform the validation/test set?

Steps to do PCA (from https://www.sciencedirect.com/science/article/pii/S0022460X0093390X):

  1. zero mean

$$\mu = \frac{1}{M}\sum_{i=1}^{M} x_{i}$$

where x is my training set

  1. centering (variance)

$$S^{2} = \frac{1}{M}\sum_{i=1}^{M} (x_{i}-\mu)^{T}(x_{i}-\mu)$$

  1. use (1) and (2) to transform my original training dataset

$$x_{new} = \frac{1}{\sqrt{M}} \frac{(x_{i} - \mu)}{S}$$

  1. calculate covariance matrix (actually correlation matrix)

$$C= x_{new}^T x_{new}$$

  1. take the k-eigenvectors (/phi) from covariance matrix and defined the new space for my new dimension training set (where k are the principal components that I choose acording my variance)

$$ x_{new dim} = x_{new}\phi$$

Ok, then I have my new dimensional training dataset after PCA (till here its right according to other papers that I have read). The question is: What I have to do now for my validation/testing set?

just the equation below?

$$y_{new dim} = y\phi $$

where y is my (for exemple) validation original dataset?

Can someone explain the right thing to do?

Thanks!!! :)

LVoltz
  • 131
  • 1
  • 6