# What is the meaning of "energy loss" in PCA?

Recently in a slide in about PCA (Principal Component Analysis) I saw a question: "How much is the data energy loss in PCA?" Then in the next slide I saw somethings that do not understand relation of last steps to the PCA and energy. Although I know that PCA tries to find principle components which have variances in decreasing order and also guess the energy means the total variance after keeping some first principle components and applying it on normalized data; but in this way I think what is described in Wikipedia as cumulative energy content for each eigenvector and what is described here, are more rational and understandable.

Here you can see part of slide that tries to show an example:

Steps are:

1. X as input
2. normalize X (as Z)
3. compute covariance matrix of Z as R
4. eigenvalues of R in descending order as $$\lambda$$ and their corresponding vectors as V.
5. $$S = V\lambda^{\frac{1}{2}}$$ where $$\lambda^{\frac{1}{2}}$$ is element-wise operation.
6. discard third column of S and sore it as S
7. calculate diagonal elements of $$S S^{T}$$ as var columnar matrix
8. 1 - var(2,1) is the amount of energy loss of PCA. Here it is 1 - 0.8420.

This is an Octave code that you can follow to get the same results:

X=[
7 4 3;4 1 8;6 3 5;8 6 1;8 5 7;
7 2 9;5 3 3;9 5 8;7 4 5;8 2 2];

rows=size(X)(1);
XMean = repmat(mean(X),rows,1);
XStd = repmat(std(X),rows,1);
X_normalized = (X - XMean)./(XStd);

Cov_mat=cov(X_normalized);

[V,lambda] = eigs(Cov_mat) %descending order
S = V * lambda^(0.5);
S1 = S(:,1:2);
var = diag(S1*S1')


Here are what I do not understand:

• What is the definition of energy loss of PCA? If it is relative to what is described in Wikipedia, then how is it relative to the steps that here are described in above slide?
• Where can I found a reference or some examples about these steps?
• Why did we keep only first two columns of S?
• Why 1 - var(2,1) is the amount of energy loss of PCA? Does 2 in 1 - var(2,1) is relative to the first two columns?
• Is 1 - var(2,1) is the amount of energy loss of PCA by using two eigenvectors relative to the two eigenvectors with largest magnitude?
• If we want to calculate energy loss for only using eigenvector that is relative to the largest eigenvalue, do we have to calculate $$[S(1,1) S(1,2) S(1,3)]^T [S(1,1) S(1,2) S(1,3)]$$ as var and 1 - var(1,1) is the amount of energy loss?
• Please, split this post into multiple ones. Here you're asking multiple questions in the same post, which is discouraged because people are discouraged to address your post as it can become too complicated. So, ideally, you should ask one question per post.
– nbro
Commented Jan 17 at 1:03