Given data = { 2, 3, 4, 5, 6, 7 ; 1, 5, 3, 6, 7, 8 }.Compute the principal component

Solution :

data = { 2, 3, 4, 5, 6, 7 ; 1, 5, 3, 6, 7, 8 }.Compute the principal component using PCA Algorithm.

First principal component: \(\text{PC}_1 = 0.570\,(x-4.5) + 0.822\,(y-5)\), with eigenvalue 9.85, explaining about 95.6% of the variance.

Data: \(x = 2,3,4,5,6,7\) and \(y = 1,5,3,6,7,8\), \(n = 6\).

Step 1: Means. \(\bar{x} = 27/6 = 4.5\), \(\bar{y} = 30/6 = 5\).

Step 2: Deviations. \(x-\bar{x}\): −2.5, −1.5, −0.5, 0.5, 1.5, 2.5; \(y-\bar{y}\): −4, 0, −2, 1, 2, 3.

Step 3: Covariance matrix (dividing by \(n-1 = 5\)):

\[\text{var}(x) = \frac{17.5}{5} = 3.5,\quad \text{var}(y) = \frac{34}{5} = 6.8\]

\[\text{cov}(x,y) = \frac{22}{5} = 4.4\]

\[S = \begin{pmatrix} 3.5 & 4.4 \\ 4.4 & 6.8 \end{pmatrix}\]

Step 4: Eigenvalues. \(|S - \lambda I| = 0\):

\[\lambda^2 - 10.3\lambda + 4.44 = 0\]

\[\lambda_1 = 9.849,\quad \lambda_2 = 0.451\]

Step 5: Eigenvector for \(\lambda_1\). \((3.5 - 9.849)v_1 + 4.4v_2 = 0\) gives \(v_2 = 1.443\,v_1\). Normalising:

\[e_1 = (0.570,\ 0.822)\]

Step 6: Principal component scores \(= 0.570(x-4.5) + 0.822(y-5)\):

−4.71, −0.85, −1.93, 1.11, 2.50, 3.89

Variance explained: \(9.849 / (9.849 + 0.451) = 95.6\%\), so the 2-D data can be reduced to this one component with little loss. (The second component is \(e_2 = (0.822,\ -0.570)\) with eigenvalue 0.451.)

If the covariance is computed with \(n = 6\) instead of \(n-1\), all eigenvalues are scaled by 5/6 but the eigenvectors and the 95.6% stay the same.