Use X’X as computed in Exercise 2.14.
(a) Compute the eigen analysis of X’X. What is the relationship between the singular values of X obtained in Exercise 2.15 and the eigenvalues obtained for X X?
(b) Use the results of the eigen analysis to compute the rank-1 approximation of X’X. Compare this result to the approximation of X’X obtained in Exercise 2.15.
(c) Show algebraically that they should be identical.
Exercise 2.15
Use X as defined in Exercise 2.14.
(a) Find the singular value decomposition of X. Explain what the singular values tell you about the rank of X.
(b) Compute the rank-1 approximation of X; call it A1. Use the singular values to state the “goodness of fit” of this rank-1 approximation.
(c) Use A1
to compute a rank-1 approximation of X’X; that is, compute A1A1. Compare tr(A1A1) with λ1
and tr(X X).
Exercise 2.14
Find the inverse of the following matrix,
(a) Compute X’X and X’Y. Verify by separate calculations that the (i, j) = (2, 2) element in X’X is the sum of squares of column 2 in X. Verify that the (2, 3) element is the sum of products between columns 2 and 3 of X. Identify the elements in X’Y in terms of sums of squares or products of the columns of X and Y.
(b) Is X of full column rank? What is the rank of X’X?
(c) Obtain (X’X)−1. What is the rank of (X’X)−1? Verify by matrix multiplication that (X’X)−1X’X = I.
(d) Compute P = X(X’X)−1X and verify by matrix multiplication that P is idempotent. Compute the trace tr(P). What is r(P)?