[Solved] MATH5473 Homework 2-Random Matrix Theory and PCA

1. Phase transition in PCA spike model: Consider a finite sample of n i.d vectors x₁,x₂,,x_n drawn from the p-dimensional Gaussian distribution N(0,²I_pp + ₀uu^T ), where ₀/² is the signal-to-noise ratio (SNR) and u R^p. In class we showed that the largest eigenvalue of the sample covariance matrix S_n

pops outside the support of the Marcenko-Pastur distribution if

or equivalently, if

SNR.

², that is, ₀ can be buried well inside the support Marcenko(Notice that < (1 + )

Pastur distribution and still the largest eigenvalue pops outside its support). All the following questions refer to the limit n and to almost surely values:

• Find given SNR > .
• Use your previous answer to explain how the SNR can be estimated from the eigenvaluesof the sample covariance matrix.
• Find the squared correlation between the eigenvector v of the sample covariance matrix (corresponding to the largest eigenvalue ) and the true signal component u, as a function of the SNR, p and n. That is, find |hu,vi|².
• Confirm your result using MATLAB, Python, or R simulations (e.g. set u = e; and choose = 1 and ₀ in different levels. Compute the largest eigenvalue and its associated eigenvector, with a comparison to the true ones.)

1

Homework 2. Random Matrix Theory and PCA 2

2. Exploring S&P500 Stock Prices: Take the Standard & Poors 500 data:

https://github.com/yao-lab/yao-lab.github.io/blob/master/data/snp452-data.mat which contains the data matrix X R^pn of n = 1258 consecutive observation days and p = 452 daily closing stock prices, and the cell variable stock collects the names, codes, and the affiliated industrial sectors of the 452 stocks. Use Matlab, Python, or R for the following exploration.

• Take the logarithmic prices Y = logX;
• For each observation time t {1,,1257}, calculate logarithmic price jumps

Y_i,t = Y_i,t Y_i,t1, i {1,,452};

• Construct the realized covariance matrix R⁴⁵²⁴⁵² by,

1257

;

=1

• Compute the eigenvalues (and eigenvectors) of and store them in a descending order by {_k,k = 1,,p}.
• Horns Parallel Analysis: the following procedure describes a so-called Parallel Analysis of PCA using random permutations on data. Given the matrix [Y_i,t], apply random permutations _i : {1,,t} {1,,t} on each of its rows: Y_i,i(_j) such that

Define as the null covariance matrix. Repeat this for R times and compute the eigenvalues of _r for each 1 r R. Evaluate the p-value for each estimated eigenvalue _k by (N_k+1)/(R+1) where N_k is the counts that _k is less than the k-th largest eigenvalue of _r over 1 r R. Eigenvalues with small p-values indicate that they are less likely arising from the spectrum of a randomly permuted matrix and thus considered to be signal. Draw your own conclusion with your observations and analysis on this data. A reference is: Buja and Eyuboglu, Remarks on Parallel Analysis, Multivariate Behavioral Research, 27(4): 509-540, 1992.

3. *Finite rank perturbations of random symmetric matrices: Wigners semi-circle law (proved by Eugene Wigner in 1951) concerns the limiting distribution of the eigenvalues of random symmetric matrices. It states, for example, that the limiting eigenvalue distribution of n n symmetric matrices whose entries w_ij on and above the diagonal (i j) are i.i.d Gaussians

) (and the entries below the diagonal are determined by symmetrization, i.e., w_ji = w_ij) is the semi-circle:

,

where the distribution is supported in the interval [1,1].

Homework 2. Random Matrix Theory and PCA 3

• Confirm Wigners semi-circle law using MATLAB, Python, or R simulations (take, e.g.,n = 400).
• Find the largest eigenvalue of a rank-1 perturbation of a Wigner matrix. That is, findthe largest eigenvalue of the matrix

W + ₀uu^T ,

where W is an n n random symmetric matrix as above, and u is some deterministic unit-norm vector. Determine the value of ₀ for which a phase transition occurs. What is the correlation between the top eigenvector of W + ₀uu^T and the vector u as a function of ₀? Use techniques similar to the ones we used in class for analyzing finite rank perturbations of sample covariance matrices.

[Some Hints about homework] For Wigner Matrix), the answer is

eigenvalue is

eigenvector satisfies (