Optimal shrinkage of eigenvalues in the spiked covariance model1

David Donoho*, Matan Gavish, Iain Johnstone

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

76 Scopus citations

Abstract

We show that in a common high-dimensional covariance model, the choice of loss function has a profound effect on optimal estimation. In an asymptotic framework based on the spiked covariance model and use of orthogonally invariant estimators, we show that optimal estimation of the population covariance matrix boils down to design of an optimal shrinker η that acts elementwise on the sample eigenvalues. Indeed, to each loss function there corresponds a unique admissible eigenvalue shrinker η dominating all other shrinkers. The shape of the optimal shrinker is determined by the choice of loss function and, crucially, by inconsistency of both eigenvalues and eigenvectors of the sample covariance matrix. Details of these phenomena and closed form formulas for the optimal eigenvalue shrinkers are worked out for a menagerie of 26 loss functions for covariance estimation found in the literature, including the Stein, Entropy, Divergence, Fréchet, Bhattacharya/Matusita, Frobenius Norm, Operator Norm, Nuclear Norm and Condition Number losses.

Original languageAmerican English
Pages (from-to)1742-1778
Number of pages37
JournalAnnals of Statistics
Volume46
Issue number4
DOIs
StatePublished - Aug 2018

Bibliographical note

Funding Information:
1Supported in part by NSF Grant DMS 0906812 (ARRA) and 1407813 and NIH RO1 EB001988. 2Supported in part by a William R. and Sara Hart Kimball Stanford Graduate Fellowship. MSC2010 subject classifications. Primary 62C20, 62H25; secondary 90C25, 90C22. We thank Amit Singer, Andrea Montanari, Sourav Chat-terjee and Boaz Nadler for helpful discussions. We also thank the anonymous referees for significantly improving the manuscript through their helpful comments.

Funding Information:
Received March 2014; revised May 2017. 1Supported in part by NSF Grant DMS 0906812 (ARRA) and 1407813 and NIH RO1 EB001988. 2Supported in part by a William R. and Sara Hart Kimball Stanford Graduate Fellowship. MSC2010 subject classifications. Primary 62C20, 62H25; secondary 90C25, 90C22. Key words and phrases. Covariance estimation, optimal shrinkage, Stein loss, entropy loss, divergence loss, Fréchet distance, Bhattacharya/Matusita affinity, condition number loss, high-dimensional ssymptotics, spiked covariance.

Publisher Copyright:
© Institute of Mathematical Statistics, 2018

Keywords

  • Bhattacharya/Matusita affinity
  • Condition number loss
  • Covariance estimation
  • Divergence loss
  • Entropy loss
  • Fréchet distance
  • High-dimensional ssymptotics
  • Optimal shrinkage
  • Spiked covariance
  • Stein loss

Fingerprint

Dive into the research topics of 'Optimal shrinkage of eigenvalues in the spiked covariance model1'. Together they form a unique fingerprint.

Cite this