Optimal Shrinkage of Singular Values

Matan Gavish, David L. Donoho

Research output: Contribution to journalArticlepeer-review

101 Scopus citations

Abstract

We consider the recovery of low-rank matrices from noisy data by shrinkage of singular values, in which a single, univariate nonlinearity is applied to each of the empirical singular values. We adopt an asymptotic framework, in which the matrix size is much larger than the rank of the signal matrix to be recovered, and the signal-To-noise ratio of the low-rank piece stays constant. For a variety of loss functions, including Mean Square Error (MSE)-(square Frobenius norm), the nuclear norm loss and the operator norm loss, we show that in this framework there is a well-defined asymptotic loss that we evaluate precisely in each case. In fact, each of the loss functions we study admits a unique admissible shrinkage nonlinearity dominating all other nonlinearities. We provide a general method for evaluating these optimal nonlinearities, and demonstrate our framework by working out simple, explicit formulas for the optimal nonlinearities in the Frobenius, nuclear and operator norm cases. For example, for a square low-rank n-by-n matrix observed in white noise with level σ, the optimal nonlinearity for MSE loss simply shrinks each data singular value y to-y2 ? 4nσ2 (or to 0 if y < 2 √ nσ ). This optimal nonlinearity guarantees an asymptotic MSE of 2nrσ2, which compares favorably with optimally tuned hard thresholding and optimally tuned soft thresholding, providing guarantees of 3nrσ2 and 6nrσ 2, respectively. Our general method also allows one to evaluate optimal shrinkers numerically to arbitrary precision. As an example, we compute optimal shrinkers for the Schatten-p norm loss, for any p > 0.

Original languageAmerican English
Article number7820186
Pages (from-to)2137-2152
Number of pages16
JournalIEEE Transactions on Information Theory
Volume63
Issue number4
DOIs
StatePublished - Apr 2017

Bibliographical note

Publisher Copyright:
© 1963-2012 IEEE.

Keywords

  • Matrix denoising
  • Schatten norm loss
  • low-rank matrix estimation
  • nuclear norm loss
  • optimal shrinkage
  • singular value shrinkage
  • spiked model
  • unique admissible

Fingerprint

Dive into the research topics of 'Optimal Shrinkage of Singular Values'. Together they form a unique fingerprint.

Cite this