TY - JOUR
T1 - The phase transition of matrix recovery from Gaussian measurements matches the minimax MSE of matrix denoising
AU - Donoho, David L.
AU - Gavish, Matan
AU - Montanari, Andrea
PY - 2013/5/21
Y1 - 2013/5/21
N2 - Let X0 be an unknown M by N matrix. In matrix recovery, one takes n1,...,yn of X 0, where yi = Tr(ATi X0) and each Ai is an M by N matrix. A popular approach for matrix recovery is nuclear norm minimization (NNM): solving the convex optimization problem min ||X||* subject to yi =Tr(ATi X) for all 1≤i ≤n, where ||·||* denotes the nuclear norm, namely, the sum of singular values. Empirical work reveals a phase transition curve, stated in terms of the undersampling fraction δ(n,M,N)= n/(MN), rank fraction ρ=rank(X0)/min{M,N}, and aspect ratio β=M/N. Specifically when the measurement matrices Ai have independent standard Gaussian random entries, a curve δ*(ρ)= δ*(ρ;β) exists such that, if δ>δ* (ρ), NNM typically succeeds for large M,N, whereas if δ<δ *(ρ), it typically fails. An apparently quite different problem is matrix denoising in Gaussian noise, in which an unknown M by N matrix X 0 is to be estimated based on direct noisy measurements Y =X 0 +Z, where the matrix Z has independent and identically distributed Gaussian entries. A popular matrix denoising scheme solves the unconstrained optimization problem min||Y-X||2F /2+λ||X|| * . When optimally tuned, this scheme achieves the asymptotic minimax mean-squared error M(ρ;β)= lim M,N→∞infλsup rank(X)≤ρ·MMSE(X,X̂λ), where M/N→β. We report extensive experiments showing that the phase transition δ*(ρ) in the first problem, matrix recovery from Gaussian measurements, coincides with the minimax risk curveM(ρ)= M(ρ;β) in the second problem, matrix denoising in Gaussian noise: δ*(ρ)=M(ρ), for any rank fraction 0<ρ<1 (at each common aspect ratio β). Our experiments considered matrices belonging to two constraint classes: real M by N matrices, of various ranks and aspect ratios, and real symmetric positive-semidefinite N by N matrices, of various ranks.
AB - Let X0 be an unknown M by N matrix. In matrix recovery, one takes n1,...,yn of X 0, where yi = Tr(ATi X0) and each Ai is an M by N matrix. A popular approach for matrix recovery is nuclear norm minimization (NNM): solving the convex optimization problem min ||X||* subject to yi =Tr(ATi X) for all 1≤i ≤n, where ||·||* denotes the nuclear norm, namely, the sum of singular values. Empirical work reveals a phase transition curve, stated in terms of the undersampling fraction δ(n,M,N)= n/(MN), rank fraction ρ=rank(X0)/min{M,N}, and aspect ratio β=M/N. Specifically when the measurement matrices Ai have independent standard Gaussian random entries, a curve δ*(ρ)= δ*(ρ;β) exists such that, if δ>δ* (ρ), NNM typically succeeds for large M,N, whereas if δ<δ *(ρ), it typically fails. An apparently quite different problem is matrix denoising in Gaussian noise, in which an unknown M by N matrix X 0 is to be estimated based on direct noisy measurements Y =X 0 +Z, where the matrix Z has independent and identically distributed Gaussian entries. A popular matrix denoising scheme solves the unconstrained optimization problem min||Y-X||2F /2+λ||X|| * . When optimally tuned, this scheme achieves the asymptotic minimax mean-squared error M(ρ;β)= lim M,N→∞infλsup rank(X)≤ρ·MMSE(X,X̂λ), where M/N→β. We report extensive experiments showing that the phase transition δ*(ρ) in the first problem, matrix recovery from Gaussian measurements, coincides with the minimax risk curveM(ρ)= M(ρ;β) in the second problem, matrix denoising in Gaussian noise: δ*(ρ)=M(ρ), for any rank fraction 0<ρ<1 (at each common aspect ratio β). Our experiments considered matrices belonging to two constraint classes: real M by N matrices, of various ranks and aspect ratios, and real symmetric positive-semidefinite N by N matrices, of various ranks.
KW - Compressed sensing
KW - Matrix completion
UR - http://www.scopus.com/inward/record.url?scp=84878157685&partnerID=8YFLogxK
U2 - 10.1073/pnas.1306110110
DO - 10.1073/pnas.1306110110
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???
AN - SCOPUS:84878157685
SN - 0027-8424
VL - 110
SP - 8405
EP - 8410
JO - Proceedings of the National Academy of Sciences of the United States of America
JF - Proceedings of the National Academy of Sciences of the United States of America
IS - 21
ER -