Communication-optimal parallel and sequential Cholesky decomposition

Grey Ballard*, James Demmel, Olga Holtz, Oded Schwartz

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

29 Scopus citations

Abstract

Numerical algorithms have two kinds of costs: arithmetic and communication, by which we mean either moving data between levels of a memory hierarchy (in the sequential case) or over a network connecting processors (in the parallel case). Communication costs often dominate arithmetic costs, so it is of interest to design algorithms minimizing communication. In this paper we first extend known lower bounds on the communication cost (both for bandwidth and for latency) of conventional (O(n3)) matrix multiplication to Cholesky factorization, which is used for solving dense symmetric positive definite linear systems. Second, we compare the costs of various Cholesky decomposition implementations to these lower bounds and identify the algorithms and data structures that attain them. In the sequential case, we consider both the two-level and hierarchical memory models. Combined with prior results in [J. Demmel et al., Communication-optimal Parallel and Sequential QR and LU Factorizations, Technical report EECS-2008-89, University of California, Berkeley, CA, 2008], [J. Demmel et al., Implementing Communication-optimal Parallel and Sequential QR and LU Factorizations, SIAM. J. Sci. Comp., submitted], and [J. Demmel, L. Grigori, and H. Xiang, Communication-avoiding Gaussian Elimination, Proceedings of the 2008 ACM/IEEE Conference on Supercomputing, 2008] this gives a set of communication-optimal algorithms for O(n3) implementations of the three basic factorizations of dense linear algebra: LU with pivoting, QR, and Cholesky. But it goes beyond this prior work on sequential LU by optimizing communication for any number of levels of memory hierarchy.

Original languageEnglish
Pages (from-to)3495-3523
Number of pages29
JournalSIAM Journal on Scientific Computing
Volume32
Issue number6
DOIs
StatePublished - 2010
Externally publishedYes

Keywords

  • Algorithm
  • Bandwidth
  • Cholesky decomposition
  • Communication avoiding
  • Latency
  • Lower bound

Fingerprint

Dive into the research topics of 'Communication-optimal parallel and sequential Cholesky decomposition'. Together they form a unique fingerprint.

Cite this