Communication optimal parallel multiplication of sparse random matrices

Grey Ballard; Aydin Buluç; James Demmel; Laura Grigori; Benjamin Lipshitz; Oded Schwartz; Sivan Toledo

doi:10.1145/2486159.2486196

Communication optimal parallel multiplication of sparse random matrices

Grey Ballard, Aydin Buluç, James Demmel, Laura Grigori, Benjamin Lipshitz, Oded Schwartz, Sivan Toledo

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

73 Scopus citations

Abstract

Parallel algorithms for sparse matrix-matrix multiplication typically spend most of their time on inter-processor communication rather than on computation, and hardware trends predict the relative cost of communication will only increase. Thus, sparse matrix multiplication algorithms must minimize communication costs in order to scale to large processor counts. In this paper, we consider multiplying sparse matrices corresponding to Erdo″s- Rényi random graphs on distributedmemory parallel machines. We prove a new lower bound on the expected communication cost for a wide class of algorithms. Our analysis of existing algorithms shows that, while some are optimal for a limited range of matrix density and number of processors, none is optimal in general. We obtain two new parallel algorithms and prove that they match the expected communication cost lower bound, and hence they are optimal.

Original language	English
Title of host publication	SPAA 2013 - Proceedings of the 25th ACM Symposium on Parallelism in Algorithms and Architectures
Publisher	Association for Computing Machinery
Pages	222-231
Number of pages	10
ISBN (Print)	9781450315722
DOIs	https://doi.org/10.1145/2486159.2486196
State	Published - 2013
Externally published	Yes
Event	25th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2013 - Montreal, QC, Canada Duration: 23 Jul 2013 → 25 Jul 2013

Publication series

Name	Annual ACM Symposium on Parallelism in Algorithms and Architectures

Conference

Conference	25th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2013
Country/Territory	Canada
City	Montreal, QC
Period	23/07/13 → 25/07/13

Keywords

Communication-avoiding algorithms
Communication-cost lower bounds
Random graphs
Sparse matrix multiplication

Access to Document

10.1145/2486159.2486196

Cite this

Ballard, G., Buluç, A., Demmel, J., Grigori, L., Lipshitz, B., Schwartz, O., & Toledo, S. (2013). Communication optimal parallel multiplication of sparse random matrices. In SPAA 2013 - Proceedings of the 25th ACM Symposium on Parallelism in Algorithms and Architectures (pp. 222-231). (Annual ACM Symposium on Parallelism in Algorithms and Architectures). Association for Computing Machinery. https://doi.org/10.1145/2486159.2486196

@inproceedings{04e5ea6657e549b08cd5ca44c62e9e9f,

title = "Communication optimal parallel multiplication of sparse random matrices",

abstract = "Parallel algorithms for sparse matrix-matrix multiplication typically spend most of their time on inter-processor communication rather than on computation, and hardware trends predict the relative cost of communication will only increase. Thus, sparse matrix multiplication algorithms must minimize communication costs in order to scale to large processor counts. In this paper, we consider multiplying sparse matrices corresponding to Erdo″s- R{\'e}nyi random graphs on distributedmemory parallel machines. We prove a new lower bound on the expected communication cost for a wide class of algorithms. Our analysis of existing algorithms shows that, while some are optimal for a limited range of matrix density and number of processors, none is optimal in general. We obtain two new parallel algorithms and prove that they match the expected communication cost lower bound, and hence they are optimal.",

keywords = "Communication-avoiding algorithms, Communication-cost lower bounds, Random graphs, Sparse matrix multiplication",

author = "Grey Ballard and Aydin Bulu{\c c} and James Demmel and Laura Grigori and Benjamin Lipshitz and Oded Schwartz and Sivan Toledo",

year = "2013",

doi = "10.1145/2486159.2486196",

language = "אנגלית",

isbn = "9781450315722",

series = "Annual ACM Symposium on Parallelism in Algorithms and Architectures",

publisher = "Association for Computing Machinery",

pages = "222--231",

booktitle = "SPAA 2013 - Proceedings of the 25th ACM Symposium on Parallelism in Algorithms and Architectures",

note = "25th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2013 ; Conference date: 23-07-2013 Through 25-07-2013",

}

Ballard, G, Buluç, A, Demmel, J, Grigori, L, Lipshitz, B, Schwartz, O & Toledo, S 2013, Communication optimal parallel multiplication of sparse random matrices. in SPAA 2013 - Proceedings of the 25th ACM Symposium on Parallelism in Algorithms and Architectures. Annual ACM Symposium on Parallelism in Algorithms and Architectures, Association for Computing Machinery, pp. 222-231, 25th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2013, Montreal, QC, Canada, 23/07/13. https://doi.org/10.1145/2486159.2486196

Communication optimal parallel multiplication of sparse random matrices. / Ballard, Grey; Buluç, Aydin; Demmel, James et al.
SPAA 2013 - Proceedings of the 25th ACM Symposium on Parallelism in Algorithms and Architectures. Association for Computing Machinery, 2013. p. 222-231 (Annual ACM Symposium on Parallelism in Algorithms and Architectures).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Communication optimal parallel multiplication of sparse random matrices

AU - Ballard, Grey

AU - Buluç, Aydin

AU - Demmel, James

AU - Grigori, Laura

AU - Lipshitz, Benjamin

AU - Schwartz, Oded

AU - Toledo, Sivan

PY - 2013

Y1 - 2013

N2 - Parallel algorithms for sparse matrix-matrix multiplication typically spend most of their time on inter-processor communication rather than on computation, and hardware trends predict the relative cost of communication will only increase. Thus, sparse matrix multiplication algorithms must minimize communication costs in order to scale to large processor counts. In this paper, we consider multiplying sparse matrices corresponding to Erdo″s- Rényi random graphs on distributedmemory parallel machines. We prove a new lower bound on the expected communication cost for a wide class of algorithms. Our analysis of existing algorithms shows that, while some are optimal for a limited range of matrix density and number of processors, none is optimal in general. We obtain two new parallel algorithms and prove that they match the expected communication cost lower bound, and hence they are optimal.

AB - Parallel algorithms for sparse matrix-matrix multiplication typically spend most of their time on inter-processor communication rather than on computation, and hardware trends predict the relative cost of communication will only increase. Thus, sparse matrix multiplication algorithms must minimize communication costs in order to scale to large processor counts. In this paper, we consider multiplying sparse matrices corresponding to Erdo″s- Rényi random graphs on distributedmemory parallel machines. We prove a new lower bound on the expected communication cost for a wide class of algorithms. Our analysis of existing algorithms shows that, while some are optimal for a limited range of matrix density and number of processors, none is optimal in general. We obtain two new parallel algorithms and prove that they match the expected communication cost lower bound, and hence they are optimal.

KW - Communication-avoiding algorithms

KW - Communication-cost lower bounds

KW - Random graphs

KW - Sparse matrix multiplication

UR - http://www.scopus.com/inward/record.url?scp=84883515454&partnerID=8YFLogxK

U2 - 10.1145/2486159.2486196

DO - 10.1145/2486159.2486196

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:84883515454

SN - 9781450315722

T3 - Annual ACM Symposium on Parallelism in Algorithms and Architectures

SP - 222

EP - 231

BT - SPAA 2013 - Proceedings of the 25th ACM Symposium on Parallelism in Algorithms and Architectures

PB - Association for Computing Machinery

T2 - 25th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2013

Y2 - 23 July 2013 through 25 July 2013

ER -

Ballard G, Buluç A, Demmel J, Grigori L, Lipshitz B, Schwartz O et al. Communication optimal parallel multiplication of sparse random matrices. In SPAA 2013 - Proceedings of the 25th ACM Symposium on Parallelism in Algorithms and Architectures. Association for Computing Machinery. 2013. p. 222-231. (Annual ACM Symposium on Parallelism in Algorithms and Architectures). doi: 10.1145/2486159.2486196

Communication optimal parallel multiplication of sparse random matrices

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this