Brief announcement: Strong scaling of matrix multiplication algorithms and memory-independent communication lower bounds

Grey Ballard*, James Demmel, Olga Holtz, Benjamin Lipshitz, Oded Schwartz

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

43 Scopus citations

Abstract

A parallel algorithm has perfect strong scaling if its running time on P processors is linear in 1/P, including all communication costs. Distributed-memory parallel algorithms for matrix multiplication with perfect strong scaling have only recently been found. One is based on classical matrix multiplication (Solomonik and Demmel, 2011), and one is based on Strassen's fast matrix multiplication (Ballard, Demmel, Holtz, Lipshitz, and Schwartz, 2012). Both algorithms scale perfectly, but only up to some number of processors where the inter-processor communication no longer scales. We obtain a memory-independent communication cost lower bound on classical and Strassen-based distributed-memory matrix multiplication algorithms. These bounds imply that no classical or Strassen-based parallel matrix multiplication algorithm can strongly scale perfectly beyond the ranges already attained by the two parallel algorithms mentioned above. The memory-independent bounds and the strong scaling bounds generalize to other algorithms. Copyright is held by the author/owner(s).

Original languageAmerican English
Title of host publicationSPAA'12 - Proceedings of the 24th ACM Symposium on Parallelism in Algorithms and Architectures
Pages77-79
Number of pages3
DOIs
StatePublished - 2012
Externally publishedYes
Event24th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA'12 - Pittsburgh, PA, United States
Duration: 25 Jun 201227 Jun 2012

Publication series

NameAnnual ACM Symposium on Parallelism in Algorithms and Architectures

Conference

Conference24th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA'12
Country/TerritoryUnited States
CityPittsburgh, PA
Period25/06/1227/06/12

Keywords

  • Communication-avoiding algorithms
  • Fast matrix multiplication
  • Strong scaling

Fingerprint

Dive into the research topics of 'Brief announcement: Strong scaling of matrix multiplication algorithms and memory-independent communication lower bounds'. Together they form a unique fingerprint.

Cite this