TY - JOUR

T1 - Graph expansion and communication costs of fast matrix multiplication

AU - Ballard, Grey

AU - Demmel, James

AU - Holtz, Olga

AU - Schwartz, Oded

PY - 2012/12

Y1 - 2012/12

N2 - The communication cost of algorithms (also known as I/O-complexity) is shown to be closely related to the expansion properties of the corresponding computation graphs.We demonstrate this on Strassen's and other fast matrix multiplication algorithms, and obtain the first lower bounds on their communication costs. In the sequential case, where the processor has a fast memory of size M, too small to store three n-by-n matrices, the lower bound on the number of words moved between fast and slow memory is, for a large class of matrix multiplication algorithms, ((n/√M)ω0 ·M), where ω0 is the exponent in the arithmetic count (e.g., ω0 = lg 7 for Strassen, and ω0 = 3 for conventional matrix multiplication). With p parallel processors, each with fast memory of size M, the lower bound is asymptotically lower by a factor of p. These bounds are attainable both for sequential and for parallel algorithms and hence optimal.

AB - The communication cost of algorithms (also known as I/O-complexity) is shown to be closely related to the expansion properties of the corresponding computation graphs.We demonstrate this on Strassen's and other fast matrix multiplication algorithms, and obtain the first lower bounds on their communication costs. In the sequential case, where the processor has a fast memory of size M, too small to store three n-by-n matrices, the lower bound on the number of words moved between fast and slow memory is, for a large class of matrix multiplication algorithms, ((n/√M)ω0 ·M), where ω0 is the exponent in the arithmetic count (e.g., ω0 = lg 7 for Strassen, and ω0 = 3 for conventional matrix multiplication). With p parallel processors, each with fast memory of size M, the lower bound is asymptotically lower by a factor of p. These bounds are attainable both for sequential and for parallel algorithms and hence optimal.

KW - Communication-avoiding algorithms

KW - Fast matrix multiplication

KW - I/Ocomplexit

UR - http://www.scopus.com/inward/record.url?scp=84872481259&partnerID=8YFLogxK

U2 - 10.1145/2395116.2395121

DO - 10.1145/2395116.2395121

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???

AN - SCOPUS:84872481259

SN - 0004-5411

VL - 59

JO - Journal of the ACM

JF - Journal of the ACM

IS - 6

M1 - 32

ER -