TY - GEN
T1 - Communication-optimal parallel recursive rectangular matrix multiplication
AU - Demmel, James
AU - Eliahu, David
AU - Fox, Armando
AU - Kamil, Shoaib
AU - Lipshitz, Benjamin
AU - Schwartz, Oded
AU - Spillinger, Omer
PY - 2013
Y1 - 2013
N2 - Communication-optimal algorithms are known for square matrix multiplication. Here, we obtain the first communication-optimal algorithm for all dimensions of rectangular matrices. Combining the dimension-splitting technique of Frigo, Leiserson, Prokop and Ramachandran (1999) with the recursive BFS/DFS approach of Ballard, Demmel, Holtz, Lipshitz and Schwartz (2012) allows for a communication-optimal as well as cache- and network-oblivious algorithm. Moreover, the implementation is simple: approximately 50 lines of code for the shared-memory version. Since the new algorithm minimizes communication across the network, between NUMA domains, and between levels of cache, it performs well in practice on both shared- and distributed-memory machines. We show significant speedups over existing parallel linear algebra libraries both on a 32-core shared-memory machine and on a distributed-memory supercomputer.
AB - Communication-optimal algorithms are known for square matrix multiplication. Here, we obtain the first communication-optimal algorithm for all dimensions of rectangular matrices. Combining the dimension-splitting technique of Frigo, Leiserson, Prokop and Ramachandran (1999) with the recursive BFS/DFS approach of Ballard, Demmel, Holtz, Lipshitz and Schwartz (2012) allows for a communication-optimal as well as cache- and network-oblivious algorithm. Moreover, the implementation is simple: approximately 50 lines of code for the shared-memory version. Since the new algorithm minimizes communication across the network, between NUMA domains, and between levels of cache, it performs well in practice on both shared- and distributed-memory machines. We show significant speedups over existing parallel linear algebra libraries both on a 32-core shared-memory machine and on a distributed-memory supercomputer.
KW - linear algebra
KW - matrix multiplication
KW - ommunication-avoiding algorithms
UR - https://www.scopus.com/pages/publications/84884883916
U2 - 10.1109/IPDPS.2013.80
DO - 10.1109/IPDPS.2013.80
M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???
AN - SCOPUS:84884883916
T3 - Proceedings - IEEE 27th International Parallel and Distributed Processing Symposium, IPDPS 2013
SP - 261
EP - 272
BT - Proceedings - IEEE 27th International Parallel and Distributed Processing Symposium, IPDPS 2013
PB - IEEE Computer Society
T2 - 27th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2013
Y2 - 20 May 2013 through 24 May 2013
ER -