## Abstract

Communication-optimal algorithms are known for square matrix multiplication. Here, we obtain the first communication-optimal algorithm for all dimensions of rectangular matrices. Combining the dimension-splitting technique of Frigo, Leiserson, Prokop and Ramachandran (1999) with the recursive BFS/DFS approach of Ballard, Demmel, Holtz, Lipshitz and Schwartz (2012) allows for a communication-optimal as well as cache- and network-oblivious algorithm. Moreover, the implementation is simple: approximately 50 lines of code for the shared-memory version. Since the new algorithm minimizes communication across the network, between NUMA domains, and between levels of cache, it performs well in practice on both shared- and distributed-memory machines. We show significant speedups over existing parallel linear algebra libraries both on a 32-core shared-memory machine and on a distributed-memory supercomputer.

Original language | American English |
---|---|

Pages | 261-272 |

Number of pages | 12 |

DOIs | |

State | Published - 2013 |

Externally published | Yes |

Event | 27th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2013 - Boston, MA, United States Duration: 20 May 2013 → 24 May 2013 |

### Conference

Conference | 27th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2013 |
---|---|

Country/Territory | United States |

City | Boston, MA |

Period | 20/05/13 → 24/05/13 |

## Keywords

- linear algebra
- matrix multiplication
- ommunication-avoiding algorithms