Revisiting the I/O-complexity of fast matrix multiplication with recomputations

Roy Nissim; Oded Schwartz

doi:10.1109/IPDPS.2019.00058

Revisiting the I/O-complexity of fast matrix multiplication with recomputations

Roy Nissim, Oded Schwartz

The Rachel and Selim Benin School of Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Scopus citations

Abstract

Communication costs, between processors and across the memory hierarchy, often dominate the runtime of algorithms. Can we trade these costs for recomputations? Most algorithms do not utilize recomputation for this end, and most communication cost lower bounds assume no recomputation, hence do not address this fundamental question. Recently, Bilardi and De Stefani (2017), and Bilardi, Scquizzato, and Silvestri (2018) showed that recomputations cannot reduce communication costs in Strassen's fast matrix multiplication and in fast Fourier transform. We extend the former bound and show that recomputations cannot reduce communication costs for a few other fast matrix multiplication algorithms.

Original language	American English
Title of host publication	Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	482-490
Number of pages	9
ISBN (Electronic)	9781728112466
DOIs	https://doi.org/10.1109/IPDPS.2019.00058
State	Published - May 2019
Event	33rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2019 - Rio de Janeiro, Brazil Duration: 20 May 2019 → 24 May 2019

Publication series

Name	Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019

Conference

Conference	33rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2019
Country/Territory	Brazil
City	Rio de Janeiro
Period	20/05/19 → 24/05/19

Bibliographical note

Funding Information:
Research was supported by grants 1878/14, and 1901/14 from the Israel Science Foundation (founded by the Israel Academy of Sciences and Humanities). This work was supported by the PetaCloud industry-academia consortium. This research was supported by a grant from the United States-Israel Binational Science Foundation (BSF), Jerusalem, Israel. This work was supported by The Federmann Cyber Security Center in conjunction with the Israel national cyber directorate.

Publisher Copyright:
© 2019 IEEE

Keywords

Fast Matrix Multiplication
I/O-complexity
Memory Hierarchy
Parallel Computation
Recomputation

Access to Document

10.1109/IPDPS.2019.00058

Cite this

Nissim, R., & Schwartz, O. (2019). Revisiting the I/O-complexity of fast matrix multiplication with recomputations. In Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019 (pp. 482-490). Article 8820960 (Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IPDPS.2019.00058

Nissim, Roy ; Schwartz, Oded. / Revisiting the I/O-complexity of fast matrix multiplication with recomputations. Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 482-490 (Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019).

@inproceedings{d38281072086415aac6626b2174b38c3,

title = "Revisiting the I/O-complexity of fast matrix multiplication with recomputations",

abstract = "Communication costs, between processors and across the memory hierarchy, often dominate the runtime of algorithms. Can we trade these costs for recomputations? Most algorithms do not utilize recomputation for this end, and most communication cost lower bounds assume no recomputation, hence do not address this fundamental question. Recently, Bilardi and De Stefani (2017), and Bilardi, Scquizzato, and Silvestri (2018) showed that recomputations cannot reduce communication costs in Strassen's fast matrix multiplication and in fast Fourier transform. We extend the former bound and show that recomputations cannot reduce communication costs for a few other fast matrix multiplication algorithms.",

keywords = "Fast Matrix Multiplication, I/O-complexity, Memory Hierarchy, Parallel Computation, Recomputation",

author = "Roy Nissim and Oded Schwartz",

note = "Funding Information: Research was supported by grants 1878/14, and 1901/14 from the Israel Science Foundation (founded by the Israel Academy of Sciences and Humanities). This work was supported by the PetaCloud industry-academia consortium. This research was supported by a grant from the United States-Israel Binational Science Foundation (BSF), Jerusalem, Israel. This work was supported by The Federmann Cyber Security Center in conjunction with the Israel national cyber directorate. Publisher Copyright: {\textcopyright} 2019 IEEE; 33rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2019 ; Conference date: 20-05-2019 Through 24-05-2019",

year = "2019",

month = may,

doi = "10.1109/IPDPS.2019.00058",

language = "American English",

series = "Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "482--490",

booktitle = "Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019",

address = "United States",

}

Nissim, R & Schwartz, O 2019, Revisiting the I/O-complexity of fast matrix multiplication with recomputations. in Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019., 8820960, Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019, Institute of Electrical and Electronics Engineers Inc., pp. 482-490, 33rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2019, Rio de Janeiro, Brazil, 20/05/19. https://doi.org/10.1109/IPDPS.2019.00058

Revisiting the I/O-complexity of fast matrix multiplication with recomputations. / Nissim, Roy; Schwartz, Oded.
Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 482-490 8820960 (Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Revisiting the I/O-complexity of fast matrix multiplication with recomputations

AU - Nissim, Roy

AU - Schwartz, Oded

N1 - Funding Information: Research was supported by grants 1878/14, and 1901/14 from the Israel Science Foundation (founded by the Israel Academy of Sciences and Humanities). This work was supported by the PetaCloud industry-academia consortium. This research was supported by a grant from the United States-Israel Binational Science Foundation (BSF), Jerusalem, Israel. This work was supported by The Federmann Cyber Security Center in conjunction with the Israel national cyber directorate. Publisher Copyright: © 2019 IEEE

PY - 2019/5

Y1 - 2019/5

N2 - Communication costs, between processors and across the memory hierarchy, often dominate the runtime of algorithms. Can we trade these costs for recomputations? Most algorithms do not utilize recomputation for this end, and most communication cost lower bounds assume no recomputation, hence do not address this fundamental question. Recently, Bilardi and De Stefani (2017), and Bilardi, Scquizzato, and Silvestri (2018) showed that recomputations cannot reduce communication costs in Strassen's fast matrix multiplication and in fast Fourier transform. We extend the former bound and show that recomputations cannot reduce communication costs for a few other fast matrix multiplication algorithms.

AB - Communication costs, between processors and across the memory hierarchy, often dominate the runtime of algorithms. Can we trade these costs for recomputations? Most algorithms do not utilize recomputation for this end, and most communication cost lower bounds assume no recomputation, hence do not address this fundamental question. Recently, Bilardi and De Stefani (2017), and Bilardi, Scquizzato, and Silvestri (2018) showed that recomputations cannot reduce communication costs in Strassen's fast matrix multiplication and in fast Fourier transform. We extend the former bound and show that recomputations cannot reduce communication costs for a few other fast matrix multiplication algorithms.

KW - Fast Matrix Multiplication

KW - I/O-complexity

KW - Memory Hierarchy

KW - Parallel Computation

KW - Recomputation

UR - http://www.scopus.com/inward/record.url?scp=85072843749&partnerID=8YFLogxK

U2 - 10.1109/IPDPS.2019.00058

DO - 10.1109/IPDPS.2019.00058

M3 - Conference contribution

AN - SCOPUS:85072843749

T3 - Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019

SP - 482

EP - 490

BT - Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 33rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2019

Y2 - 20 May 2019 through 24 May 2019

ER -

Nissim R, Schwartz O. Revisiting the I/O-complexity of fast matrix multiplication with recomputations. In Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 482-490. 8820960. (Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019). doi: 10.1109/IPDPS.2019.00058

Revisiting the I/O-complexity of fast matrix multiplication with recomputations

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this