Matrix multiplication, a little faster

Elaye Karstadt; Oded Schwartz

doi:10.1145/3087556.3087579

Matrix multiplication, a little faster

Elaye Karstadt, Oded Schwartz

The Rachel and Selim Benin School of Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

15 Scopus citations

Abstract

Strassen's algorithm (1969) was the first sub-cubic matrix multiplication algorithm. Winograd (1971) improved its complexity by a constant factor. Many asymptotic improvements followed. Unfortunately, most of them have done so at the cost of very large, o.en gigantic, hidden constants. Consequently, Strassen-Winograd's O (nlog2⁷) algorithm o.en outperforms other matrix multiplication algorithms for all feasible matrix dimensions. The leading coefficient of Strassen-Winograd's algorithm was believed to be optimal for matrix multiplication algorithms with 2 × 2 base case, due to a lower bound of Probert (1976). Surprisingly, we obtain a faster matrix multiplication algorithm, with the same base case size and asymptotic complexity as Strassen- Winograd's algorithm, but with the coefficient reduced from 6 to 5. To this end, we extend Bodrato's (2010) method for matrix squaring, and transform matrices to an alternative basis. We prove a generalization of Probert's lower bound that holds under change of basis, showing that for matrix multiplication algorithms with a 2×2 base case, the leading coefficient of our algorithm cannot be further reduced, hence optimal. We apply our technique to other Strassenlike algorithms, improving their arithmetic and communication costs by significant constant factors.

Original language	American English
Title of host publication	SPAA 2017 - Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures
Publisher	Association for Computing Machinery
Pages	101-110
Number of pages	10
ISBN (Electronic)	9781450345934
DOIs	https://doi.org/10.1145/3087556.3087579
State	Published - 24 Jul 2017
Event	29th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2017 - Washington, United States Duration: 24 Jul 2017 → 26 Jul 2017

Publication series

Name	Annual ACM Symposium on Parallelism in Algorithms and Architectures
Volume	Part F129316

Conference

Conference	29th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2017
Country/Territory	United States
City	Washington
Period	24/07/17 → 26/07/17

Bibliographical note

Funding Information:
Research is supported by grants 1878/14, and 1901/14 from the Israel Science Foundation (founded by the Israel Academy of Sciences

Funding Information:
and Humanities) and grant 3-10891 from the Ministry of Science and Technology, Israel. Research is also supported by the Einstein Foundation and the Minerva Foundation. Œis work was supported by the PetaCloud industry-academia consortium. Œis research was supported by a grant from the United States-Israel Bi-national Science Foundation (BSF), Jerusalem, Israel. Œis work was supported by the HUJI Cyber Security Research Center in conjunction with the Israel National Cyber Bureau in the Prime Minister’s Oce. We acknowledge PRACE for awarding us access to Hazel Hen at GCS@HLRS, Germany.

Publisher Copyright:
© 2017 Copyright held by the owner/author(s).

Keywords

Bilinear algorithms
Fast matrix multiplication

Access to Document

10.1145/3087556.3087579

Cite this

@inproceedings{88b722fd005d4029a6f1a35a9ef32ab9,

title = "Matrix multiplication, a little faster",

abstract = "Strassen's algorithm (1969) was the first sub-cubic matrix multiplication algorithm. Winograd (1971) improved its complexity by a constant factor. Many asymptotic improvements followed. Unfortunately, most of them have done so at the cost of very large, o.en gigantic, hidden constants. Consequently, Strassen-Winograd's O (nlog27) algorithm o.en outperforms other matrix multiplication algorithms for all feasible matrix dimensions. The leading coefficient of Strassen-Winograd's algorithm was believed to be optimal for matrix multiplication algorithms with 2 × 2 base case, due to a lower bound of Probert (1976). Surprisingly, we obtain a faster matrix multiplication algorithm, with the same base case size and asymptotic complexity as Strassen- Winograd's algorithm, but with the coefficient reduced from 6 to 5. To this end, we extend Bodrato's (2010) method for matrix squaring, and transform matrices to an alternative basis. We prove a generalization of Probert's lower bound that holds under change of basis, showing that for matrix multiplication algorithms with a 2×2 base case, the leading coefficient of our algorithm cannot be further reduced, hence optimal. We apply our technique to other Strassenlike algorithms, improving their arithmetic and communication costs by significant constant factors.",

keywords = "Bilinear algorithms, Fast matrix multiplication",

author = "Elaye Karstadt and Oded Schwartz",

note = "Funding Information: Research is supported by grants 1878/14, and 1901/14 from the Israel Science Foundation (founded by the Israel Academy of Sciences Funding Information: and Humanities) and grant 3-10891 from the Ministry of Science and Technology, Israel. Research is also supported by the Einstein Foundation and the Minerva Foundation. {\OE}is work was supported by the PetaCloud industry-academia consortium. {\OE}is research was supported by a grant from the United States-Israel Bi-national Science Foundation (BSF), Jerusalem, Israel. {\OE}is work was supported by the HUJI Cyber Security Research Center in conjunction with the Israel National Cyber Bureau in the Prime Minister{\textquoteright}s Oce. We acknowledge PRACE for awarding us access to Hazel Hen at GCS@HLRS, Germany. Publisher Copyright: {\textcopyright} 2017 Copyright held by the owner/author(s).; 29th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2017 ; Conference date: 24-07-2017 Through 26-07-2017",

year = "2017",

month = jul,

day = "24",

doi = "10.1145/3087556.3087579",

language = "American English",

series = "Annual ACM Symposium on Parallelism in Algorithms and Architectures",

publisher = "Association for Computing Machinery",

pages = "101--110",

booktitle = "SPAA 2017 - Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures",

}

Karstadt, E & Schwartz, O 2017, Matrix multiplication, a little faster. in SPAA 2017 - Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures. Annual ACM Symposium on Parallelism in Algorithms and Architectures, vol. Part F129316, Association for Computing Machinery, pp. 101-110, 29th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2017, Washington, United States, 24/07/17. https://doi.org/10.1145/3087556.3087579

Matrix multiplication, a little faster. / Karstadt, Elaye; Schwartz, Oded.
SPAA 2017 - Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures. Association for Computing Machinery, 2017. p. 101-110 (Annual ACM Symposium on Parallelism in Algorithms and Architectures; Vol. Part F129316).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Matrix multiplication, a little faster

AU - Karstadt, Elaye

AU - Schwartz, Oded

N1 - Funding Information: Research is supported by grants 1878/14, and 1901/14 from the Israel Science Foundation (founded by the Israel Academy of Sciences Funding Information: and Humanities) and grant 3-10891 from the Ministry of Science and Technology, Israel. Research is also supported by the Einstein Foundation and the Minerva Foundation. Œis work was supported by the PetaCloud industry-academia consortium. Œis research was supported by a grant from the United States-Israel Bi-national Science Foundation (BSF), Jerusalem, Israel. Œis work was supported by the HUJI Cyber Security Research Center in conjunction with the Israel National Cyber Bureau in the Prime Minister’s Oce. We acknowledge PRACE for awarding us access to Hazel Hen at GCS@HLRS, Germany. Publisher Copyright: © 2017 Copyright held by the owner/author(s).

PY - 2017/7/24

Y1 - 2017/7/24

N2 - Strassen's algorithm (1969) was the first sub-cubic matrix multiplication algorithm. Winograd (1971) improved its complexity by a constant factor. Many asymptotic improvements followed. Unfortunately, most of them have done so at the cost of very large, o.en gigantic, hidden constants. Consequently, Strassen-Winograd's O (nlog27) algorithm o.en outperforms other matrix multiplication algorithms for all feasible matrix dimensions. The leading coefficient of Strassen-Winograd's algorithm was believed to be optimal for matrix multiplication algorithms with 2 × 2 base case, due to a lower bound of Probert (1976). Surprisingly, we obtain a faster matrix multiplication algorithm, with the same base case size and asymptotic complexity as Strassen- Winograd's algorithm, but with the coefficient reduced from 6 to 5. To this end, we extend Bodrato's (2010) method for matrix squaring, and transform matrices to an alternative basis. We prove a generalization of Probert's lower bound that holds under change of basis, showing that for matrix multiplication algorithms with a 2×2 base case, the leading coefficient of our algorithm cannot be further reduced, hence optimal. We apply our technique to other Strassenlike algorithms, improving their arithmetic and communication costs by significant constant factors.

AB - Strassen's algorithm (1969) was the first sub-cubic matrix multiplication algorithm. Winograd (1971) improved its complexity by a constant factor. Many asymptotic improvements followed. Unfortunately, most of them have done so at the cost of very large, o.en gigantic, hidden constants. Consequently, Strassen-Winograd's O (nlog27) algorithm o.en outperforms other matrix multiplication algorithms for all feasible matrix dimensions. The leading coefficient of Strassen-Winograd's algorithm was believed to be optimal for matrix multiplication algorithms with 2 × 2 base case, due to a lower bound of Probert (1976). Surprisingly, we obtain a faster matrix multiplication algorithm, with the same base case size and asymptotic complexity as Strassen- Winograd's algorithm, but with the coefficient reduced from 6 to 5. To this end, we extend Bodrato's (2010) method for matrix squaring, and transform matrices to an alternative basis. We prove a generalization of Probert's lower bound that holds under change of basis, showing that for matrix multiplication algorithms with a 2×2 base case, the leading coefficient of our algorithm cannot be further reduced, hence optimal. We apply our technique to other Strassenlike algorithms, improving their arithmetic and communication costs by significant constant factors.

KW - Bilinear algorithms

KW - Fast matrix multiplication

UR - http://www.scopus.com/inward/record.url?scp=85027875930&partnerID=8YFLogxK

U2 - 10.1145/3087556.3087579

DO - 10.1145/3087556.3087579

M3 - Conference contribution

AN - SCOPUS:85027875930

T3 - Annual ACM Symposium on Parallelism in Algorithms and Architectures

SP - 101

EP - 110

BT - SPAA 2017 - Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures

PB - Association for Computing Machinery

T2 - 29th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2017

Y2 - 24 July 2017 through 26 July 2017

ER -

Matrix multiplication, a little faster

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this