Trade-offs between communication throughput and parallel time

Yishay Mansour*, Noam Nisan, Uzi Vishkin

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

We study the effect of limited communication throughput on parallel computation in a setting where the number of processors is much smaller than the length of the input. Our model haspprocessors that communicate through a shared memory of sizem. The input has sizenand can be read directly by all the processors. We will be primarily interested in studying cases wheren≫p≫m. As a test case we study the list reversal problem. For this problem we prove a time lower bound ofΩ(n/mp). (A similar lower bound holds also for the problems of sorting, finding all unique elements, convolution, and universal hashing.) This result demonstrates that limiting the communication (i.e., smallm) could have significant effect on parallel computation. We show an almost matching upper bound ofO((n/mp)logO(1)n). The upper bound requires the development of a few interesting techniques which can alleviate the limited communication in some general settings. Specifically, we show how to emulate a large shared memory on a limited shared memory efficiently. The lower bound applies even to randomized machines, and the upper bound is a randomized algorithm. We also argue that some standard methodology for designing parallel algorithms appears to require a relatively high level of communication throughput. Our results suggest that new alternative methodologies that need a lower such level must be invented for parallel machines that enable a low level of communication throughput, since otherwise those machines will be severly handicapped as general purpose parallel machines. Although we do not rule that out, we cannot offer any encouraging evidence to suggest that such new methodologies are likely to be found.

Original languageAmerican English
Pages (from-to)148-166
Number of pages19
JournalJournal of Complexity
Volume15
Issue number1
DOIs
StatePublished - Mar 1999

Bibliographical note

Funding Information:
* This author was supported in part by The Israel Science Foundation administered by The Israel Academy of Science and Humanities and by a grant of the Israelly Ministry of Science and Technology. -This author was supported in part by BSF grant 92-00043 and by a Wolfeson award administered by the Israeli Academy of Sciences. This author was supported in part by NSF grants CCR-9111348 and CCR-9416890.

Fingerprint

Dive into the research topics of 'Trade-offs between communication throughput and parallel time'. Together they form a unique fingerprint.

Cite this