The Pig Mix Benchmark on Pig, MapReduce, and HPCC Systems

Keren Ouaknine; Michael Carey; Scott Kirkpatrick

doi:10.1109/BigDataCongress.2015.99

The Pig Mix Benchmark on Pig, MapReduce, and HPCC Systems

Keren Ouaknine
, Michael Carey
, Scott Kirkpatrick

The Rachel and Selim Benin School of Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

16 Scopus citations

Abstract

Soon after Google published MapReduce, their paradigm for processing large amounts of data, the open-source world followed with the Hadoop ecosystem. Later on, Lexis Nexis, the company behind the world's largest database of legal documents, open-sourced its Big Data processing platform, called the High-Performance Computing Cluster (HPCC). This paper makes three contributions. First, we describe our additions and improvements to the Pig Mix benchmark, the set of queries originally written for Apache Pig, and the porting of Pig Mix to HPCC. Second, we compare the performance of queries written in Pig, Java MapReduce, and ECL. Last, we draw conclusions and issue recommendations for future system benchmarks and large-scale data-processing platforms.

Original language	English
Title of host publication	Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015
Editors	Carminati Barbara, Latifur Khan
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	643-648
Number of pages	6
ISBN (Electronic)	9781467372787
DOIs	https://doi.org/10.1109/BigDataCongress.2015.99
State	Published - 17 Aug 2015
Event	4th IEEE International Congress on Big Data, BigData Congress 2015 - New York City, United States Duration: 27 Jun 2015 → 2 Jul 2015

Publication series

Name	Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015

Conference

Conference	4th IEEE International Congress on Big Data, BigData Congress 2015
Country/Territory	United States
City	New York City
Period	27/06/15 → 2/07/15

Bibliographical note

Publisher Copyright:
© 2015 IEEE.

Keywords

Benchmark
Big Data
HPCC Systems
MapReduce
Performance
PigMix

Access to Document

10.1109/BigDataCongress.2015.99

Cite this

Ouaknine, K., Carey, M., & Kirkpatrick, S. (2015). The Pig Mix Benchmark on Pig, MapReduce, and HPCC Systems. In C. Barbara, & L. Khan (Eds.), Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015 (pp. 643-648). (Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigDataCongress.2015.99

Ouaknine, Keren ; Carey, Michael ; Kirkpatrick, Scott. / The Pig Mix Benchmark on Pig, MapReduce, and HPCC Systems. Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015. editor / Carminati Barbara ; Latifur Khan. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 643-648 (Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015).

@inproceedings{cafe67a421ca4330bb23ac7319f3683c,

title = "The Pig Mix Benchmark on Pig, MapReduce, and HPCC Systems",

abstract = "Soon after Google published MapReduce, their paradigm for processing large amounts of data, the open-source world followed with the Hadoop ecosystem. Later on, Lexis Nexis, the company behind the world's largest database of legal documents, open-sourced its Big Data processing platform, called the High-Performance Computing Cluster (HPCC). This paper makes three contributions. First, we describe our additions and improvements to the Pig Mix benchmark, the set of queries originally written for Apache Pig, and the porting of Pig Mix to HPCC. Second, we compare the performance of queries written in Pig, Java MapReduce, and ECL. Last, we draw conclusions and issue recommendations for future system benchmarks and large-scale data-processing platforms.",

keywords = "Benchmark, Big Data, HPCC Systems, MapReduce, Performance, PigMix",

author = "Keren Ouaknine and Michael Carey and Scott Kirkpatrick",

note = "Publisher Copyright: {\textcopyright} 2015 IEEE.; 4th IEEE International Congress on Big Data, BigData Congress 2015 ; Conference date: 27-06-2015 Through 02-07-2015",

year = "2015",

month = aug,

day = "17",

doi = "10.1109/BigDataCongress.2015.99",

language = "אנגלית",

series = "Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "643--648",

editor = "Carminati Barbara and Latifur Khan",

booktitle = "Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015",

address = "ארצות הברית",

}

Ouaknine, K, Carey, M & Kirkpatrick, S 2015, The Pig Mix Benchmark on Pig, MapReduce, and HPCC Systems. in C Barbara & L Khan (eds), Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015. Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015, Institute of Electrical and Electronics Engineers Inc., pp. 643-648, 4th IEEE International Congress on Big Data, BigData Congress 2015, New York City, United States, 27/06/15. https://doi.org/10.1109/BigDataCongress.2015.99

The Pig Mix Benchmark on Pig, MapReduce, and HPCC Systems. / Ouaknine, Keren; Carey, Michael; Kirkpatrick, Scott.
Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015. ed. / Carminati Barbara; Latifur Khan. Institute of Electrical and Electronics Engineers Inc., 2015. p. 643-648 (Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - The Pig Mix Benchmark on Pig, MapReduce, and HPCC Systems

AU - Ouaknine, Keren

AU - Carey, Michael

AU - Kirkpatrick, Scott

PY - 2015/8/17

Y1 - 2015/8/17

N2 - Soon after Google published MapReduce, their paradigm for processing large amounts of data, the open-source world followed with the Hadoop ecosystem. Later on, Lexis Nexis, the company behind the world's largest database of legal documents, open-sourced its Big Data processing platform, called the High-Performance Computing Cluster (HPCC). This paper makes three contributions. First, we describe our additions and improvements to the Pig Mix benchmark, the set of queries originally written for Apache Pig, and the porting of Pig Mix to HPCC. Second, we compare the performance of queries written in Pig, Java MapReduce, and ECL. Last, we draw conclusions and issue recommendations for future system benchmarks and large-scale data-processing platforms.

AB - Soon after Google published MapReduce, their paradigm for processing large amounts of data, the open-source world followed with the Hadoop ecosystem. Later on, Lexis Nexis, the company behind the world's largest database of legal documents, open-sourced its Big Data processing platform, called the High-Performance Computing Cluster (HPCC). This paper makes three contributions. First, we describe our additions and improvements to the Pig Mix benchmark, the set of queries originally written for Apache Pig, and the porting of Pig Mix to HPCC. Second, we compare the performance of queries written in Pig, Java MapReduce, and ECL. Last, we draw conclusions and issue recommendations for future system benchmarks and large-scale data-processing platforms.

KW - Benchmark

KW - Big Data

KW - HPCC Systems

KW - MapReduce

KW - Performance

KW - PigMix

UR - https://www.scopus.com/pages/publications/84959550992

U2 - 10.1109/BigDataCongress.2015.99

DO - 10.1109/BigDataCongress.2015.99

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:84959550992

T3 - Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015

SP - 643

EP - 648

BT - Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015

A2 - Barbara, Carminati

A2 - Khan, Latifur

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 4th IEEE International Congress on Big Data, BigData Congress 2015

Y2 - 27 June 2015 through 2 July 2015

ER -

Ouaknine K, Carey M, Kirkpatrick S. The Pig Mix Benchmark on Pig, MapReduce, and HPCC Systems. In Barbara C, Khan L, editors, Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 643-648. (Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015). doi: 10.1109/BigDataCongress.2015.99

The Pig Mix Benchmark on Pig, MapReduce, and HPCC Systems

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this