Heuristics for resource matching in Intel's compute farm

Ohad Shai, Edi Shmueli, Dror G. Feitelson*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

12 Scopus citations

Abstract

In this paper we investigate the issue of resource matching between jobs and machines in Intel's compute farm. We show that common heuristics such as Best-Fit and Worse-Fit may fail to properly utilize the available resources when applied to either cores or memory in isolation. In an attempt to overcome the problem we propose Mix-Fit, a heuristic which attempts to balance usage between resources. While this indeed usually improves upon the single-resource heuristics, it too fails to be optimal in all cases. As a solution we default to Max-Jobs, a meta-heuristic that employs all the other heuristics as sub-routines, and selects the one which matches the highest number of jobs. Extensive simulations that are based on real workload traces from four different Intel sites demonstrate that Max-Jobs is indeed the most robust heuristic for diverse workloads and system configurations, and provides up to 22 % reduction in the average wait time of jobs.

Original languageAmerican English
Title of host publicationJob Scheduling Strategies for Parallel Processing - 17th International Workshop, JSSPP 2013, Revised Selected Papers
PublisherSpringer Verlag
Pages116-135
Number of pages20
ISBN (Print)9783662437780
DOIs
StatePublished - 2014
Event17th International Workshop on Job Scheduling Strategies for Parallel Processing, JSSPP 2013 - Boston, MA, United States
Duration: 24 May 201424 May 2014

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume8429 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference17th International Workshop on Job Scheduling Strategies for Parallel Processing, JSSPP 2013
Country/TerritoryUnited States
CityBoston, MA
Period24/05/1424/05/14

Keywords

  • Best-Fit
  • First-Fit
  • Job scheduling
  • NetBatch
  • Resource matching
  • Simulation
  • Worse-Fit

Fingerprint

Dive into the research topics of 'Heuristics for resource matching in Intel's compute farm'. Together they form a unique fingerprint.

Cite this