Parallel job scheduling under dynamic workloads

Eitan Frachtenberg, Dror G. Feitelson, Juan Fernandez, Fabrizio Petrini

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review

15 Scopus citations

Abstract

Jobs that run on parallel systems that use gang scheduling for multiprogramming may interact with each other in various ways. These interactions are affected by system parameters such as the level of multiprogramming and the scheduling time quantum. A careful evaluation is therefore required in order to find parameter values that lead to optimal performance. We perform a detailed performance evaluation of three factors affecting scheduling systems running dynamic workloads: multiprogramming level, time quantum, and the use of backfilling for queue management - and how they depend on offered load. Our evaluation is based on synthetic MPI applications running on a real cluster that actually implements the various scheduling schemes. Our results demonstrate the importance of both components of the gang-scheduling plus backfilling combination: gang scheduling reduces response time and slowdown, and backfilling allows doing so with a limited multiprogramming level. This is further improved by using flexible coscheduling rather than strict gang scheduling, as this reduces the constraints and allows for a denser packing.

Original languageAmerican English
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
EditorsDror Feitelson, Larry Rudolph, Uwe Schwiegelshohn
PublisherSpringer Verlag
Pages208-227
Number of pages20
ISBN (Print)9783540397274
DOIs
StatePublished - 2003

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume2862
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Keywords

  • Cluster computing
  • Dynamic workloads
  • Flexible coscheduling
  • Gang scheduling
  • Heterogeneous clusters
  • Job scheduling
  • Parallel architectures
  • STORM

Fingerprint

Dive into the research topics of 'Parallel job scheduling under dynamic workloads'. Together they form a unique fingerprint.

Cite this