A Co-Plot analysis of logs and models of parallel workloads

David Talby*, Dror G. Feitelson, Adi Raveh

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

7 Scopus citations

Abstract

We present a multivariate analysis technique called Co-Plot that is especially suitable for few samples of many variables. Co-Plot embeds the multidimensional samples in two dimensions, in a way that allows key variables to be identified, and relations between both variables and observations to be analyzed together. When applied to the workloads on parallel supercomputers, we find two stable perpendicular axes of highly correlated variables, one representing individual job attributes and the other representing multijob attributes. The different workloads, on the other hand, are rather different from one another, and may also change over time. Synthetic models for workload generation are also analyzed, and found to be reasonable in the sense that they span the same range of variable combinations as the real workloads. However, the spread of real workloads implies that a single model cannot be similar to all of them. This leads us to construct a parameterized model, with parameters that correspond to the two axes identified above. We also find that existing models do not model the temporal structure of the workload well, and hence are wanting for tasks such as comparing schedulers, and that the common methodology for load manipulation of workloads is problematic.

Original languageAmerican English
Article number1243993
JournalACM Transactions on Modeling and Computer Simulation
Volume17
Issue number3
DOIs
StatePublished - 1 Jul 2007

Keywords

  • Co-plot
  • Load manipulation
  • Multivariate analysis
  • Nonstationary workload
  • Parallel workloads
  • Parametric model
  • Synthetic workload
  • Workload modeling

Fingerprint

Dive into the research topics of 'A Co-Plot analysis of logs and models of parallel workloads'. Together they form a unique fingerprint.

Cite this