Abstract
This paper presents the Modular Workload Format (MWF), a proposal for extending the widely accepted Standard Workload Format (SWF) for job scheduling evaluation. David Talby and Dror Feitelson proposed the SWF in 1999, allowing to describe data center workload in a synthesized way. Its simplicity, representing each job by a single line in a text file and including details to make job scheduling evaluation quite accurate, was part of its success. Using these years’ experience but considering new system and workload characteristics, we propose an extension to support multiple steps in a single job, heterogeneous jobs, and relevant inputs not covered by the SWF as energy/power references. The goal of this contribution is to adapt the SWF to current trends in architectures and workloads. Moreover, we propose a simple approach for converting any already existing SWF trace file into an MWF trace file to be able to reuse already existing traces.
This work is partially supported from the European Union’s Horizon 2020 under grant agreement No. 754304 (DEEP-EST Project) and the Spanish grant PID2019-107255GB-C21.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The fields description comes from the SWF web page.
References
DEEP-EST project. http://www.deep-projects.eu/
DEEP-ER Deliverables. http://www.deep-projects.eu/project/deliverables.html
JUBE Online Documentation. https://apps.fz-juelich.de/jsc/jube/jube2/docu/index.html
MPI LinkTest. http://www.fz-juelich.de/ias/jsc/EN/Expertise/Support/Software/LinkTest/_node.html
SIONlib. http://www.fz-juelich.de/ias/jsc/EN/Expertise/Support/Software/SIONlib/_node.html
HDF5 Group. https://support.hdfgroup.org/HDF5/
The Standard Workload Format. http://www.cs.huji.ac.il/labs/parallel/workload/swf.html
Chapin, S.J., et al.: Benchmarks and standards for the evaluation of parallel job schedulers. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1999. LNCS, vol. 1659, pp. 67–90. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-47954-6_4
Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple linux utility for resource management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 44–60. Springer, Heidelberg (2003). https://doi.org/10.1007/10968987_3
SchedMD: Slurm Workload Manager. https://slurm.schedmd.com/
sbatch: Submit a batch script to Slurm. https://slurm.schedmd.com/sbatch.html
Heterogeneous Resources and MPMD. https://slurm.schedmd.com/SLUG15/Heterogeneous_Resources_and_MPMD.pdf
SLURM: Heterogeneous job Support. https://slurm.schedmd.com/SLUG17/HeterogeneousJobs.pdf
DEEP-EST Deliverable 1.1 Application co-design input. https://www.deep-projects.eu/images/materials/D11.pdf
BSC Slurm Simulator code. https://github.com/BSC-RM/slurm_simulator_tools
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Corbalan, J., D’Amico, M. (2021). Modular Workload Format: Extending SWF for Modular Systems. In: Klusáček, D., Cirne, W., Rodrigo, G.P. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2021. Lecture Notes in Computer Science(), vol 12985. Springer, Cham. https://doi.org/10.1007/978-3-030-88224-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-88224-2_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88223-5
Online ISBN: 978-3-030-88224-2
eBook Packages: Computer ScienceComputer Science (R0)