Abstract
This chapter describes methodologies to perform in situ computations at desired intervals along with the simulations for different execution modes. This needs to be done in a way such that the simulation throughput is minimally impacted and the analysis output is available immediately within desired intervals. We describe the formulation of optimal resource allocation for simulation and in situ analysis computations as constrained integer linear programs so that the end-to-end simulation-analysis time is minimized. In particular, we describe the scheduling of in situ analyses as a numerical optimization problem to maximize the number of online analyses and minimize overall runtime, subject to resource constraints such as I/O bandwidth, network bandwidth, rate of computation and available memory. We also demonstrate the effectiveness of our approach through real application case studies on supercomputers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Argonne Leadership Computing Facility’s Supercomputer Mira. http://www.alcf.anl.gov/mira
Computing Resources at Argonne Leadership Computing Facility. https://www.alcf.anl.gov/computing-resources
Allen, M.P., Tildesley, D.J.: Computer Simulation of Liquids. Oxford Science Publications (1989)
Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press (2004)
Brooke, A., Kendrick, D., Meeraus, A.: GAMS: A User’s Guide. The Scientific Press, South San Francisco, California (1988)
Multi-objective Management in Freight Logistics: Increasing Capacity, Service Level and Safety with Optimization Algorithms. Multi-objective Optimization, pp. 11–36. Springer London, London (2008)
Fourer, R., Gay, D.M., Kernighan, B.W.: AMPL: A Modeling Language for Mathematical Programming, 2 edn. Duxbury Press (2003)
Fryxell, B., Olson, K., Ricker, P., Timmes, F.X., Zingale, M., Lamb, D.Q., MacNeice, P., Rosner, R., Truran, J.W., Tufo, H.: FLASH: an adaptive mesh hydrodynamics code for modeling astrophysical thermonuclear flashes. Astrophys. J. Suppl. 131, 273–334 (2000)
Gerber, R., Allcock, W., Beggio, C., Campbell, S., Cherry, A., Cholia, S., Dart, E., England, C., Fahey, T., Foertter, F., Goldstone, R., Hick, J., Karelitz, D., Kelly, K., Monroe, L., Prabhat, Skinner, D., White, J.: DOE High Performance Computing Operational Review (HPCOR): Enabling Data-Driven Scientific Discovery at HPC Facilities. Techmical report, Berkeley, CA (US) (2014)
LAMMPS Molecular Dynamics Simulator. http://lammps.sandia.gov
Malakar, P., Vishwanath, V., Knight, C., Munson, T., Papka, M.E.: Optimal execution of co-analysis for large-scale molecular dynamics simulations. In: SC16: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 702–715 (2016)
Malakar, P., Vishwanath, V., Munson, T., Knight, C., Hereld, M., Leyffer, S., Papka, M.E.: Optimal scheduling of in-situ analysis for large-scale scientific simulations. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (2015)
Mittal, S., Vetter, J.S.: A survey of software techniques for using non-volatile memories for storage and main memory systems. IEEE Trans. Parallel Distrib Syst (TPDS) (2015)
Nemhauser, G.L., Wolsey, L.A.: Integer and Combinatorial Optimization. Wiley, New York, NY (1988)
Plimpton, S.: Fast parallel algorithms for short-range molecular dynamics. J. Comput. Phys. 117(1), 1–19 (1995)
Acknowledgements
This research has been funded in part and used resources of the Argonne Leadership Computing Facility at Argonne National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under contract no. DE-AC02-06CH11357. This research used resources of the NERSC supported by the Office of Science of the U.S. Department of Energy under contract no. DE-AC02-05CH11231. This work was supported in part by the DOE Office of Science, ASCR, under award numbers 57L38, 57L32, 57L11, 57K50, and 5080500.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Malakar, P., Vishwanath, V., Knight, C., Munson, T., Papka, M.E. (2022). Resource-Aware Optimal Scheduling of In Situ Analysis. In: Childs, H., Bennett, J.C., Garth, C. (eds) In Situ Visualization for Computational Science. Mathematics and Visualization. Springer, Cham. https://doi.org/10.1007/978-3-030-81627-8_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-81627-8_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-81626-1
Online ISBN: 978-3-030-81627-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)