Reproducible and User-Controlled Software Environments in HPC with Guix
Support teams of high-performance computing (HPC) systems often find themselves between a rock and a hard place: on one hand, they understandably administrate these large systems in a conservative way, but on the other hand, they try to satisfy their users by deploying up-to-date tool chains as well as libraries and scientific software. HPC system users often have no guarantee that they will be able to reproduce results at a later point in time, even on the same system—software may have been upgraded, removed, or recompiled under their feet, and they have little hope of being able to reproduce the same software environment elsewhere. We present GNU Guix and the functional package management paradigm and show how it can improve reproducibility and sharing among researchers with representative use cases.
KeywordsDirected Acyclic Graph Software Environment Support Team Environment Module Tool Chain
We would like to thank Florent Pruvost, Emmanuel Agullo, and Andreas Enge at Inria and Eric Bavier at Cray Inc. for insightful discussions and comments on an earlier draft. We are grateful to the Guix contributors who keep improving the system.
- 2.Courtès, L.: Functional package management with guix. In: European Lisp Symposium (2013)Google Scholar
- 3.Dolstra, E., de Jonge, M., Visser, E.: Nix: a safe and policy-free system for software deployment. In: Proceedings of the 18th Large Installation System Administration Conference (LISA 2004), USENIX, pp. 79–92 (2004)Google Scholar
- 4.Furlani, J.L.: Providing a flexible user environment. In: Proceedings of the 5th Large Installation System Administration (LISA V), pp. 141–152 (1991)Google Scholar
- 5.Gamblin, T.: Spack Web Site (2015). http://scalability-llnl.github.io/spack/
- 7.Geimer, M., Hoste, K., McLay, R.: Modern scientific software management using easybuild and Lmod. In: Proceedings of the 1st Workshop on HPC User Support Tools (HUST 2014), pp. 41–51. IEEE Press (2014)Google Scholar
- 9.Heydon, A., Levin, R., Yu, Y.: Caching function calls using precise dependencies. In: Proceedings of the ACM SIGPLAN 2000 Conference on Programming Language Design and Implementation, pp. 311–320. ACM (2000)Google Scholar
- 10.Jeanvoine, E., Sarzyniec, L., Nussbaum, L.: Kadeploy3: efficient and scalable operating system provisioning. USENIX; login 38(1), 38–44 (2013)Google Scholar
- 12.Stanisic, Luka, Legrand, Arnaud: Effective reproducible research with org-mode and git. In: Lopes, L., et al. (eds.) Euro-Par 2014, Part I. LNCS, vol. 8805, pp. 475–486. Springer, Heidelberg (2014) Google Scholar