Tackling Exascale Software Challenges in Molecular Dynamics Simulations with GROMACS

  • Szilárd Páll
  • Mark James Abraham
  • Carsten Kutzner
  • Berk Hess
  • Erik Lindahl
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8759)

Abstract

GROMACS is a widely used package for biomolecular simulation, and over the last two decades it has evolved from small-scale efficiency to advanced heterogeneous acceleration and multi-level parallelism targeting some of the largest supercomputers in the world. Here, we describe some of the ways we have been able to realize this through the use of parallelization on all levels, combined with a constant focus on absolute performance. Release 4.6 of GROMACS uses SIMD acceleration on a wide range of architectures, GPU offloading acceleration, and both OpenMP and MPI parallelism within and between nodes, respectively. The recent work on acceleration made it necessary to revisit the fundamental algorithms of molecular simulation, including the concept of neighborsearching, and we discuss the present and future challenges we see for exascale simulation - in particular a very fine-grained task parallelism. We also discuss the software management, code peer review and continuous integration testing required for a project of this complexity.

References

  1. 1.
    Intel Thread Building Blocks. https://www.threadingbuildingblocks.org
  2. 2.
    Abraham, M.J., Gready, J.E.: Optimization of parameters for molecular dynamics simulation using smooth particle-mesh Ewald in GROMACS 4.5. J. Comput. Chem. 32(9), 2031–2040 (2011)CrossRefGoogle Scholar
  3. 3.
    Amdahl, G.M.: Validity of the single processor approach to achieving large scale computing capabilities. In: Proceedings of the Spring Joint Computer Conference, AFIPS 1967 (Spring), pp. 483–485. ACM, New York, NY, USA (1967). http://doi.acm.org/10.1145/1465482.1465560
  4. 4.
    Anderson, J.A., Lorenz, C.D., Travesset, A.: General purpose molecular dynamics simulations fully implemented on graphics processing units. J. Comput. Phys. 227, 5324–5329 (2008)CrossRefGoogle Scholar
  5. 5.
    Andoh, Y., Yoshii, N., Fujimoto, K., Mizutani, K., Kojima, H., Yamada, A., Okazaki, S., Kawaguchi, K., Nagao, H., Iwahashi, K., Mizutani, F., Minami, K., Ichikawa, S.I., Komatsu, H., Ishizuki, S., Takeda, Y., Fukushima, M.: MODYLAS: a highly parallelized general-purpose molecular dynamics simulation program for large-scale systems with long-range forces calculated by Fast Multipole Method (FMM) and highly scalable fine-grained new parallel processing algorithms. J. Chem. Theory Comput. 9(7), 3201–3209 (2013). http://pubs.acs.org/doi/abs/10.1021/ct400203a CrossRefGoogle Scholar
  6. 6.
    Arnold, A., Fahrenberger, F., Holm, C., Lenz, O., Bolten, M., Dachsel, H., Halver, R., Kabadshow, I., Gähler, F., Heber, F., Iseringhausen, J., Hofmann, M., Pippig, M., Potts, D., Sutmann, G.: Comparison of scalable fast methods for long-range interactions. Phys. Rev. E 88, 063308 (2013). http://link.aps.org/doi/10.1103/PhysRevE.88.063308 CrossRefGoogle Scholar
  7. 7.
    Bowers, K.J., Dror, R.O., Shaw, D.E.: Overview of neutral territory methods for the parallel evaluation of pairwise particle interactions. J. Phys. Conf. Ser. 16(1), 300 (2005). http://stacks.iop.org/1742-6596/16/i=1/a=041 CrossRefGoogle Scholar
  8. 8.
    Bowers, K.J., Dror, R.O., Shaw, D.E.: Zonal methods for the parallel execution of range-limited n-body simulations. J. Comput. Phys. 221(1), 303–329 (2007). http://dx.doi.org/10.1016/j.jcp.2006.06.014 CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    Brown, W.M., Wang, P., Plimpton, S.J., Tharrington, A.N.: Implementing molecular dynamics on hybrid high performance computers - short range forces. Comp. Phys. Comm. 182, 898–911 (2011)CrossRefMATHGoogle Scholar
  10. 10.
    Eastman, P., Pande, V.S.: Efficient nonbonded interactions for molecular dynamics on a graphics processing unit. J. Comput. Chem. 31, 1268–1272 (2010)Google Scholar
  11. 11.
    Eleftheriou, M., Moreira, J.E., Fitch, B.G., Germain, R.S.: A volumetric FFT for BlueGene/L. In: Pinkston, T.M., Prasanna, V.K. (eds.) HiPC 2003. LNCS (LNAI), vol. 2913, pp. 194–203. Springer, Heidelberg (2003) CrossRefGoogle Scholar
  12. 12.
    Essmann, U., Perera, L., Berkowitz, M.L., Darden, T., Lee, H., Pedersen, L.G.: A smooth particle mesh Ewald method. J. Chem. Phys. 103(19), 8577–8593 (1995)CrossRefGoogle Scholar
  13. 13.
    Faradjian, A., Elber, R.: Computing time scales from reaction coordinates by milestoning. J. Chem. Phys. 120, 10880–10889 (2004)CrossRefGoogle Scholar
  14. 14.
    Hess, B., Kutzner, C., van der Spoel, D., Lindahl, E.: GROMACS 4: algorithms for highly efficient, load-balanced, and scalable molecular simulation. J. Chem. Theor. Comput. 4(3), 435–447 (2008)CrossRefGoogle Scholar
  15. 15.
    Humphrey, W., Dalke, A., Schulten, K.: VMD: visual molecular dynamics. J. Mol. Graph. 14(1), 33–38 (1996)CrossRefGoogle Scholar
  16. 16.
    Jagode, H.: Fourier transforms for the BlueGene/L communication network. Ph.D. thesis, The University of Edinburgh, Edinburgh, UK (2005)Google Scholar
  17. 17.
    Páll, S., Hess, B.: A flexible algorithm for calculating pair interactions on SIMD architectures. Comput. Phys. Commun. 184(12), 2641–2650 (2013). http://www.sciencedirect.com/science/article/pii/S0010465513001975 CrossRefGoogle Scholar
  18. 18.
    Phillips, J.C., Braun, R., Wang, W., Gumbart, J., Tajkhorshid, E., Villa, E., Chipot, C., Skeel, R.D., Kale, L., Schulten, K.: Scalable molecular dynamics with NAMD. J. Comput. Chem. 26, 1781–1802 (2005)CrossRefGoogle Scholar
  19. 19.
    Pronk, S., Larsson, P., Pouya, I., Bowman, G.R., Haque, I.S., Beauchamp, K., Hess, B., Pande, V.S., Kasson, P.M., Lindahl, E.: Copernicus: A new paradigm for parallel adaptive molecular dynamics. In: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2011, pp. 60:1–60:10. ACM, New York, NY, USA (2011) http://doi.acm.org/10.1145/2063384.2063465
  20. 20.
    Pronk, S., Páll, S., Schulz, R., Larsson, P., Bjelkmar, P., Apostolov, R., Shirts, M.R., Smith, J.C., Kasson, P.M., van der Spoel, D., Hess, B., Lindahl, E.: GROMACS 4.5: a high-throughput and highly parallel open source molecular simulation toolkit. Bioinformatics 29(7), 845–854 (2013). http://bioinformatics.oxfordjournals.org/content/29/7/845.abstract CrossRefGoogle Scholar
  21. 21.
    Reyes, R., Turner, A., Hess, B.: Introducing SHMEM into the GROMACS molecular dynamics application: experience and results. In: Weiland, M., Jackson, A., Johnson, N. (eds.) Proceedings of the 7th International Conference on PGAS Programming Models. The University of Edinburgh, October 2013. http://www.pgas2013.org.uk/sites/default/files/pgas2013proceedings.pdf
  22. 22.
    Schütte, C., Winkelmann, S., Hartmann, C.: Optimal control of molecular dynamics using Markov state models. Math. Program. (Series B) 134, 259–282 (2012)CrossRefMATHGoogle Scholar
  23. 23.
    Shirts, M., Pande, V.S.: Screen savers of the world unite!. Science 290(5498), 1903–1904 (2000). http://www.sciencemag.org/content/290/5498/1903.short CrossRefGoogle Scholar
  24. 24.
    Sugita, Y., Okamoto, Y.: Replica-exchange molecular dynamics method for protein folding. Chem. Phys. Lett. 314, 141–151 (1999)CrossRefGoogle Scholar
  25. 25.
    Verlet, L.: Computer “Experiments” on classical fluids. I. Thermodynamical properties of Lennard-Jones molecules. Phys. Rev. 159, 98–103 (1967). http://link.aps.org/doi/10.1103/PhysRev.159.98 CrossRefGoogle Scholar
  26. 26.
    Wilson, G., Aruliah, D.A., Brown, C.T., Chue Hong, N.P., Davis, M., Guy, R.T., Haddock, S.H.D., Huff, K.D., Mitchell, I.M., Plumbley, M.D., Waugh, B., White, E.P., Wilson, P.: Best practices for scientific computing. PLoS Biol 12(1), e1001745 (2014). http://dx.doi.org/10.1371/journal.pbio.1001745 CrossRefGoogle Scholar
  27. 27.
    Yokota, R., Barba, L.A.: A tuned and scalable fast multipole method as a preeminent algorithm for exascale systems. Int. J. High Perform. Comput. Appl. 26(4), 337–346 (2012). http://hpc.sagepub.com/content/26/4/337.abstract CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Szilárd Páll
    • 1
  • Mark James Abraham
    • 1
  • Carsten Kutzner
    • 2
  • Berk Hess
    • 1
  • Erik Lindahl
    • 1
    • 3
  1. 1.Department of Theoretical Biophysics, Science for Life LaboratoryKTH Royal Institute of TechnologySolnaSweden
  2. 2.Theoretical and Computational Biophysics DepartmentMax Planck Institute for Biophysical ChemistryGöttingenGermany
  3. 3.Department of Biochemistry & Biophysics, Center for Biomembrane ResearchStockholm UniversityStockholmSweden

Personalised recommendations