On Reducing I/O Overheads in Large-Scale Invariant Subspace Projections

  • Hasan Metin Aktulga
  • Chao Yang
  • Ümit V. Çatalyürek
  • Pieter Maris
  • James P. Vary
  • Esmond G. Ng
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7155)

Abstract

Obtaining highly accurate predictions on properties of light atomic nuclei using the Configuration Interaction (CI) method requires computing the lowest eigenvalues and associated eigenvectors of a large many-body nuclear Hamiltonian, H. One particular approach, the J-scheme, requires the projection of the H matrix into an invariant subspace. Since the matrices can be very large, enormous computing power is needed while significant stresses are put on the memory and I/O sub-systems. By exploiting the inherent localities in the problem and making use of the MPI one-sided communication routines backed by RDMA operations available in the new parallel architectures, we show that it is possible to reduce the I/O overheads drastically for large problems. This is demonstrated in the subspace projection phase of J-scheme calculations on 6Li nucleus, where our new implementation based on one-sided MPI communications outperforms the previous I/O based implementation by almost a factor of 10.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Vary, J.P., Maris, P., Ng, E., Yang, C., Sosonkina, M.: Ab initio nuclear structure: The Large sparse matrix eigenvalue problem. J. Phys. Conf. Ser. 180, 012083 (2009)CrossRefGoogle Scholar
  2. 2.
    Maris, P., Shirokov, A.M., Vary, J.P.: Ab initio nuclear structure simulations: The Speculative F-14 nucleus. Phys. Rev. C 81, 021301 (2010)CrossRefGoogle Scholar
  3. 3.
    Maris, P., Vary, J.P., Navratil, P., Ormand, W.E., Nam, H., Dean, D.J.: Origin of the anomalous long lifetime of 14C. Phys. Rev. Lett. 106, 202502 (2011)CrossRefGoogle Scholar
  4. 4.
    Shirokov, A.M., Mazur, A.I., Zaytsev, S.A., Vary, J.P., Weber, T.A.: Nucleon nucleon interaction in the J matrix inverse scattering approach and few nucleon systems. Phys. Rev. C 70, 044005 (2004)CrossRefGoogle Scholar
  5. 5.
    Shirokov, A.M., Mazur, A.I., Vary, J.P., Mazur, E.A.: Inverse scattering J-matrix approach to nucleon-nucleus scattering and the shell model. Phys. Rev. C 79, 014610 (2009)CrossRefGoogle Scholar
  6. 6.
    Aktulga, H.M., Yang, C., Ng, E., Maris, P., Vary, J.P.: Large-scale Parallel null space calculation for nuclear configuration interaction. In: Proc. of HPCS 2011, Istanbul, Turkey, July 4 - 8 (2011)Google Scholar
  7. 7.
    Gropp, W., Huss-Lederman, S., Lumsdaine, A., Lusk, E., Nitzberg, B., Saphir, W., Snir, M.: MPI – The Complete Reference. The MPI-2 Extensions, vol. 2. MIT Press, Cambridge (1998)Google Scholar
  8. 8.
    Gropp, W., Thakur, R.: An Evaluation of Implementation Options for MPI One-Sided Communication. In: Di Martino, B., Kranzlmüller, D., Dongarra, J. (eds.) EuroPVM/MPI 2005. LNCS, vol. 3666, pp. 415–424. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  9. 9.
    Çatalyürek, Ü.V., Kaya, K., Uçar, B.: Integrated Data Placement and Task Assignment for Scientific Workflows in Clouds. In: Proc. of HPDC, The Fourth International Workshop on Data Intensive Distributed Computing (DIDC) (June 2011)Google Scholar
  10. 10.
    Nieplocha, J., Tipparaju, V., Krishnan, M., Panda, D.: High performance remote memory access comunications: The ARMCI approach. International Journal of High Performance Computing and Applications 20(2), 233–253 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Hasan Metin Aktulga
    • 1
  • Chao Yang
    • 1
  • Ümit V. Çatalyürek
    • 2
  • Pieter Maris
    • 3
  • James P. Vary
    • 3
  • Esmond G. Ng
    • 1
  1. 1.Lawrence Berkeley National LaboratoryBerkeleyUSA
  2. 2.The Ohio State UniversityColumbusUSA
  3. 3.Iowa State UniversityAmesUSA

Personalised recommendations