Skip to main content
SpringerLink
Log in

OpenMP/MPI Hybrid vs. Flat MPI on the Earth Simulator: Parallel Iterative Solvers for Finite Element Method

  • Conference paper
High Performance Computing (ISHPC 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2858))

Included in the following conference series:

Abstract.

An efficient parallel iterative method for finite element method has been developed for symmetric multiprocessor (SMP) cluster architectures with vector processors such as the Earth Simulator. The method is based on a three-level hybrid parallel programming model, including message passing for inter-SMP node communication, loop directives by OpenMP for intra-SMP node parallelization and vectorization for each processing element (PE). Simple 3D linear elastic problems with more than 2.2 x 109 DOF have been solved using 3 x 3 block ICCG(0) method with additive Schwarz domain decomposition and PDJDS/CM-RCM reordering on 176 nodes of the Earth Simulator, achieving performance of 3.80 TFLOPS.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Barrett, R., Berry, M., Chan, T.F., Donato, J., Dongarra, J.J., Eijkhout, V., Pozo, R., Romine, C., van der Vorst, H.: Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. SIAM, Philadelphia (1994)

    Google Scholar 

  2. Cappelo, F., Etiemble, D.: MPI versus MPI+OpenMP on the IBM SP for the NAS Benchmarks, SC 2000 Technical Paper, Dallas, Texas (2000)

    Google Scholar 

  3. Djomehri, M.J., Jin, H.H.: Hybrid MPI+OpenMP Programming of an Overset CFD Solver and Performance Investigations. NASA/NAS Technical Report (NASA Ames Research Center), NAS-02-002 16 (2002)

    Google Scholar 

  4. Doi, S., Washio, T.: Using Multicolor Ordering with Many Colors to Strike a Better Balance between Parallelism and Convergence. In: RIKEN Symposium on Linear Algebra and its Applications, The Institute of Physical and Chemical Research, pp. 19–26 (1999)

    Google Scholar 

  5. Dongara, J.J., Duff, I.S., Sorensen, D.C., van der Vorst, H.A.: Solving Linear Systems on Vector and Shared Memory Computers. SIAM, Philadelphia (1990)

    Google Scholar 

  6. Falgout, R., Jones, J.: Multigrid on Massively Parallel Architectures. In: Sixth European Multigrid Conference, Ghent, Belgium, September 27-30 (1999)

    Google Scholar 

  7. Garatani, K., Nakamura, H., Okuda, H., Yagawa, G.: GeoFEM: High Performance Parallel FEM for Solid Earth. In: Sloot, P.M.A., Hoekstra, A.G., Bubak, M., Hertzberger, B. (eds.) HPCN-Europe 1999. LNCS, vol. 1593, pp. 133–140. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  8. Kerbyson, D.J., Hoisie, A., Wasserman, H.: A Comparison Between the Earth Simulator and Alpha Server Systems using Predictive Application Performance Models, LA-UR-02-5222, Los Alamos National Laboratory, USA (2002)

    Google Scholar 

  9. Nakajima, K., Okuda, H.: Parallel Iterative Solvers with Localized ILU Preconditioning for Unstructured Grids on Workstation Clusters. International Journal for Computational Fluid Dynamics 12, 315–322 (1999)

    Article  MATH  Google Scholar 

  10. Nakajima, K., Okuda, H.: Parallel Iterative Solvers for Unstructured Grids using an OpenMP/MPI Hybrid Programming Model for the GeoFEM Platform on SMP Cluster Architectures. In: Zima, H.P., Joe, K., Sato, M., Seo, Y., Shimasaki, M. (eds.) ISHPC 2002. LNCS, vol. 2327, pp. 437–448. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  11. Nakajima, K.: Parallel Iterative Solvers of GeoFEM with Selective Blocking Preconditioning for Nonlinear Contact Problems on the Earth Simulator, RIST/Tokyo GeoFEM Report 2003-006 (2003), http://geofem.tokyo.rist.or.jp/report_en/2003_006.html/

  12. Oliker, L., Li, X., Husbands, P., Biswas, R.: Effects of Ordering Strategies and Programming Paradigms on Sparse Matrix Computations. SIAM Review 44(3), 373–393 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  13. Rabenseifner, R.: Communication Bandwidth of Parallel Programming Models on Hybrid Architectures. In: Zima, H.P., Joe, K., Sato, M., Seo, Y., Shimasaki, M. (eds.) ISHPC 2002. LNCS, vol. 2327, pp. 437–448. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  14. Saad, Y.: Iterative Methods for Sparse Linear Systems. PWS Publishing Company (1996)

    Google Scholar 

  15. Smith, B., Bjφrstad, P., Gropp, W.: Domain Decomposition, Parallel Multilevel Methods for Elliptic Partial Differential Equations. Cambridge Press, Cambridge (1996)

    MATH  Google Scholar 

  16. Washio, T., Maruyama, K., Osoda, T., Shimizu, F., Doi, S.: Efficient implementations of block sparse matrix operations on shared memory vector machines. In: SNA 2000: The Fourth International Conference on Supercomputing in Nuclear Applications (2000)

    Google Scholar 

  17. Accelerated Strategic Computing Initiative (ASCI) Web Site: http://www.llnl.gov/asci/

  18. Earth Simulator Center Web Site: http://www.es.jamstec.go.jp/

  19. GeoFEM Web Site: http://geofem.tokyo.rist.or.jp/

  20. MPI Web Site: http://www.mpi.org/

  21. OpenMP Web Site: http://www.openmp.org/

Download references

Author information

Authors and Affiliations

  1. Research Organization for Information Science and Technology (RIST), 2-2-54 Naka-Meguro, Meguro-ku, 153-0061, Tokyo, Japan

    Kengo Nakajima

Authors
  1. Kengo Nakajima
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of California (UCI), 3019 Donald Bren Hall, 92697-3435, Irvine, CA, USA

    Alex Veidenbaum

  2. Department of Information and Computer Science, Faculty of Science, Nara women’s University, Kitauoyanishi-machi, Nara-city, 630-8506, Nara, Japan

    Kazuki Joe

  3. Keio University, Hiyoshi, Kohoku, Yokohama, 223–8522, Kanagawa, Japan

    Hideharu Amano

  4. Tokyo University of Technology, 1404-1 Katakura, Hachioji, 192-0982, Tokyo, Japan

    Hideo Aiso

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Nakajima, K. (2003). OpenMP/MPI Hybrid vs. Flat MPI on the Earth Simulator: Parallel Iterative Solvers for Finite Element Method. In: Veidenbaum, A., Joe, K., Amano, H., Aiso, H. (eds) High Performance Computing. ISHPC 2003. Lecture Notes in Computer Science, vol 2858. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39707-6_44

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-39707-6_44

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-20359-9

  • Online ISBN: 978-3-540-39707-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation