Advertisement

Journal of Computer Science and Technology

, Volume 30, Issue 2, pp 259–272 | Cite as

High Performance Interconnect Network for Tianhe System

  • Xiang-Ke LiaoEmail author
  • Zheng-Bin Pang
  • Ke-Fei Wang
  • Yu-Tong Lu
  • Min Xie
  • Jun Xia
  • De-Zun Dong
  • Guang Suo
Regular Paper

Abstract

In this paper, we present the Tianhe-2 interconnect network and message passing services. We describe the architecture of the router and network interface chips, and highlight a set of hardware and software features effectively supporting high performance communications, ranging over remote direct memory access, collective optimization, hardware enable reliable end-to-end communication, user-level message passing services, etc. Measured hardware performance results are also presented.

Keywords

Tianhe-2 supercomputer interconnect network router architecture network interface architecture user-level message passing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Liao X, Xiao L, Yang C et al. Milkyway-2 supercomputer system and application. Frontiers of Computer Science, 2014, 8(3): 345–356.CrossRefMathSciNetGoogle Scholar
  2. [2]
    Pritchard H, Gorodetsky I, Buntinas D. A uGNI-based MPICH2 Nemesis network module for the cray XE. In Proc. the 18th European MPI Users' Group Conference on Recent Advances in the Message Passing Interface, Sept. 2011, pp.110-119.Google Scholar
  3. [3]
    Xie M, Lu Y, Liu L et al. Implementation and evaluation of network interface and message passing services for TianHe-1A supercomputer. In Proc. the 19th IEEE Annual Symposium on High Performance Interconnects, Aug. 2011, pp.78-86.Google Scholar
  4. [4]
    Kim J, Dally W J, Towles B, Gupta A K. Microarchitecture of a high radix router. In Proc. the 32nd Annual International Symposium on Computer Architecture, June 2005, pp.420-431.Google Scholar
  5. [5]
    Schoinas I, Hill M D. Address translation mechanisms in network interfaces. In Proc. the 4th International Symposium on High-Performance Computer Architecture, Feb. 1998, pp.219-230.Google Scholar
  6. [6]
    Chun B N, Mainwaring A, Culler D E. Virtual network transport protocols for Myrinet. IEEE Micro, 1998, 18(1): 53–63.CrossRefGoogle Scholar
  7. [7]
    Araki S, Bilas A, Dubnicki C et al. User-space communication: A quantitative study. In Proc. ACM/IEEE Conference on Supercomputing, Nov. 1998.Google Scholar
  8. [8]
    Bhoedjang R A F, Ruhl T, Bal H E. User-level network interface protocols. Computer, 1998, 31(11): 53–60.CrossRefGoogle Scholar
  9. [9]
    Graham R L, Poole S, Shamis P et al. Overlapping computation and communication: Barrier algorithms and ConnectX-2 CORE-Direct capabilities. In Proc. IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum, April 2010.Google Scholar
  10. [10]
    Kandalla K, Subramoni H, Vienne J et al. Designing nonblocking broadcast with collective offload on InfiniBand clusters: A case study with HPL. In Proc. the 19th IEEE Annual Symposium on High Performance Interconnects, Aug. 2011, pp.27-34.Google Scholar
  11. [11]
    Buntinas D, Goglin B, Goodell D et al. Cache-efficient, intranode, large-message MPI communication with MPICH2-Nemesis. In Proc. International Conference on Parallel Processing, Sept. 2009, pp.462-469.Google Scholar
  12. [12]
    Lauria M, Pakin S, Chien A. Efficient layering for high speed communication: Fast messages 2.x. In Proc. the 7th International Symposium on High Performance Distributed Computing, July 1998, pp.10-20.Google Scholar
  13. [13]
    Liu J, Panda D K. Implementing efficient and scalable flow control schemes in MPI over InfiniBand. In Proc. the 18th International Parallel and Distributed Processing Symposium, April 2004.Google Scholar
  14. [14]
    Vetter J S, Mueller F. Communication characteristics of large-scale scientific applications for contemporary cluster architectures. Journal of Parallel and Distributed Computing, 2003, 63(9): 853–865.CrossRefzbMATHGoogle Scholar
  15. [15]
    Tezuka H, O’Carroll F, Hori A et al. Pin-down cache: A virtual memory management technique for zero-copy communication. In Proc. Symposium on Parallel and Distributed Processing, Mar. 30-Apr. 3, 1998, pp.308-314.Google Scholar
  16. [16]
    IBM Blue Gene team. The IBM Blue Gene project. IBM J. Res. Dev., 2013, 57(1/2): 0:1–0:6.CrossRefGoogle Scholar
  17. [17]
    Chen D, Eisley N A, Heidelberger P et al. The IBM Blue Gene/Q interconnection fabric. IEEE Micro, 2012, 32(1): 32–43.CrossRefzbMATHGoogle Scholar
  18. [18]
    Ajima Y, Inoue T, Hiramota S et al. The Tofu interconnect. IEEE Micro, 2012, 32(1): 21–31.CrossRefGoogle Scholar
  19. [19]
    Alverson R, Roweth D, Kaplan L. The Gemini system interconnect. In Proc. the 18th IEEE Symposium on High Performance Interconnects, Aug. 2010, pp.83-87.Google Scholar
  20. [20]
    Schroeder B, Gibson G. Understanding failures in petascale computers. J. Physics: Conference Series, 2007, 78: 012022.Google Scholar
  21. [21]
    Graham R L, Poole S, Shamis P et al. ConnectX-2 Infini-Band management queues: First investigation of the new support for network offloaded collective operations. In Proc. the 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, May 2010, pp.53-62.Google Scholar
  22. [22]
    Subramoni H, Kandalla K, Sur S et al. Design and evaluation of generalized collective communication primitives with overlap using connectX-2 offload engine. In Proc. the 18th IEEE Annual Symposium on High Performance Interconnects, Aug. 2010, pp.40-49.Google Scholar
  23. [23]
    Arimilli B, Arimilli R, Chung V et al. The PERCS highperformance interconnect. In Proc. the 18th IEEE Symposium on High Performance Interconnects, Aug. 2010, pp.75-82.Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Xiang-Ke Liao
    • 1
    • 2
    Email author
  • Zheng-Bin Pang
    • 1
    • 2
  • Ke-Fei Wang
    • 1
  • Yu-Tong Lu
    • 1
    • 3
  • Min Xie
    • 1
    • 3
  • Jun Xia
    • 1
  • De-Zun Dong
    • 1
    • 2
  • Guang Suo
    • 1
    • 3
  1. 1.College of ComputerNational University of Defense TechnologyChangshaChina
  2. 2.Science and Technology on Parallel and Distributed Processing LaboratoryNational University of Defense TechnologyChangshaChina
  3. 3.State Key Laboratory of High Performance ComputingNational University of Defense TechnologyChangshaChina

Personalised recommendations