Abstract
An increasing number of supercomputers adopt a heterogeneous architecture, consisting of both general purpose CPUs and specialized accelerators. Such design is beneficial for scalability and power, but on the other hand, heterogeneity brings new challenges in communication systems to connect heterogeneous components and provide support for programming. The communication system of the Dawning 6000 connects two kinds of heterogeneous processors, Loongson and AMD, and adopts a three layer architecture with an intranode layer between heterogeneous components. To efficiently connect heterogeneous components, the system forms a global address space and provides a mechanism for message transmission via an in-node global store; and employing Infiniband network, provides an OS-bypassing virtualization method to share an Infiniband card between nodes. To facilitate programming on heterogeneous processors, it supports unified parallel C (UPC), with a modified complier based on global address space. Also, a special collective network is implemented for collective operations. Results obtained from a prototype system prove these features to be both feasible and efficient.
Similar content being viewed by others
References
Sun N, Li K, Chen M. HPP: an architecture for high performance and utility computing. Chinese Journal of Computers, 2008, 31(9): 1503–1508
Carrera E V, Rao S, Iftode L, Bianchini R. User-level communication in cluster-based servers. In: Proceedings of 8th International Symposium on High-Performance Computer Architecture. 2002, 275–286
Ries R. Communication patterns [message-passing patterns]. In: Proceedings of 20th International Parallel and Distributed Processing Symposium, 2006
Consortium UPC. UPC Ianguage Specifications v1.2. Lawrence Berkeley National Lab Tech Report LBNL-59208, 2005
Buyya R, Cortes T, Jin H. Single system image. International Journal of High Performance Computing Applications, 2001, 15(2): 124–135
Gara A, Blumrich M A, Chen D, et al. Overview of the blue gene/l system architecture. IBM Journal of Research and Development, 2005, 49(2): 195–212
Wright C. Roadrunner Tutorial. Hybrid Programming: DaCS and ALF Examples. Los Alamos National LaboratoryLA-UR-08-2817. http://www.lanl.gov/orgs/hpc/roadrunner/pdfs/Roadrunner-tutorial-session-5-web1.pdf, 2008
Stokes J. Clearing up the confusion over Intel’s Larrabee. Ars Technica. http://arstechnica.com/articles/paedia/hardware/clearingup-the-confusion-over-intels-larrabee.ars. Retrieved Jun. 1st, 2007
Zhang P Y, Meng D, Huo Z G. Research of collectives optimization on modern multicore clusters. Chinese Journal of Computers, 2010, 33(2): 317–325
Barham P, Dragovic B, Fraser K, et al. Xen and the art of virtualization. In: Proceedings of the 19th ACM symposium on Operating systems principles. 2003, 164–177
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Li, Q., Li, B., Huo, Z. et al. Design and implementation of communication system of the Dawning 6000 supercomputer. Front. Comput. Sci. China 4, 466–474 (2010). https://doi.org/10.1007/s11704-010-0114-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11704-010-0114-3