The Journal of Supercomputing

, Volume 61, Issue 1, pp 141–165 | Cite as

Design of scalable Java message-passing communications over InfiniBand

  • Roberto R. Expósito
  • Guillermo L. TaboadaEmail author
  • Juan Touriño
  • Ramón Doallo


This paper presents ibvdev a scalable and efficient low-level Java message-passing communication device over InfiniBand. The continuous increase in the number of cores per processor underscores the need for efficient communication support for parallel solutions. Moreover, current system deployments are aggregating a significant number of cores through advanced network technologies, such as InfiniBand, increasing the complexity of communication protocols, especially when dealing with hybrid shared/distributed memory architectures such as clusters. Here, Java represents an attractive choice for the development of communication middleware for these systems, as it provides built-in networking and multithreading support. As the gap between Java and compiled languages performance has been narrowing for the last years, Java is an emerging option for High Performance Computing (HPC).

The developed communication middleware ibvdev increases Java applications performance on clusters of multicore processors interconnected via InfiniBand through: (1) providing Java with direct access to InfiniBand using InfiniBand Verbs API, somewhat restricted so far to MPI libraries; (2) implementing an efficient and scalable communication protocol which obtains start-up latencies and bandwidths similar to MPI performance results; and (3) allowing its integration in any Java parallel and distributed application. In fact, it has been successfully integrated in the Java messaging library MPJ Express.

The experimental evaluation of this middleware on an InfiniBand cluster of multicore processors has shown significant point-to-point performance benefits, up to 85% start-up latency reduction and twice the bandwidth compared to previous Java middleware on InfiniBand. Additionally, the impact of ibvdev on message-passing collective operations is significant, achieving up to one order of magnitude performance increases compared to previous Java solutions, especially when combined with multithreading. Finally, the efficiency of this middleware, which is even competitive with MPI in terms of performance, increments the scalability of communications intensive Java HPC applications.


Message-Passing in Java (MPJ) InfiniBand Multicore architectures High performance computing Remote Direct Memory Access (RDMA) Performance evaluation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Taboada GL, Ramos S, Expósito RR, Touriño J, Doallo R (2011, in press) Java in the high performance computing arena: research, practice and experience. Sci Comput Program. doi: 10.1016/j.scico.2011.06.002 Google Scholar
  2. 2.
    Blount B, Chatterjee S (1999) An evaluation of Java for numerical computing. Sci Program 7(2):97–110 Google Scholar
  3. 3.
    Shafi A, Carpenter B, Baker M (2009) Nested parallelism for multi-core HPC systems using Java. J Parallel Distrib Comput 69(6):532–545 CrossRefGoogle Scholar
  4. 4.
    InfiniBand Trade Association (2004) Infiniband architecture specification volume 1, release 1.2.1. [Last visited: April 2011]
  5. 5.
    Baker M, Carpenter B, Shafi A (2005) A pluggable architecture for high-performance Java messaging. IEEE Distrib Syst Online 6(10):1–4 CrossRefGoogle Scholar
  6. 6.
    IETF Draft. IP over IB. [Last visited: April 2011]
  7. 7.
    Hongwei Z, Wan H, Jizhong H, Jin H, Lisheng Z (2007) A performance study of Java communication stacks over InfiniBand and Gigabit Ethernet. In: Proc 4th IFIP intl conf network and parallel computing (NPC’07), Dalian, China, pp 602–607 Google Scholar
  8. 8.
    Veldema R, Hofman RFH, Bhoedjang R, Bal HE (2003) Run-time optimizations for a Java DSM implementation. Concurr Comput 15(3–5):299–316 zbMATHCrossRefGoogle Scholar
  9. 9.
    Righi RDR, Navaux POA, Cera MC, Pasin M (2005) Asynchronous communication in Java over InfiniBand and DECK. In: Proc 17th intl symp on computer architecture and high performance computing (SBAC-PAD’05), Rio de Janeiro, Brazil, pp 176–183 CrossRefGoogle Scholar
  10. 10.
    Taboada GL, Touriño J, Doallo R (2008) Java Fast Sockets: enabling high-speed Java communications on high performance clusters. Comput Commun 31(17):4049–4059 CrossRefGoogle Scholar
  11. 11.
    Wan H, Hongwei Z, Jin H, Jizhong H, Lisheng Z (2007) Jdib: Java applications interface to unshackle the communication capabilities of InfiniBand networks. In: Proc 4th IFIP intl conf network and parallel computing (NPC’07), Dalian, China, pp 596–601 Google Scholar
  12. 12.
    Wan H, Jizhong H, Jin H, Lisheng Z, Yao L (2008) Enabling RDMA capability of InfiniBand network for Java applications. In: Proc 4th IFIP intl conf on networking, architecture, and storage (NAS’08), Chongqing, China, pp 187–188 Google Scholar
  13. 13.
    Yao L, Jizhong H, Jinjun G, Xubin H (2009) uStream: a user-level stream protocol over InfiniBand. In: Proc 15th intl conf on parallel and distributed systems (ICPADS’09), Shenzhen, China, pp 65–71 Google Scholar
  14. 14.
    Dunning D, Regnier G, McAlpine G, Cameron D, Shubert B, Berry F, Merritt AM, Gronke E, Dodd C (1998) The Virtual Interface Architecture. IEEE MICRO 18(2):66–76 CrossRefGoogle Scholar
  15. 15.
    Compaq, Intel and Microsoft Corporations (1997) The Virtual Interface Architecture specification, version 1.0 Google Scholar
  16. 16.
    OpenFabrics Alliance Website. [Last visited: April 2011]
  17. 17.
    Carpenter B, Fox G, Ko S-H, Lim S mpiJava 1.2: API specification. [Last visited: April 2011]
  18. 18.
    The mpiJava project. [Last visited: April 2011]
  19. 19.
    Bornemann M, van Nieuwpoort RV, Kielmann T (2005) MPJ/Ibis: a flexible and efficient message passing platform for Java. In: Proc. 12th European PVM/MPI users’ group meeting (EuroPVM/MPI’05), Sorrento, Italy, pp 217–224 Google Scholar
  20. 20.
    Taboada GL, Touriño J, Doallo R (2011) F-MPJ: scalable Java message-passing communications on parallel systems. J Supercomput. doi: 10.1007/s11227-009-0270-0 Google Scholar
  21. 21.
    Java Grande Forum. http// [Last visited: April 2011]
  22. 22.
    van Nieuwpoort RV, Maassen J, Wrzesinska G, Hofman R, Jacobs C, Kielmann T, Bal HE (2005) Ibis: a flexible and efficient Java-based grid programming environment. Concurr Comput 17(7–8):1079–1107 CrossRefGoogle Scholar
  23. 23.
    MX User’s Guide. [Last visited: April 2011]
  24. 24.
    Open Source High Performance MPI Library. [Last visited: April 2011]
  25. 25.
    MPI over InfiniBand, 10GigE/iWARP and RDMAoE. [Last visited: April 2011]
  26. 26.
    Tezuka H, O’Carroll F, Hori A, Ishikaw Y (1998) Pin-down cache: a virtual memory management technique for zero-copy communication. In: Proc 12th intl parallel processing symposium/9th symposium on parallel and distributed processing (IPPS/SPDP’98), Orlando, FL, USA, pp 308–314 Google Scholar
  27. 27.
    Baker M, Carpenter B, Shafi A (2007) A buffering layer to support derived types and proprietary networks for Java HPC. Scalable Comput Pract Experience 8(4):343–358 Google Scholar
  28. 28.
    Taboada GL, Touriño J, Doallo R, Shafi A, Baker M, Carpenter B (2011, in press) Device level communication libraries for high-performance computing in Java. Concurr Comput. doi: 110.1002/cpe.1777 Google Scholar
  29. 29.
    MPJ Express Website. http// [Last visited: April 2011]
  30. 30.
    Taboada GL, Touriño J, Doallo R (2003) Performance analysis of Java message-passing libraries on Fast Ethernet, Myrinet and SCI clusters. In: Proc 5th IEEE intl conf on cluster computing (CLUSTER’03), Hong Kong, China, pp 118–126 CrossRefGoogle Scholar
  31. 31.
    Mallón DA, Taboada GL, Touriño J, Doallo R (2009) NPB-MPJ: NAS parallel benchmarks implementation for message-passing in Java. In: Proc 17th Euromicro intl conf on parallel, distributed, and network-based processing (PDP’09), Weimar, Germany, pp 181–190 Google Scholar
  32. 32.
    Baker M, Carpenter B, Shafi A (2006) MPJ Express meets Gadget: towards a Java code for cosmological simulations. In: 13th European PVM/MPI users’ group meeting (EuroPVM/MPI’06), Bonn, Germany, pp 358–365 Google Scholar
  33. 33.
    Springel V (2005) The cosmological simulation code GADGET-2. Mon Not R Astron Soc 364(4):1105–1134 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Roberto R. Expósito
    • 1
  • Guillermo L. Taboada
    • 1
    Email author
  • Juan Touriño
    • 1
  • Ramón Doallo
    • 1
  1. 1.Computer Architecture Group, Dept. of Electronics and SystemsUniversity of A CoruñaA CoruñaSpain

Personalised recommendations