Skip to main content
SpringerLink
Log in

Speculative Defragmentation – Leading Gigabit Ethernet to True Zero-Copy Communication

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Clusters of Personal Computers (CoPs) offer excellent compute performance at a low price. Workstations with “Gigabit to the Desktop” can give workers access to a new game of multimedia applications. Networking PCs with their modest memory subsystem performance requires either extensive hardware acceleration for protocol processing or alternatively, a highly optimized software system to reach the full Gigabit/sec speeds in applications. So far this could not be achieved, since correctly defragmenting packets of the various communication protocols in hardware remains an extremely complex task and prevented a clean “zero-copy” solution in software. We propose and implement a defragmenting driver based on the same speculation techniques that are common to improve processor performance with instruction level parallelism. With a speculative implementation we are able to eliminate the last copy of a TCP/IP stack even on simple, existing Ethernet NIC hardware. We integrated our network interface driver into the Linux TCP/IP protocol stack and added the well known page remapping and fast buffer strategies to reach an overall zero-copy solution. An evaluation with measurement data indicates three trends: (1) for Gigabit Ethernet the CPU load of communication can be reduced processing significantly, (2) speculation will succeed in most cases, and (3) the performance for burst transfers can be improved by a factor of 1.5–2 over the standard communication software in Linux 2.2. Finally we can suggest simple hardware improvements to increase the speculation success rates based on our implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alteon WebSystems Inc., Jumbo frames whitepaper, Product Webpage: www.alteonwebsystems.com.

  2. D. Bailey, E. Barszcz, J. Barton, D. Browning, R. Carter, L. Dagum, R. Fatoohi, S. Fineberg, P. Frederickson, T. Lasinski, R. Schreiber, H. Simon, V. Venkatakrishnan and S. Weeratunga, NAS Parallel Benchmark, Technical Report RNR–94–007, NASA Ames Research Center, March 1994.

  3. N.J. Boden, R.E. Felderman, A.E. Kulawik, Ch.L. Seitz, J.N. Seizovic and W. Su, Myrinet - A Gigabit per second local area network, IEEE Micro 15(1) (February 1995) 29–36.

    Google Scholar 

  4. J.C. Brustoloni and P. Steenkiste, Effects of buffering semantics on I/O performance, in: Proc. 2nd Symp. on Operating Systems Design and Implementation (OSDI), Seattle, WA, October 1996 (USENIX) pp. 277–291.

  5. J.C. Brustoloni and P. Steenkiste, Copy Emulation in Checksummed, Multiple-Packet Communication, in: Proc. INFOCOM'97, Kobe, Japan, April 1997 (IEEE).

  6. J.B. Carter and W. Zwaenepoel, Optimistic Implementation of Bulk Data Transfer Protocols, in: Proc. of the 1989 Sigmetrics Conference, May 1989, pp. 61–69.

  7. H.K. Jerry Chu, Zero-Copy TCP in Solaris, in: Proc. of the USENIX 1996 Annual Technical Conference, San Diego, CA, USA, January 1996 (The USENIX Association) pp. 253–264.

  8. Dolphin Interconnect Solutions, PCI SCI Cluster Adapter Specification (1996).

  9. P. Druschel and L.L. Peterson, FBufs: A high-bandwidth crossdomain transfer facility, in: Proc. Fourteenth ACM Symp. on Operating System Principles, Asheville, NC, December 1993, pp. 189–202.

  10. C. Dubnicki, E.W. Felten, L. Iftode and K. Li, Software support for virtual memory-mapped communication, in: Proc. 10th IEEE Int. Parallel Proc. Symp., Honolulu, HI, April 1996, pp. 372–381.

  11. D. Dunning, G. Regnier, G. McAlpine, D. Cameron, B. Shubert, F. Berry, A. Merritt, E. Gronke and C. Dodd, The virtual interface architecture, IEEE Micro 18(2) (March- April 1998) 66–76.

    Google Scholar 

  12. P. Geoffray, L. Prylli and B. Tourancheau, BIP-SMP: High performance message passing over a cluster of commodity SMPs, in: Proc. SC99, Portland USA, November 1999 (ACM).

  13. GigaNet Inc., Product Webpage: www.giganet.com.

  14. T. Gross and D. O'Hallaron, iWarp: Anatomy of a Parallel Computing System (MIT Press, 1998).

  15. InterProphet, SiliconTCPTM, Product Webpage: www.interprophet.com.

  16. H.T. Kung, R. Sansom, S. Schlick, P. Steenkiste, M. Arnould, F. Bitz, F. Christianson, E. Cooper, O. Menzilcioglu, D. Ombres and B. Zill, Network-based Multicomputers: An emerging parallel architecture, in: Proc. Supercomputing' 91, Albuquerque, NM, November 1991 (IEEE) pp. 664–673.

  17. Ch. Kurmann and T. Stricker, A comparison of three Gigabit technologies: SCI, Myrinet and SGI/Cray T3D, in: SCI Based Cluster Computing, eds. H. Hellwagner and A. Reinefeld (Springer, Berlin, 1999). An earlier version appeared in Proc. of the SCI Europe'98 Conference, EMM-SEC'98, 28- 30 September 1998, Bordeaux, France.

    Google Scholar 

  18. F.W. Miller, P. Keleher and S.K. Tripathi, General Data Streaming, in: Proc. 19th IEEE Real-Time Systems Symposium, Madrid, December 1998 (IEEE) pp. 232–241.

  19. F. O'Carroll, H. Tezuka, A. Hori and Y. Ishikawa, The design and implementation of zero copy MPI using commodity hardware with a high performance network, in: ICS' 98. Conference Proceedings of the 1998 International Conference on Supercomputing, Melbourne, July 1998 (ACM) pp. 243–250.

  20. S.W. O'Malley, M.B. Abbot, N.C. Hutchinson and L.L. Peterson, A transparent blast facility, Internetworking: Research and Experience 1(2) (December 1990).

  21. V.S. Pai, P. Druschel and W. Zwaenepoel, I/O-Lite: A unified I/O buffering and caching system, in: Proc. of the Third Symposium on Operating Systems Design and Implementation (OSDI' 99) (1999) pp. 15–28.

  22. F. Rauch, Ch. Kurmann and T.M. Stricker, Partition cast - modelling and optimizing the distribution of large data sets on PC clusters, in: Euro-Par 2000 Parallel Processing, 6th International Euro-Par Conference Munich, Munich, 2000, Lecture Notes in Computer Science, Vol. 1900, eds. A. Bode, T. Ludwig, W. Karl and R. Wismüller (Springer). Also available as Technical Report 343, Department of Computer Science, ETH Zürich, http://www.inf.ethz.ch/.

  23. R. Seifert, Gigabit Ethernet: Technology and Applications for High-Speed LANs (Addison-Wesley, 1998). ISBN: 0201185539.

  24. M. Taufer, Personal communication, ETH Zurich, September 2000.

  25. T. von Eicken, A. Basu, V. Buch and W. Vogels, U-Net: A user-level network interface for parallel and distributed computing, in: Proc. of 15th Symposium on Operating Systems Principles (SOSP-15), Cooper Mountain, CO, USA, December 1995 (ACM).

Download references

Author information

Authors and Affiliations

  1. Laboratory for Computer Systems, Swiss Federal Institute of Technology (ETH), CH-8092, Zürich, Switzerland

    Christian Kurmann, Felix Rauch & Thomas M. Stricker

Authors
  1. Christian Kurmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Felix Rauch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas M. Stricker
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kurmann, C., Rauch, F. & Stricker, T.M. Speculative Defragmentation – Leading Gigabit Ethernet to True Zero-Copy Communication. Cluster Computing 4, 7–18 (2001). https://doi.org/10.1023/A:1011456024871

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1011456024871

Search

Navigation