On a NIC’s Operating System, Schedulers and High-Performance Networking Applications

  • Yaron Weinsberg
  • Tal Anker
  • Danny Dolev
  • Scott Kirkpatrick
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4208)


Two critical issues impact the overall performance of Linux clusters based on Intel servers: inter-process communication latency and data throughput. Underlying both of these performance challenges is the inefficient use of computational power and server CPU cycles to process the network protocols. Today’s modern high-end Network Interface Cards (NICs) are equipped with an onboard CPU. In most cases, these CPU’s are only used by the vendor and are operated by a proprietary OS, which makes them inaccessible to the HPC application developer. In this paper we present a design and implementation of a framework for building high-performance networking applications. The framework consists of an embedded NIC Operating System with a specialized scheduler. The main challenge in developing such a scheduler is the lack of a preemption mechanism in most high-end NICs. Our scheduler provides finer-grained schedules than the alternatives. We have implemented several network applications, and were able to increase their throughput while decreasing the host’s CPU utilization.


Schedule Algorithm High Performance Computing Periodic Task Network Interface Card High Performance Computing Cluster 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Currid, A.: TCP offload to the rescue. Queue 2(3), 58–65 (2004)CrossRefGoogle Scholar
  2. 2.
    Fiuczynski, M.E., Martin, R.P., Owa, T., Bershad, B.N.: Spine: a safe programmable and integrated network environment. In: EW 8: Proceedings of the 8th ACM SIGOPS European workshop on Support for composing distributed applications, pp. 7–12. ACM Press, New York (1998)CrossRefGoogle Scholar
  3. 3.
    Pratt, I., Fraser, K.: Arsenic: A user-accessible gigabit ethernet interface. In: INFOCOM, pp. 67–76 (2001)Google Scholar
  4. 4.
    Shivam, P., Wyckoff, P., Panda, D.: Emp: zero-copy os-bypass nic-driven gigabit ethernet message passing. In: Supercomputing 2001: Proceedings of the 2001 ACM/IEEE conference on Supercomputing (CDROM), p. 57. ACM Press, New York (2001)CrossRefGoogle Scholar
  5. 5.
    Wilson, P.R., Johnstone, M.S., Neely, M., Boles, D.: Dynamic storage allocation: A survey and critical review. In: Proc. Int. Workshop on Memory Management, Kinross Scotland (UK) (1995)Google Scholar
  6. 6.
    Cheng, S.C., Stankovic, J.A., Ramamritham, K.: Scheduling algorithms for hard real-time systems: a brief survey, pp. 150–173 (1989)Google Scholar
  7. 7.
    Liu, C.L., Layland, J.W.: Scheduling algorithms for multiprogramming in a hard-real-time environment. Journal of the ACM 20(1), 46–61 (1973)MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Shmueli, E., Feitelson, D.G.: Backfilling with lookahead to optimize the performance of parallel job scheduling. In: Job Scheduling Strategies for Parallel Processing (2003)Google Scholar
  9. 9.
    Weinsberg, Y., Pavlov, E., Amir, Y., Gat, G., Wulff, S.: Putting it on the NIC: A case study on application offloading to a Network Interface Card (NIC). In: IEEE CCNC (2006)Google Scholar
  10. 10.
    (Super-fast Transport Over Replicated Machines (Storm)), Available at site: http://www.cs.huji.ac.il/~wyaron.
  11. 11.
    Lamport, L.: Time, clocks, and the ordering of events in a distributed system. Commun. ACM 21(7), 558–565 (1978)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yaron Weinsberg
    • 1
  • Tal Anker
    • 2
  • Danny Dolev
    • 1
  • Scott Kirkpatrick
    • 1
  1. 1.The Hebrew University Of Jerusalem 
  2. 2.RADLAN – A Marvell CompanyIsrael

Personalised recommendations