Advertisement

Zero-Copy Receive for Virtualized Network Devices

  • Kalman Meth
  • Joel NiderEmail author
  • Mike Rapoport
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 881)

Abstract

When receiving network traffic on guest VMs (virtual machines) running on the KVM hypervisor, data is copied from hypervisor buffers to guest buffers. This copy incurs CPU and latency overhead. It seems desirable to avoid this copy, if possible, and to use guest buffers directly to receive network data. A number of challenges must be overcome in order to accomplish this zero-copy receive, and some additional overhead is incurred in setting it up. We present the technical details on a zero-copy receive design and implementation and discuss the resulting performance degradation. In our implementation, the savings in CPU from avoiding the copy is overshadowed by the extra handling required for the smaller and often underutilized buffers .

Keywords

Zero-copy Virtual networking KVM 

Notes

Acknowledgment

This work is part of the OPERA project, which has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 688386 (OPERA) and 645402 (Mikelangelo).

References

  1. 1.
    KVM: Kernel Virtual Machine (2007). https://www.linux-kvm.org/page/Main_Page
  2. 2.
    Russell, R.: Virtio: towards a de-facto standard for virtual I/O devices. SIGOPS Oper. Syst. Rev. 42, 95–103 (2008)CrossRefGoogle Scholar
  3. 3.
    Intel: Frequently Asked Questions for SR-IOV on Intel® Ethernet Server Adapters. Intel Inc., 8 March 2018. https://www.intel.com/content/www/us/en/support/articles/000005722/network-and-i-o/Ethernet-products.html
  4. 4.
    Robison, C.B.: Configure SR-IOV Network Virtual Functions in Linux* KVM*. Intel Inc., June 2017. https://software.intel.com/en-us/articles/configure-sr-iov-network-virtual-functions-in-linux-kvm
  5. 5.
    KernelNewbies.org: LinuxVirt: MacVTap, KernelNewbies.org, 30 December 2017. https://virt.kernelnewbies.org/MacVTap
  6. 6.
    Ronciak, J.A., Brandeburg, J., Venkatesan, G.: Page-flip technology for use within the linux networking stack. In: Ottawa Linux Symposium, Ottawa (2004)Google Scholar
  7. 7.
    Miller, D.: David Miller’s Google + Public Blog, 18 October 2016. https://plus.google.com/+DavidMiller/posts/EUDiGoXD6Xv
  8. 8.
    The Linux Foundation: Data Plane Development Kit. Intel Inc. (2016). http://dpdk.org/
  9. 9.
    Rizzo, L.: Netmap: a novel framework for fast packet I/O. In: Proceedings of the 2012 USENIX Annual Technical Conference, Boston (2012)Google Scholar
  10. 10.
    Rottenkolber, M.: Snabb Snabbco, 18 September 2013. https://github.com/snabbco/snabb
  11. 11.
    The Linux Foundation: vmxnet3-usermap. VMWare Inc. http://dpdk.org/doc/vmxnet3-usermap
  12. 12.
    Tsirkin, M.S.: tun zerocopy support. Red Hat Inc., 20 July 2012. https://lwn.net/Articles/507716/
  13. 13.
    de Bruijn, W.: socket sendmsg MSG_ZEROCOPY. Google Inc., 18 June 2017. https://lwn.net/Articles/725825/
  14. 14.
    Töpel, B.: Introducing AF_XDP support. Intel Inc., 31 Jan 2018. https://lwn.net/Articles/745934/
  15. 15.
    Beaupré, A.: New approaches to network fast paths. LWN.net, 13 April 2017. https://lwn.net/Articles/719850/
  16. 16.
    Gorman, M.: Fast noirq bulk page allocator. LKML.ORG, 4 January 2017. https://lkml.org/lkml/2017/1/4/251
  17. 17.
  18. 18.
    Duyck, A.: macvlan/macvtap: Add support for L2 forwarding offloads with macvtap. Intel Inc., 17 October 2017. https://patchwork.ozlabs.org/patch/826704/

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.IBM Research Labs - HaifaHaifaIsrael

Personalised recommendations