Journal of Network and Systems Management

, Volume 26, Issue 2, pp 314–338 | Cite as

Throughput and Latency of Virtual Switching with Open vSwitch: A Quantitative Analysis

  • Paul Emmerich
  • Daniel Raumer
  • Sebastian Gallenmüller
  • Florian Wohlfart
  • Georg Carle


Virtual switches, like Open vSwitch, have emerged as an important part of today’s data centers. They connect interfaces of virtual machines and provide an uplink to the physical network via network interface cards. We discuss usage scenarios for virtual switches involving physical and virtual network interfaces. We present extensive black-box tests to quantify the throughput and latency of software switches with emphasis on the market leader, Open vSwitch. Finally, we explain the observed effects using white-box measurements.


Network measurement Cloud Performance evaluation Performance characterization MoonGen 



This research has been supported by the DFG (German Research Foundation) as part of the MEMPHIS Project (CA 595/5-2) and in the framework of the CELTIC EUREKA Project SENDATE-PLANETS (Project ID C2015/3-1), partly funded by the German BMBF (Project ID 16KIS0460K). The authors alone are responsible for the content of the paper.


  1. 1.
    Angrisani, L., Ventre, G., Peluso, L., Tedesco, A.: Measurement of processing and queuing delays introduced by an open-source router in a single-hop network. IEEE Trans. Instrum. Meas. 55(4), 1065–1076 (2006)CrossRefGoogle Scholar
  2. 2.
    Beifuß, A., Raumer, D., Emmerich, P., Runge, T.M., Wohlfart, F., Wolfinger, B.E., Carle, G.: A study of networking software induced latency. In: 2nd International Conference on Networked Systems 2015. Cottbus, Germany (2015)Google Scholar
  3. 3.
    Bianco, A., Birke, R., Giraudo, L., Palacin, M.: Openflow switching: data plane performance. In: International Conference on Communications (ICC). IEEE (2010)Google Scholar
  4. 4.
    Bolla, R., Bruschi, R.: Linux software router: data plane optimization and performance evaluation. J. Netw. 2(3), 6–17 (2007)Google Scholar
  5. 5.
    Bradner, S., McQuaid, J.: Benchmarking methodology for network interconnect devices. RFC 2544 (Informational) (1999)Google Scholar
  6. 6.
    Cardigliano, A., Deri, L., Gasparakis, J., Fusco, F.: vPFRING: towards wirespeed network monitoring using virtual machines. In: ACM Internet Measurement Conference (2011)Google Scholar
  7. 7.
    Deri, L.: nCap: Wire-speed packet capture and transmission. In: IEEE Workshop on End-to-End Monitoring Techniques and Services, pp. 47–55 (2005)Google Scholar
  8. 8.
    Dobrescu, M., Argyraki, K., Ratnasamy, S.: Toward predictable performance in software packet-processing platforms. In: USENIX Conference on Networked Systems Design and Implementation (NSDI) (2012)Google Scholar
  9. 9.
    Dobrescu, M., Egi, N., Argyraki, K., Chun, B., Fall, K., Iannaccone, G., Knies, A., Manesh, M., Ratnasamy, S.: RouteBricks: exploiting parallelism to scale software routers. In: 22nd ACM Symposium on Operating Systems Principles (SOSP) (2009)Google Scholar
  10. 10.
    DPDK Project: DPDK 16.11 Release Notes. (2016). Last visited 2016-03-27
  11. 11.
    Emmerich, P., Gallenmüller, S., Raumer, D., Wohlfart, F., Carle, G.: MoonGen: A scriptable high-speed packet generator. In: 15th ACM SIGCOMM Conference on Internet Measurement (IMC’15) (2015)Google Scholar
  12. 12.
    Emmerich, P., Raumer, D., Wohlfart, F., Carle, G.: A study of network stack latency for game servers. In: 13th Annual Workshop on Network and Systems Support for Games (NetGames’14). Nagoya, Japan (2014)Google Scholar
  13. 13.
    Emmerich, P., Raumer, D., Wohlfart, F., Carle, G.: Performance characteristics of virtual switching. In: 2014 IEEE 3rd International Conference on Cloud Networking (CloudNet’14). Luxembourg (2014)Google Scholar
  14. 14.
    ETSI: Network Functions Virtualisation (NFV); Architectural Framework, V1.1.1 (2013)Google Scholar
  15. 15.
    Han, S., Jang, K., Panda, A., Palkar, S., Han, D., Ratnasamy, S.: Softnic: A software NIC to augment hardware. Technical Report UCB/EECS-2015-155, EECS Department, University of California, Berkeley (2015)Google Scholar
  16. 16.
    He, Z., Liang, G.: Research and evaluation of network virtualization in cloud computing environment. In: Networking and Distributed Computing (ICNDC), pp. 40–44. IEEE (2012)Google Scholar
  17. 17.
    Huggahalli, R., Iyer, R., Tetrick, S.: Direct cache access for high bandwidth network I/O. In: Proceedings of the 32nd Annual International Symposium on Computer Architecture, pp. 50–59 (2005)Google Scholar
  18. 18.
    Hwang, J., Ramakrishnan, K.K., Wood, T.: Netvm: High performance and flexible networking using virtualization on commodity platforms. In: 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14), pp. 445–458. USENIX Association, Seattle (2014)Google Scholar
  19. 19.
    Impressive Packet Processing Performance Enables Greater Workload Consolidation (2013)Google Scholar
  20. 20.
  21. 21.
    Intel DPDK vSwitch. Last visited 2016-03-27
  22. 22.
    Intel DPDK: Data Plane Development Kit. Last visited 2016-03-27
  23. 23.
    Intel I/O Acceleration Technology. Last visited 2016-03-27
  24. 24.
    Jarschel, M., Oechsner, S., Schlosser, D., Pries, R., Goll, S., Tran-Gia, P.: Modeling and performance evaluation of an OpenFlow architecture. In: Proceedings of the 23rd International Teletraffic Congress. ITCP (2011)Google Scholar
  25. 25.
    Kang, N., Liu, Z., Rexford, J., Walker, D.: Optimizing the “one big switch” abstraction in software-defined networks. In: Proceedings of the Ninth ACM Conference on Emerging Networking Experiments and Technologies, CoNEXT ’13, pp. 13–24. ACM, New York (2013). doi: 10.1145/2535372.2535373
  26. 26.
    Kevin, Traynor: OVS, DPDK and Software Dataplane Acceleration. (2016). Last visited 2016-03-27
  27. 27.
    Kohler, E., Morris, R., Chen, B., Jannotti, J., Kaashoek, M.F.: The click modular router. ACM Trans. Comput. Syst. (TOCS) 18(3), 263–297 (2000). doi: 10.1145/354871.354874 CrossRefGoogle Scholar
  28. 28.
    Larsen, S., Sarangam, P., Huggahalli, R., Kulkarni, S.: Architectural breakdown of end-to-end latency in a TCP/IP network. Int. J. Parallel Program. 37(6), 556–571 (2009)CrossRefzbMATHGoogle Scholar
  29. 29.
    Martins, J., Ahmed, M., Raiciu, C., Olteanu, V., Honda, M., Bifulco, R., Huici, F.: ClickOS and the art of network function virtualization. In: 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14), pp. 459–473. USENIX Association, Seattle (2014)Google Scholar
  30. 30.
    Meyer, T., Wohlfart, F., Raumer, D., Wolfinger, B., Carle, G.: Validated model-based prediction of multi-core software router performance. Praxis der Informationsverarbeitung und Kommunikation (PIK) (2014)Google Scholar
  31. 31.
    Munch, B.: Hype Cycle for Networking and Communications. Report, Gartner (2013)Google Scholar
  32. 32.
    Niu, Z., Xu, H., Tian, Y., Liu, L., Wang, P., Li, Z.: Benchmarking NFV software dataplanes. (2016). arXiv:1605.05843
  33. 33.
    Open vSwitch. Last visited 2016-03-27
  34. 34.
    OpenNebula. Last visited 2016-03-27
  35. 35.
    OpenStack. Last visited 2016-03-27
  36. 36.
    OpenStack: Networking Guide: Deployment Scenarios. (2015). Last visited 2016-03-27
  37. 37.
    Panda, A., Han, S., Jang, K., Walls, M., Ratnasamy, S., Shenker, S.: Netbricks: Taking the V out of NFV. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pp. 203–216. USENIX Association, GA (2016)Google Scholar
  38. 38.
    Pettit, J., Gross, J., Pfaff, B., Casado, M., Crosby, S.: Virtual switching in an era of advanced edges. In: 2nd Workshop on Data Center Converged and Virtual Ethernet Switching (DC-CAVES) (2011)Google Scholar
  39. 39.
    Pfaff, B., Pettit, J., Koponen, T., Amidon, K., Casado, M., Shenker, S.: Extending networking into the virtualization layer. In: Proceedings of workshop on Hot Topics in Networks (HotNets-VIII) (2009)Google Scholar
  40. 40.
    Pfaff, B., Pettit, J., Koponen, T., Jackson, E., Zhou, A., Rajahalme, J., Gross, J., Wang, A., Stringer, J., Shelar, P., Amidon, K., Casado, M.: The design and implementation of open vswitch. In: 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15). USENIX Association (2015)Google Scholar
  41. 41.
    Pongracz, G., Molnar, L., Kis, Z.L.: Removing roadblocks from SDN: OpenFlow software switch performance on Intel DPDK. In: Second European Workshop on Software Defined Networks (EWSDN’13) pp. 62–67 (2013)Google Scholar
  42. 42.
    Ram, K.K., Cox, A.L., Chadha, M., Rixner, S.: Hyper-Switch: A scalable software virtual switching architecture. In: presented as part of the 2013 USENIX annual technical conference (USENIX ATC 13), pp. 13–24. USENIX, San Jose (2013)Google Scholar
  43. 43.
    Rizzo, L., Carbone, M., Catalli, G.: Transparent acceleration of software packet forwarding using Netmap. In: INFOCOM, pp. 2471–2479. IEEE (2012)Google Scholar
  44. 44.
    Rizzo, L.: Netmap: a novel framework for fast packet I/O. In: USENIX Annual Technical Conference (2012)Google Scholar
  45. 45.
    Rizzo, L., Lettieri, G.: VALE, a switched ethernet for virtual machines. In: Proceedings of the 8th International Conference on Emerging Networking Experiments and Technologies (CoNEXT '12), pp. 61–72. ACM, New York (2012)Google Scholar
  46. 46.
    Rotsos, C., Sarrar, N., Uhlig, S., Sherwood, R., Moore, A.W.: Oflops: an open framework for OpenFlow switch evaluation. In: Passive and Active Measurement, pp. 85–95. Springer (2012)Google Scholar
  47. 47.
    Russell, R., Tsirkin, M.S., Huck, C., Moll, P. (eds.): Virtual I/O Device (VIRTIO) Version 1.0. OASIS Committee Specification 04 (2016)Google Scholar
  48. 48.
    Salim, J.H., Olsson, R., Kuznetsov, A.: Beyond softnet. In: Proceedings of the 5th annual Linux Showcase & Conference, vol. 5, pp. 18–18 (2001)Google Scholar
  49. 49.
    Tedesco, A., Ventre, G., Angrisani, L., Peluso, L.: Measurement of processing and queuing delays introduced by a software router in a single-hop network. In: IEEE Instrumentation and Measurement Technology Conference, pp. 1797–1802 (2005)Google Scholar
  50. 50.
    Thomas, Monjalon: dropping librte_ivshmem. (2016). Mailing list discussion
  51. 51.
    Virtual Machine Device Queues: Technical White Paper (2008)Google Scholar
  52. 52.
    Wang, G., Ng, T.E.: The impact of virtualization on network performance of amazon ec2 data center. In: INFOCOM, pp. 1–9. IEEE (2010)Google Scholar
  53. 53.
    Whiteaker, J., Schneider, F., Teixeira, R.: Explaining packet delays under virtualization. ACM SIGCOMM Comput. Commun. Rev. 41(1), 38–44 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Paul Emmerich
    • 1
  • Daniel Raumer
    • 1
  • Sebastian Gallenmüller
    • 1
  • Florian Wohlfart
    • 1
  • Georg Carle
    • 1
  1. 1.Department of Informatics, Chair of Network Architectures and ServicesTechnical University of MunichGarchingGermany

Personalised recommendations