Labeled Network Stack: A Co-designed Stack for Low Tail-Latency and High Concurrency in Datacenter Services

  • Wenli ZhangEmail author
  • Ke Liu
  • Hui Song
  • Lan Yu
  • Mingyu Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11276)


Many Internet, mobile Internet, and IoT services require both low tail-latency and high concurrency in datacenters. The current protocol stack design pays more attention to throughput and average performance, considering little on tail latency and priority. We address this question by proposing a hardware-software co-designed Labeled Network Stack (LNS) for future datacenters. The key innovation is a payload labeling mechanism that distinguishes data packets in a TCP link across the full network stack, including the application, the TCP/IP and the Ethernet layer. This design enables prioritized data packets processing and forwarding along the full data path, to reduce the tail latency of critical requests. We built a prototype datacenter server to evaluate the LNS design against a standard Linux kernel stack and the mTCP research, using IoT kernel benchmark MCC. Experiment results show that the LNS design can provide an order of magnitude improvement on tail latency and concurrency.


Tail latency High concurrent server Priority Label Network stack 



This work is supported by National Key Research and Development Plan of China under Grant No. 2017YFB1001602.


  1. 1.
    Zhang, Y., et al.: Treadmill: attributing the source of tail latency through precise load testing and statistical inference. ISCA-43 (2016)Google Scholar
  2. 2.
    Dean, J., et al.: The tail at scale. Commun. ACM 56(2), 74–80 (2013)CrossRefGoogle Scholar
  3. 3.
    Zats, D., et al.: DeTail: reducing the flow completion time tail in datacenter networks. ACM SIGCOMM Comput. Commun. Rev. 42, 139–150 (2012)CrossRefGoogle Scholar
  4. 4.
    Asanovi’c, K.: FireBox: a Hardware Building Block for 2020 Warehouse-Scale Computers. In: FAST (2014)Google Scholar
  5. 5.
    Xu, Z., et al.: Low-entropy cloud computing systems. ASCIENTIA SINICA Informationis. Scholar
  6. 6.
    Jeong, E., et al.: mTCP: A highly scalable user-level TCP stack for multicore systems. In: Proceedings of NSDI 2014 (2014)Google Scholar
  7. 7.
    Li, J., et al.: Tales of the tail: hardware, OS, and application-level sources of tail latency. In: Symposium on Cloud Computing, pp. 1–14 (2014)Google Scholar
  8. 8.

Copyright information

© IFIP International Federation for Information Processing 2018

Authors and Affiliations

  • Wenli Zhang
    • 1
    Email author
  • Ke Liu
    • 1
  • Hui Song
    • 1
  • Lan Yu
    • 1
  • Mingyu Chen
    • 1
    • 2
  1. 1.Institute of Computing TechnologyChinese Academy of SciencesBeijingPeople’s Republic of China
  2. 2.University of Chinese Academy of SciencesBeijingPeople’s Republic of China

Personalised recommendations