Labeled Network Stack: A Co-designed Stack for Low Tail-Latency and High Concurrency in Datacenter Services
Many Internet, mobile Internet, and IoT services require both low tail-latency and high concurrency in datacenters. The current protocol stack design pays more attention to throughput and average performance, considering little on tail latency and priority. We address this question by proposing a hardware-software co-designed Labeled Network Stack (LNS) for future datacenters. The key innovation is a payload labeling mechanism that distinguishes data packets in a TCP link across the full network stack, including the application, the TCP/IP and the Ethernet layer. This design enables prioritized data packets processing and forwarding along the full data path, to reduce the tail latency of critical requests. We built a prototype datacenter server to evaluate the LNS design against a standard Linux kernel stack and the mTCP research, using IoT kernel benchmark MCC. Experiment results show that the LNS design can provide an order of magnitude improvement on tail latency and concurrency.
KeywordsTail latency High concurrent server Priority Label Network stack
This work is supported by National Key Research and Development Plan of China under Grant No. 2017YFB1001602.
- 1.Zhang, Y., et al.: Treadmill: attributing the source of tail latency through precise load testing and statistical inference. ISCA-43 (2016)Google Scholar
- 4.Asanovi’c, K.: FireBox: a Hardware Building Block for 2020 Warehouse-Scale Computers. In: FAST (2014)Google Scholar
- 6.Jeong, E., et al.: mTCP: A highly scalable user-level TCP stack for multicore systems. In: Proceedings of NSDI 2014 (2014)Google Scholar
- 7.Li, J., et al.: Tales of the tail: hardware, OS, and application-level sources of tail latency. In: Symposium on Cloud Computing, pp. 1–14 (2014)Google Scholar