Where Has My Time Gone?
Time matters. In a networked world, we would like mobile devices to provide a crisp user experience and applications to instantaneously return results. Unfortunately, application performance does not depend solely on processing time, but also on a number of different components that are commonly counted in the overall system latency. Latency is more than just a nuisance to the user, poorly accounted-for, it degrades application performance. In fields such as high frequency trading, as well as in many data centers, latency translates easily to financial losses. Research to date has focused on specific contributions to latency: from improving latency within the network to latency control on the application level. This paper takes an holistic approach to latency, and aims to provide a break-down of end-to-end latency from the application level to the wire. Using a set of crafted experiments, we explore the many contributors to latency. We assert that more attention should be paid to the latency within the host, and show that there is no silver bullet to solve the end-to-end latency challenge in data centers. We believe that a better understanding of the key elements influencing data center latency can trigger a more focused research, improving the user’s quality of experience.
KeywordsVirtual Machine Round Trip Time User Space High Frequency Trading Spine Switch
We would like to thank the many people who contributed to this paper. We would like to thank Salvator Galea and Robert N Watson, who contributed to early work on this paper. This work has received funding from the EPSRC grant EP/K034723/1, Leverhulme Trust Early Career Fellowship ECF-2016-289, European Union’s Horizon 2020 research and innovation programme 2014-2018 under the SSICLOPS (grant agreement No. 644866), ENDEAVOUR (grant agreement No. 644960) and EU FP7 Marie Curie ITN METRICS (grant agreement No. 607728).
Dataset. A reproduction environment of the experiments, and the experimental results, are both available at http://www.cl.cam.ac.uk/research/srg/netos/projects/latency/pam2017/ and https://doi.org/10.17863/CAM.7418.
- 1.Barroso, L.A.: Landheld Computing. In: IEEE International Solid State Circuits Conference (ISSCC) (2014). KeynoteGoogle Scholar
- 2.Cheshire, S.: It’s the latency, stupid. http://www.stuartcheshire.org/rants/Latency.html. Accessed July 2016
- 3.Guo, C., et al.: RDMA over commodity ethernet at scale. In: SIGCOMM 2016 (2016)Google Scholar
- 4.Hemminger, S.: NetEm - Network Emulator. http://man7.org/linux/man-pages/man8/tc-netem.8.html. Accessed July 2016
- 5.Kalia, A., et al.: Design guidelines for high performance RDMA systems. In: USENIX ATC, vol. 16, pp. 437–450 (2016)Google Scholar
- 6.Mayer, M.: What Google knows. In: Web 2.0 Summit (2006)Google Scholar
- 8.Nussbaum, L., Richard, O.: A comparative study of network link emulators. In: SpringSim 2009, pp. 85:1–85:8 (2009)Google Scholar
- 9.Oracle: Oracle VM VirtualBox. https://www.virtualbox.org/. Accessed Oct 2016
- 10.Paoloni, G.: How to benchmark code execution times on Intel IA-32 and IA-64 instruction set architectures. Technical report 324264–001, Intel (2010)Google Scholar
- 12.Rumble, S.M., et al.: It’s time for low latency. In: HotOS 2013, p. 11. USENIX Association (2011)Google Scholar
- 13.SAP: Big data and smart trading (2012)Google Scholar
- 15.Tolly Enterprises: Mellanox spectrum vs. broadcom StrataXGS Tomahawk 25GbE & 100GbE performance evaluation - evaluating consistency & predictability. Technical report 216112 (2016)Google Scholar