Measuring Web Quality of Experience in Cellular Networks

  • Alemnew Sheferaw AsreseEmail author
  • Ermias Andargie Walelgne
  • Vaibhav Bajpai
  • Andra Lutu
  • Özgü Alay
  • Jörg Ott
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11419)


Measuring and understanding the end-user browsing Quality of Experience (QoE) is crucial to Mobile Network Operators (MNOs) to retain their customers and increase revenue. MNOs often use traffic traces to detect the bottlenecks and study their end-users experience. Recent studies show that Above The Fold (ATF) time better approximates the user browsing QoE compared to traditional metrics such as Page Load Time (PLT). This work focuses on developing a methodology to measure the web browsing QoE over operational Mobile Broadband (MBB) networks. We implemented a web performance measurement tool WebLAR (it stands for Web Latency And Rendering) that measures web Quality of Service (QoS) such as TCP connect time, and Time To First Byte (TTFB) and web QoE metrics including PLT and ATF time. We deployed WebLAR on 128 MONROE (a European-wide mobile measurement platform) nodes, and conducted two weeks long (May and July 2018) web measurement campaign towards eight websites from six operational MBB networks. The result shows that, in the median case, the TCP connect time and TTFB in Long Term Evolution (LTE) networks are, respectively, 160% and 30% longer than fixed-line networks. The DNS lookup time and TCP connect time of the websites varies significantly across MNOs. Most of the websites do not show a significant difference in PLT and ATF time across operators. However, Yahoo shows longer ATF time in Norwegian operators than that of the Swedish operators. Moreover, user mobility has a small impact on the ATF time of the websites. Furthermore, the website design should be taken into consideration when approximating the ATF time.


  1. 1.
    ImageMagick: tool to create, edit, compose, or convert bitmap images. Accessed 12 Oct 2018
  2. 2.
    WebPageTest. Accessed 09 Jan 2019
  3. 3.
  4. 4.
    Ahmad, S., Haamid, A.L., Qazi, Z.A., Zhou, Z., Benson, T., Qazi, I.A.: A view from the other side: understanding mobile phone characteristics in the developing world. In: ACM IMC (2016).
  5. 5.
    Akamai White Paper: Measuring Real Customer Experiences over Mobile Networks. Accessed 12 Oct 2017
  6. 6.
    Alay, Ö., et al.: Experience: an open platform for experimentation with commercial mobile broadband networks. In: ACM MobiCom (2017).
  7. 7.
    Asrese, A.S.: WebLAR: A Web Performance Measurement Tool (2019).
  8. 8.
    Asrese, A.S., Eravuchira, S.J., Bajpai, V., Sarolahti, P., Ott, J.: Measuring web latency and rendering performance: method, tools & longitudinal dataset. IEEE Trans. Netw. Serv. Manag. (2019, to appear)Google Scholar
  9. 9.
    Asrese, A.S., Sarolahti, P., Boye, M., Ott, J.: WePR: a tool for automated web performance measurement. In: IEEE Globecom Workshops (2016).
  10. 10.
    Asrese, A.S., Walelgne, E., Bajpai, V., Lutu, A., Alay, Ö., Ott, J.: Measuring web quality of experience in cellular networks (dataset) (2019).
  11. 11.
    Bajpai, V., Kühlewind, M., Ott, J., Schönwälder, J., Sperotto, A., Trammell, B.: Challenges with reproducibility. In: SIGCOMM Reproducibility Workshop, pp. 1–4 (2017).
  12. 12.
    Bajpai, V., Schönwälder, J.: A survey on internet performance measurement platforms and related standardization efforts. IEEE Commun. Surv. Tutor. 17(3), 1313–1341 (2015). Scholar
  13. 13.
    Balachandran, A., et al.: Modeling web quality-of-experience on cellular networks. In: ACM MobiCom (2014).
  14. 14.
    Barakovic, S., Skorin-Kapov, L.: Multidimensional modelling of quality of experience for mobile web browsing. Comput. Hum. Behav. 50, 314–332 (2015). Scholar
  15. 15.
    Brutlag, J., Abrams, Z., Meenan, P.: Above the Fold Time: Measuring Web Page Performance Visually.
  16. 16.
    Cao, Y., Nejati, J., Wajahat, M., Balasubramanian, A., Gandhi, A.: Deconstructing the energy consumption of the mobile page load. Proc. ACM Meas. Anal. Comput. Syst. 1(1), 6 (2017). Scholar
  17. 17.
    Casas, P., Seufert, M., Wamser, F., Gardlo, B., Sackl, A., Schatz, R.: Next to you: monitoring quality of experience in cellular networks from the end-devices. IEEE Trans. Netw. Serv. Manag. 13(2), 181–196 (2016). Scholar
  18. 18.
    Cecchet, E., Sims, R., He, X., Shenoy, P.J.: mBenchLab: measuring QoE of Web applications using mobile devices. In: International Symposium on Quality of Service, IWQoS (2013).
  19. 19.
    Chen, Q.A., et al.: QoE doctor: diagnosing mobile app QoE with automated UI control and cross-layer analysis. In: ACM Internet Measurement Conference (2014).
  20. 20.
    Dasari, M., Vargas, S., Bhattacharya, A., Balasubramanian, A., Das, S.R., Ferdman, M.: Impact of device performance on mobile internet QoE. In: Internet Measurement Conference, pp. 1–7 (2018).
  21. 21.
    DoubleClick: The Need for Mobile Speed: Better User Experiences, Greater Publisher Revenue. Accessed 26 Feb 2018
  22. 22.
    Eravuchira, S.J., Bajpai, V., Schönwälder, J., Crawford, S.: Measuring web similarity from dual-stacked hosts. In: Conference on Network and Service Management, pp. 181–187 (2016).
  23. 23.
    FFmpeg: FFmpeg: a complete, cross-platform solution to record, convert and stream audio and video. Accessed 12 Oct 2018
  24. 24.
    Google: Lighthouse: an open-source, automated tool for improving the quality of web pages. Accessed 09 Jan 2019
  25. 25.
    da Hora, D.N., Asrese, A.S., Christophides, V., Teixeira, R., Rossi, D.: Narrowing the gap between QoS metrics and Web QoE using above-the-fold metrics. In: Beverly, R., Smaragdakis, G., Feldmann, A. (eds.) PAM 2018. LNCS, vol. 10771, pp. 31–43. Springer, Cham (2018). Scholar
  26. 26.
    Hosek, J., et al.: Mobile web QoE study for smartphones. In: IEEE GLOBECOM Workshop (2013).
  27. 27.
    Hoßfeld, T., Metzger, F., Rossi, D.: Speed index: relating the industrial standard for user perceived web performance to web QoE. In: IEEE International Conference on Quality of Multimedia Experience (2018).
  28. 28.
    Li, L., et al.: A longitudinal measurement study of TCP performance and behavior in 3G/4G networks over high speed rails. IEEE/ACM Trans. Netw. 25(4), 2195–2208 (2017). Scholar
  29. 29.
    Mandalari, A.M., et al.: Experience: implications of roaming in Europe. In: MOBICOM, pp. 179–189 (2018).
  30. 30.
  31. 31.
    Nejati, J., Balasubramanian, A.: An in-depth study of mobile browser performance. In: Conference on World Wide Web, pp. 1305–1315 (2016).
  32. 32.
    OpenSignal: Meteor. Accessed 12 May 2017
  33. 33.
    Sackl, A., Casas, P., Schatz, R., Janowski, L., Irmer, R.: Quantifying the impact of network bandwidth fluctuations and outages on Web QoE. In: IEEE International Workshop on Quality of Multimedia Experience (2015).
  34. 34.
    Sonntag, S., Manner, J., Schulte, L.: Netradar - measuring the wireless world. In: IEEE International Symposium and Workshops on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks (2013).
  35. 35.
    Varvello, M., Blackburn, J., Naylor, D., Papagiannaki, K.: EYEORG: a platform for crowdsourcing web quality of experience measurements. In: ACM Conference on emerging Networking EXperiments and Technologies (2016).
  36. 36.
    Walelgne, E.A., Kim, S., Bajpai, V., Neumeier, S., Manner, J., Ott, J.: Factors affecting performance of web flows in cellular networks. In: IFIP Networking (2018)Google Scholar
  37. 37.
    Walelgne, E.A., Manner, J., Bajpai, V., Ott, J.: Analyzing throughput and stability in cellular networks. In: IEEE/IFIP Network Operations and Management Symposium, pp. 1–9 (2018).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Alemnew Sheferaw Asrese
    • 1
    Email author
  • Ermias Andargie Walelgne
    • 1
  • Vaibhav Bajpai
    • 2
  • Andra Lutu
    • 4
  • Özgü Alay
    • 3
  • Jörg Ott
    • 2
  1. 1.Aalto UniversityEspooFinland
  2. 2.Technische Universität MünchenMunichGermany
  3. 3.Simula MetropolitanOsloNorway
  4. 4.Telefonica ResearchBarcelonaSpain

Personalised recommendations