Advertisement

Measuring Web Quality of Experience in Cellular Networks

  • Alemnew Sheferaw AsreseEmail author
  • Ermias Andargie Walelgne
  • Vaibhav Bajpai
  • Andra Lutu
  • Özgü Alay
  • Jörg Ott
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11419)

Abstract

Measuring and understanding the end-user browsing Quality of Experience (QoE) is crucial to Mobile Network Operators (MNOs) to retain their customers and increase revenue. MNOs often use traffic traces to detect the bottlenecks and study their end-users experience. Recent studies show that Above The Fold (ATF) time better approximates the user browsing QoE compared to traditional metrics such as Page Load Time (PLT). This work focuses on developing a methodology to measure the web browsing QoE over operational Mobile Broadband (MBB) networks. We implemented a web performance measurement tool WebLAR (it stands for Web Latency And Rendering) that measures web Quality of Service (QoS) such as TCP connect time, and Time To First Byte (TTFB) and web QoE metrics including PLT and ATF time. We deployed WebLAR on 128 MONROE (a European-wide mobile measurement platform) nodes, and conducted two weeks long (May and July 2018) web measurement campaign towards eight websites from six operational MBB networks. The result shows that, in the median case, the TCP connect time and TTFB in Long Term Evolution (LTE) networks are, respectively, 160% and 30% longer than fixed-line networks. The DNS lookup time and TCP connect time of the websites varies significantly across MNOs. Most of the websites do not show a significant difference in PLT and ATF time across operators. However, Yahoo shows longer ATF time in Norwegian operators than that of the Swedish operators. Moreover, user mobility has a small impact on the ATF time of the websites. Furthermore, the website design should be taken into consideration when approximating the ATF time.

References

  1. 1.
    ImageMagick: tool to create, edit, compose, or convert bitmap images. https://imagemagick.org. Accessed 12 Oct 2018
  2. 2.
    WebPageTest. https://www.webpagetest.org. Accessed 09 Jan 2019
  3. 3.
  4. 4.
    Ahmad, S., Haamid, A.L., Qazi, Z.A., Zhou, Z., Benson, T., Qazi, I.A.: A view from the other side: understanding mobile phone characteristics in the developing world. In: ACM IMC (2016). http://dl.acm.org/citation.cfm?id=2987470
  5. 5.
    Akamai White Paper: Measuring Real Customer Experiences over Mobile Networks. https://www.akamai.com/jp/ja/multimedia/documents/white-paper/measuring-real-customer-experiences-over-mobile-networks-report.pdf. Accessed 12 Oct 2017
  6. 6.
    Alay, Ö., et al.: Experience: an open platform for experimentation with commercial mobile broadband networks. In: ACM MobiCom (2017).  https://doi.org/10.1145/3117811.3117812
  7. 7.
    Asrese, A.S.: WebLAR: A Web Performance Measurement Tool (2019). https://github.com/alemnew/weblar
  8. 8.
    Asrese, A.S., Eravuchira, S.J., Bajpai, V., Sarolahti, P., Ott, J.: Measuring web latency and rendering performance: method, tools & longitudinal dataset. IEEE Trans. Netw. Serv. Manag. (2019, to appear)Google Scholar
  9. 9.
    Asrese, A.S., Sarolahti, P., Boye, M., Ott, J.: WePR: a tool for automated web performance measurement. In: IEEE Globecom Workshops (2016).  https://doi.org/10.1109/GLOCOMW.2016.7849082
  10. 10.
    Asrese, A.S., Walelgne, E., Bajpai, V., Lutu, A., Alay, Ö., Ott, J.: Measuring web quality of experience in cellular networks (dataset) (2019). https://github.com/alemnew/2019-pam-weblar
  11. 11.
    Bajpai, V., Kühlewind, M., Ott, J., Schönwälder, J., Sperotto, A., Trammell, B.: Challenges with reproducibility. In: SIGCOMM Reproducibility Workshop, pp. 1–4 (2017).  https://doi.org/10.1145/3097766.3097767
  12. 12.
    Bajpai, V., Schönwälder, J.: A survey on internet performance measurement platforms and related standardization efforts. IEEE Commun. Surv. Tutor. 17(3), 1313–1341 (2015).  https://doi.org/10.1109/COMST.2015.2418435CrossRefGoogle Scholar
  13. 13.
    Balachandran, A., et al.: Modeling web quality-of-experience on cellular networks. In: ACM MobiCom (2014).  https://doi.org/10.1145/2639108.2639137
  14. 14.
    Barakovic, S., Skorin-Kapov, L.: Multidimensional modelling of quality of experience for mobile web browsing. Comput. Hum. Behav. 50, 314–332 (2015).  https://doi.org/10.1016/j.chb.2015.03.071CrossRefGoogle Scholar
  15. 15.
    Brutlag, J., Abrams, Z., Meenan, P.: Above the Fold Time: Measuring Web Page Performance Visually. https://conferences.oreilly.com/velocity/velocity-mar2011/public/schedule/detail/18692
  16. 16.
    Cao, Y., Nejati, J., Wajahat, M., Balasubramanian, A., Gandhi, A.: Deconstructing the energy consumption of the mobile page load. Proc. ACM Meas. Anal. Comput. Syst. 1(1), 6 (2017).  https://doi.org/10.1145/3084443CrossRefGoogle Scholar
  17. 17.
    Casas, P., Seufert, M., Wamser, F., Gardlo, B., Sackl, A., Schatz, R.: Next to you: monitoring quality of experience in cellular networks from the end-devices. IEEE Trans. Netw. Serv. Manag. 13(2), 181–196 (2016).  https://doi.org/10.1109/TNSM.2016.2537645CrossRefGoogle Scholar
  18. 18.
    Cecchet, E., Sims, R., He, X., Shenoy, P.J.: mBenchLab: measuring QoE of Web applications using mobile devices. In: International Symposium on Quality of Service, IWQoS (2013).  https://doi.org/10.1109/IWQoS.2013.6550259
  19. 19.
    Chen, Q.A., et al.: QoE doctor: diagnosing mobile app QoE with automated UI control and cross-layer analysis. In: ACM Internet Measurement Conference (2014).  https://doi.org/10.1145/2663716.2663726
  20. 20.
    Dasari, M., Vargas, S., Bhattacharya, A., Balasubramanian, A., Das, S.R., Ferdman, M.: Impact of device performance on mobile internet QoE. In: Internet Measurement Conference, pp. 1–7 (2018).  https://doi.org/10.1145/3278532.3278533
  21. 21.
    DoubleClick: The Need for Mobile Speed: Better User Experiences, Greater Publisher Revenue. https://goo.gl/R4Lmfh. Accessed 26 Feb 2018
  22. 22.
    Eravuchira, S.J., Bajpai, V., Schönwälder, J., Crawford, S.: Measuring web similarity from dual-stacked hosts. In: Conference on Network and Service Management, pp. 181–187 (2016).  https://doi.org/10.1109/CNSM.2016.7818415
  23. 23.
    FFmpeg: FFmpeg: a complete, cross-platform solution to record, convert and stream audio and video. https://ffmpeg.org. Accessed 12 Oct 2018
  24. 24.
    Google: Lighthouse: an open-source, automated tool for improving the quality of web pages. https://developers.google.com/web/tools/lighthouse. Accessed 09 Jan 2019
  25. 25.
    da Hora, D.N., Asrese, A.S., Christophides, V., Teixeira, R., Rossi, D.: Narrowing the gap between QoS metrics and Web QoE using above-the-fold metrics. In: Beverly, R., Smaragdakis, G., Feldmann, A. (eds.) PAM 2018. LNCS, vol. 10771, pp. 31–43. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-76481-8_3CrossRefGoogle Scholar
  26. 26.
    Hosek, J., et al.: Mobile web QoE study for smartphones. In: IEEE GLOBECOM Workshop (2013).  https://doi.org/10.1109/GLOCOMW.2013.6825149
  27. 27.
    Hoßfeld, T., Metzger, F., Rossi, D.: Speed index: relating the industrial standard for user perceived web performance to web QoE. In: IEEE International Conference on Quality of Multimedia Experience (2018).  https://doi.org/10.1109/QoMEX.2018.8463430
  28. 28.
    Li, L., et al.: A longitudinal measurement study of TCP performance and behavior in 3G/4G networks over high speed rails. IEEE/ACM Trans. Netw. 25(4), 2195–2208 (2017).  https://doi.org/10.1109/TNET.2017.2689824CrossRefGoogle Scholar
  29. 29.
    Mandalari, A.M., et al.: Experience: implications of roaming in Europe. In: MOBICOM, pp. 179–189 (2018).  https://doi.org/10.1145/3241539.3241577
  30. 30.
  31. 31.
    Nejati, J., Balasubramanian, A.: An in-depth study of mobile browser performance. In: Conference on World Wide Web, pp. 1305–1315 (2016).  https://doi.org/10.1145/2872427.2883014
  32. 32.
    OpenSignal: Meteor. https://meteor.opensignal.com. Accessed 12 May 2017
  33. 33.
    Sackl, A., Casas, P., Schatz, R., Janowski, L., Irmer, R.: Quantifying the impact of network bandwidth fluctuations and outages on Web QoE. In: IEEE International Workshop on Quality of Multimedia Experience (2015).  https://doi.org/10.1109/QoMEX.2015.7148078
  34. 34.
    Sonntag, S., Manner, J., Schulte, L.: Netradar - measuring the wireless world. In: IEEE International Symposium and Workshops on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks (2013). http://ieeexplore.ieee.org/document/6576402/
  35. 35.
    Varvello, M., Blackburn, J., Naylor, D., Papagiannaki, K.: EYEORG: a platform for crowdsourcing web quality of experience measurements. In: ACM Conference on emerging Networking EXperiments and Technologies (2016).  https://doi.org/10.1145/2999572.2999590
  36. 36.
    Walelgne, E.A., Kim, S., Bajpai, V., Neumeier, S., Manner, J., Ott, J.: Factors affecting performance of web flows in cellular networks. In: IFIP Networking (2018)Google Scholar
  37. 37.
    Walelgne, E.A., Manner, J., Bajpai, V., Ott, J.: Analyzing throughput and stability in cellular networks. In: IEEE/IFIP Network Operations and Management Symposium, pp. 1–9 (2018).  https://doi.org/10.1109/NOMS.2018.8406261

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Alemnew Sheferaw Asrese
    • 1
    Email author
  • Ermias Andargie Walelgne
    • 1
  • Vaibhav Bajpai
    • 2
  • Andra Lutu
    • 4
  • Özgü Alay
    • 3
  • Jörg Ott
    • 2
  1. 1.Aalto UniversityEspooFinland
  2. 2.Technische Universität MünchenMunichGermany
  3. 3.Simula MetropolitanOsloNorway
  4. 4.Telefonica ResearchBarcelonaSpain

Personalised recommendations