Skip to main content
SpringerLink
Log in

Measuring Web Quality of Experience in Cellular Networks

  • Conference paper
  • First Online:
Passive and Active Measurement (PAM 2019)

Part of the book series: Lecture Notes in Computer Science ((LNCCN,volume 11419))

Included in the following conference series:

Abstract

Measuring and understanding the end-user browsing Quality of Experience (QoE) is crucial to Mobile Network Operators (MNOs) to retain their customers and increase revenue. MNOs often use traffic traces to detect the bottlenecks and study their end-users experience. Recent studies show that Above The Fold (ATF) time better approximates the user browsing QoE compared to traditional metrics such as Page Load Time (PLT). This work focuses on developing a methodology to measure the web browsing QoE over operational Mobile Broadband (MBB) networks. We implemented a web performance measurement tool WebLAR (it stands for Web Latency And Rendering) that measures web Quality of Service (QoS) such as TCP connect time, and Time To First Byte (TTFB) and web QoE metrics including PLT and ATF time. We deployed WebLAR on 128 MONROE (a European-wide mobile measurement platform) nodes, and conducted two weeks long (May and July 2018) web measurement campaign towards eight websites from six operational MBB networks. The result shows that, in the median case, the TCP connect time and TTFB in Long Term Evolution (LTE) networks are, respectively, 160% and 30% longer than fixed-line networks. The DNS lookup time and TCP connect time of the websites varies significantly across MNOs. Most of the websites do not show a significant difference in PLT and ATF time across operators. However, Yahoo shows longer ATF time in Norwegian operators than that of the Swedish operators. Moreover, user mobility has a small impact on the ATF time of the websites. Furthermore, the website design should be taken into consideration when approximating the ATF time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. ImageMagick: tool to create, edit, compose, or convert bitmap images. https://imagemagick.org. Accessed 12 Oct 2018

  2. WebPageTest. https://www.webpagetest.org. Accessed 09 Jan 2019

  3. WebPagetest Metrics: SpeedIndex. https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index. Accessed 15 Oct 2018

  4. Ahmad, S., Haamid, A.L., Qazi, Z.A., Zhou, Z., Benson, T., Qazi, I.A.: A view from the other side: understanding mobile phone characteristics in the developing world. In: ACM IMC (2016). http://dl.acm.org/citation.cfm?id=2987470

  5. Akamai White Paper: Measuring Real Customer Experiences over Mobile Networks. https://www.akamai.com/jp/ja/multimedia/documents/white-paper/measuring-real-customer-experiences-over-mobile-networks-report.pdf. Accessed 12 Oct 2017

  6. Alay, Ö., et al.: Experience: an open platform for experimentation with commercial mobile broadband networks. In: ACM MobiCom (2017). https://doi.org/10.1145/3117811.3117812

  7. Asrese, A.S.: WebLAR: A Web Performance Measurement Tool (2019). https://github.com/alemnew/weblar

  8. Asrese, A.S., Eravuchira, S.J., Bajpai, V., Sarolahti, P., Ott, J.: Measuring web latency and rendering performance: method, tools & longitudinal dataset. IEEE Trans. Netw. Serv. Manag. (2019, to appear)

    Google Scholar 

  9. Asrese, A.S., Sarolahti, P., Boye, M., Ott, J.: WePR: a tool for automated web performance measurement. In: IEEE Globecom Workshops (2016). https://doi.org/10.1109/GLOCOMW.2016.7849082

  10. Asrese, A.S., Walelgne, E., Bajpai, V., Lutu, A., Alay, Ö., Ott, J.: Measuring web quality of experience in cellular networks (dataset) (2019). https://github.com/alemnew/2019-pam-weblar

  11. Bajpai, V., Kühlewind, M., Ott, J., Schönwälder, J., Sperotto, A., Trammell, B.: Challenges with reproducibility. In: SIGCOMM Reproducibility Workshop, pp. 1–4 (2017). https://doi.org/10.1145/3097766.3097767

  12. Bajpai, V., Schönwälder, J.: A survey on internet performance measurement platforms and related standardization efforts. IEEE Commun. Surv. Tutor. 17(3), 1313–1341 (2015). https://doi.org/10.1109/COMST.2015.2418435

    Article  Google Scholar 

  13. Balachandran, A., et al.: Modeling web quality-of-experience on cellular networks. In: ACM MobiCom (2014). https://doi.org/10.1145/2639108.2639137

  14. Barakovic, S., Skorin-Kapov, L.: Multidimensional modelling of quality of experience for mobile web browsing. Comput. Hum. Behav. 50, 314–332 (2015). https://doi.org/10.1016/j.chb.2015.03.071

    Article  Google Scholar 

  15. Brutlag, J., Abrams, Z., Meenan, P.: Above the Fold Time: Measuring Web Page Performance Visually. https://conferences.oreilly.com/velocity/velocity-mar2011/public/schedule/detail/18692

  16. Cao, Y., Nejati, J., Wajahat, M., Balasubramanian, A., Gandhi, A.: Deconstructing the energy consumption of the mobile page load. Proc. ACM Meas. Anal. Comput. Syst. 1(1), 6 (2017). https://doi.org/10.1145/3084443

    Article  Google Scholar 

  17. Casas, P., Seufert, M., Wamser, F., Gardlo, B., Sackl, A., Schatz, R.: Next to you: monitoring quality of experience in cellular networks from the end-devices. IEEE Trans. Netw. Serv. Manag. 13(2), 181–196 (2016). https://doi.org/10.1109/TNSM.2016.2537645

    Article  Google Scholar 

  18. Cecchet, E., Sims, R., He, X., Shenoy, P.J.: mBenchLab: measuring QoE of Web applications using mobile devices. In: International Symposium on Quality of Service, IWQoS (2013). https://doi.org/10.1109/IWQoS.2013.6550259

  19. Chen, Q.A., et al.: QoE doctor: diagnosing mobile app QoE with automated UI control and cross-layer analysis. In: ACM Internet Measurement Conference (2014). https://doi.org/10.1145/2663716.2663726

  20. Dasari, M., Vargas, S., Bhattacharya, A., Balasubramanian, A., Das, S.R., Ferdman, M.: Impact of device performance on mobile internet QoE. In: Internet Measurement Conference, pp. 1–7 (2018). https://doi.org/10.1145/3278532.3278533

  21. DoubleClick: The Need for Mobile Speed: Better User Experiences, Greater Publisher Revenue. https://goo.gl/R4Lmfh. Accessed 26 Feb 2018

  22. Eravuchira, S.J., Bajpai, V., Schönwälder, J., Crawford, S.: Measuring web similarity from dual-stacked hosts. In: Conference on Network and Service Management, pp. 181–187 (2016). https://doi.org/10.1109/CNSM.2016.7818415

  23. FFmpeg: FFmpeg: a complete, cross-platform solution to record, convert and stream audio and video. https://ffmpeg.org. Accessed 12 Oct 2018

  24. Google: Lighthouse: an open-source, automated tool for improving the quality of web pages. https://developers.google.com/web/tools/lighthouse. Accessed 09 Jan 2019

  25. da Hora, D.N., Asrese, A.S., Christophides, V., Teixeira, R., Rossi, D.: Narrowing the gap between QoS metrics and Web QoE using above-the-fold metrics. In: Beverly, R., Smaragdakis, G., Feldmann, A. (eds.) PAM 2018. LNCS, vol. 10771, pp. 31–43. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76481-8_3

    Chapter  Google Scholar 

  26. Hosek, J., et al.: Mobile web QoE study for smartphones. In: IEEE GLOBECOM Workshop (2013). https://doi.org/10.1109/GLOCOMW.2013.6825149

  27. Hoßfeld, T., Metzger, F., Rossi, D.: Speed index: relating the industrial standard for user perceived web performance to web QoE. In: IEEE International Conference on Quality of Multimedia Experience (2018). https://doi.org/10.1109/QoMEX.2018.8463430

  28. Li, L., et al.: A longitudinal measurement study of TCP performance and behavior in 3G/4G networks over high speed rails. IEEE/ACM Trans. Netw. 25(4), 2195–2208 (2017). https://doi.org/10.1109/TNET.2017.2689824

    Article  Google Scholar 

  29. Mandalari, A.M., et al.: Experience: implications of roaming in Europe. In: MOBICOM, pp. 179–189 (2018). https://doi.org/10.1145/3241539.3241577

  30. Mozilla: Using the Resource Timing API. https://developer.mozilla.org/en-US/docs/Web/API/Resource_Timing_API/Using_the_Resource_Timing_API. Accessed 24 May 2018

  31. Nejati, J., Balasubramanian, A.: An in-depth study of mobile browser performance. In: Conference on World Wide Web, pp. 1305–1315 (2016). https://doi.org/10.1145/2872427.2883014

  32. OpenSignal: Meteor. https://meteor.opensignal.com. Accessed 12 May 2017

  33. Sackl, A., Casas, P., Schatz, R., Janowski, L., Irmer, R.: Quantifying the impact of network bandwidth fluctuations and outages on Web QoE. In: IEEE International Workshop on Quality of Multimedia Experience (2015). https://doi.org/10.1109/QoMEX.2015.7148078

  34. Sonntag, S., Manner, J., Schulte, L.: Netradar - measuring the wireless world. In: IEEE International Symposium and Workshops on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks (2013). http://ieeexplore.ieee.org/document/6576402/

  35. Varvello, M., Blackburn, J., Naylor, D., Papagiannaki, K.: EYEORG: a platform for crowdsourcing web quality of experience measurements. In: ACM Conference on emerging Networking EXperiments and Technologies (2016). https://doi.org/10.1145/2999572.2999590

  36. Walelgne, E.A., Kim, S., Bajpai, V., Neumeier, S., Manner, J., Ott, J.: Factors affecting performance of web flows in cellular networks. In: IFIP Networking (2018)

    Google Scholar 

  37. Walelgne, E.A., Manner, J., Bajpai, V., Ott, J.: Analyzing throughput and stability in cellular networks. In: IEEE/IFIP Network Operations and Management Symposium, pp. 1–9 (2018). https://doi.org/10.1109/NOMS.2018.8406261

Download references

Author information

Authors and Affiliations

  1. Aalto University, Espoo, Finland

    Alemnew Sheferaw Asrese & Ermias Andargie Walelgne

  2. Technische Universität München, Munich, Germany

    Vaibhav Bajpai & Jörg Ott

  3. Simula Metropolitan, Oslo, Norway

    Özgü Alay

  4. Telefonica Research, Barcelona, Spain

    Andra Lutu

Authors
  1. Alemnew Sheferaw Asrese
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ermias Andargie Walelgne
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vaibhav Bajpai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andra Lutu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Özgü Alay
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jörg Ott
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alemnew Sheferaw Asrese .

Editor information

Editors and Affiliations

  1. Northeastern University, Boston, MA, USA

    David Choffnes

  2. Federal University of Rio Grande do Sul, Porto Alegre, Rio Grande do Sul, Brazil

    Marinho Barcellos

Appendices

Appendix A List and Category of Measured Webpages

The websites are selected from different categories such as social media, news websites, and WIKI pages. Moreover, while selecting these websites, the design of the websites (from simple to media-rich complex webpages) and the purpose of the websites are taken into consideration. Furthermore, for each website we selected a specific webpage that does not require user interaction to show meaningful contents to the user.

Appendix B Additional Observations

Although not specific to mobility scenario, Fig. 5(2) also shows that PLT can under- or over-estimate the web QoE. For instance, for Facebook, the onLoad event fires before all the necessary web objects in the above-the-fold area are downloaded. For these types of websites the PLT underestimates the user QoE. On the other hand, for websites like Yahoo and Reddit, the ATF is shorter compared with PLT time, which overestimates the user QoE.

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Asrese, A.S., Walelgne, E.A., Bajpai, V., Lutu, A., Alay, Ö., Ott, J. (2019). Measuring Web Quality of Experience in Cellular Networks. In: Choffnes, D., Barcellos, M. (eds) Passive and Active Measurement. PAM 2019. Lecture Notes in Computer Science(), vol 11419. Springer, Cham. https://doi.org/10.1007/978-3-030-15986-3_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-15986-3_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-15985-6

  • Online ISBN: 978-3-030-15986-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation