Advertisement

Computing

, Volume 101, Issue 11, pp 1585–1607 | Cite as

Understanding and estimating quality of experience in WebRTC applications

  • Boni GarcíaEmail author
  • Micael Gallego
  • Francisco Gortázar
  • Antonia Bertolino
Article

Abstract

WebRTC comprises a set of technologies and standards that provide real-time communication with web browsers, simplifying the embedding of voice and video communication in web applications and mobile devices. The perceived quality of WebRTC communication can be measured using quality of experience (QoE) indicators. QoE is defined as the degree of delight or annoyance of the user with an application or service. This paper is focused on the QoE assessment of WebRTC-based applications and its contribution is threefold. First, an analysis of how WebRTC topologies affect the quality perceived by users is provided. Second, a group of Key Performance Indicators for estimating the QoE of WebRTC users is proposed. Finally, a systematic survey of the literature on QoE assessment in the WebRTC arena is presented.

Keywords

WebRTC Quality of experience QoE management 

Mathematics Subject Classification

68N30 Mathematical aspects of software engineering (specification, verification, metrics, requirements, etc.) 

Notes

Acknowledgements

This work has been supported by the European Commission under project ElasTest (H2020-ICT-10-2016, GA-731535); by the Regional Government of Madrid (CM) under project Cloud4BigData (S2013/ICE-2894) cofunded by FSE & FEDER; and the Spanish Government under project LERNIM (RTC-2016-4674-7) cofunded by the Ministry of Economy and Competitiveness, FEDER & AEI.

References

  1. 1.
    Alvestrand H, Holmer S (2012) A Google congestion control for real-time communication on the World Wide Web. Tech. rep, IETFGoogle Scholar
  2. 2.
    Ammar D, De Moor K, Xie M, Fiedler M, Heegaard P (2016) Video QoE killer and performance statistics in WebRTC-based video communication. In: Communications and Electronics (ICCE), 2016 IEEE Sixth International Conference on, IEEE, pp 429–436Google Scholar
  3. 3.
    Bandung Y, Subekti LB, Tanjung D, Chrysostomou C (2017) QoS analysis for WebRTC videoconference on bandwidth-limited network. In: 2017 20th International symposium on wireless personal multimedia communications (WPMC), pp 547–553Google Scholar
  4. 4.
    Bevan N (1999) Quality in use: meeting user needs for quality. J Syst Softw 49(1):89–96CrossRefGoogle Scholar
  5. 5.
    Boubendir A, Bertin E, Simoni N (2016) On-demand, dynamic and at-the-edge VNF deployment model application to Web Real-Time Communications. In: Network and service management (CNSM), 2016 12th international conference on, IEEE, pp 318–323Google Scholar
  6. 6.
    Brunnström K, Beker SA, De Moor K, Dooms A, Egger S, Garcia MN, Hossfeld T, Jumisko-Pyykkö S, Keimel C, Larabi MC, et al (2013) Qualinet white paper on definitions of quality of experienceGoogle Scholar
  7. 7.
    Carlucci G, De Cicco L, Holmer S, Mascolo S (2016) Analysis and design of the Google congestion control for web real-time communication (WebRTC). In: Proceedings of the 7th international conference on multimedia systems, ACM, pp 13:1–13:12Google Scholar
  8. 8.
    Carullo G, Tambasco M, Di Mauro M, Longo M (2016) A performance evaluation of WebRTC over LTE. In: Wireless on-demand network systems and services (WONS), 2016 12th annual conference on, IEEE, pp 1–6Google Scholar
  9. 9.
    Chandler DM, Hemami SS (2007) VSNR: a wavelet-based visual signal-to-noise ratio for natural images. IEEE Trans Image Process 16(9):2284–2298MathSciNetCrossRefGoogle Scholar
  10. 10.
    Chen Y, Wu K, Zhang Q (2015) From QoS to QoE: a tutorial on video quality assessment. IEEE Commun Surv Tutor 17(2):1126–1165CrossRefGoogle Scholar
  11. 11.
    Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: a classification, review, and performance comparison. IEEE Trans Broadcast 57(2):165–182CrossRefGoogle Scholar
  12. 12.
    Chodorek RR, Chodorek A, Rzym G, Wajda K (2017) A comparison of QoS parameters of WebRTC videoconference with conference bridge placed in private and public cloud. In: Enabling technologies: infrastructure for collaborative enterprises (WETICE), 2017 IEEE 26th international conference on, IEEE, pp 86–91Google Scholar
  13. 13.
    Chong HM, Matthews HS (2004) Comparative analysis of traditional telephone and voice-over-internet protocol (voip) systems. In: Electronics and the environment, 2004. Conference record. 2004 IEEE international symposium on, IEEE, pp 106–111Google Scholar
  14. 14.
    Cisco VNI (2017) Forecast and Methodology, 2016–2021. White PaperGoogle Scholar
  15. 15.
    Edan NM, Al-Sherbaz A, Turner S (2017) WebNSM: A novel scalable WebRTC signalling mechanism for many-to-many video conferencing. In: Collaboration and internet computing (CIC), 2017 IEEE 3rd international conference on, IEEE, pp 27–33Google Scholar
  16. 16.
    Egger S, Schatz R, Scherer S (2010) It takes two to tango—assessing the impact of delay on conversational interactivity on perceived speech quality. In: 11th Annual conference of the international speech communication association (ISCA), pp 1321–1324Google Scholar
  17. 17.
    García B, Gortázar F, López-Fernández L, Gallego M (2017a) WebRTC testing: challenges and practical solutions. IEEE Commun Stand Mag 1(2):36–42CrossRefGoogle Scholar
  18. 18.
    García B, López-Fernández L, Gallego M, Gortázar F (2017b) Kurento: the Swiss army knife of WebRTC media servers. IEEE Commun Stand Mag 1(2):44–51CrossRefGoogle Scholar
  19. 19.
    Garvin DA (1984) What does “product quality” really mean? MIT Sloan Manag Rev 26(1):25–43Google Scholar
  20. 20.
    Grigorik I (2013) High performance browser networking: what every web developer should know about networking and web performance. O’Reilly Media, IncGoogle Scholar
  21. 21.
    Handley M, Perkins C, Jacobson V (2006) RFC 4566. Session description protocol. Tech. rep., IETF, SDPGoogle Scholar
  22. 22.
    Hekstra AP, Beerends JG, Ledermann D, De Caluwe F, Kohler S, Koenen R, Rihs S, Ehrsam M, Schlauss D (2002) PVQM: a perceptual video quality measure. Signal Process: Image Commun 17(10):781–798Google Scholar
  23. 23.
    Herrero R (2017) Integrating HEC with circuit breakers and multipath RTP to improve RTC media quality. Telecommun Syst 64(1):211–221CrossRefGoogle Scholar
  24. 24.
    Hoßfeld T, Schatz R, Varela M, Timmerer C (2012) Challenges of QoE management for cloud applications. IEEE Commun Mag 50(4):28–36CrossRefGoogle Scholar
  25. 25.
    Husić JB, Baraković S, Veispahić A (2017) What factors influence the quality of experience for WebRTC video calls? In: Information and communication technology, electronics and microelectronics (MIPRO), 2017 40th international convention on, IEEE, pp 428–433Google Scholar
  26. 26.
    Huynh-Thu Q, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assessment. Electron Lett 44(13):800–801CrossRefGoogle Scholar
  27. 27.
    ISO (2005) ISO 9000: Quality management systems—fundamentals and vocabulary. International Organization for StandardizationGoogle Scholar
  28. 28.
    ITU-R (2007) Recommendation BT.1788. Methodology for the subjective assessment of video quality in multimedia applicationsGoogle Scholar
  29. 29.
    ITU-T (2001) Recommendation G.1010. End-user multimedia QoS categoriesGoogle Scholar
  30. 30.
    ITU-T (2003) Recommendation G.114. Transmission systems and media digital systems and networksGoogle Scholar
  31. 31.
    ITU-T (2006) Recommendation P.10. Vocabulary for performance and quality of serviceGoogle Scholar
  32. 32.
    ITU-T (2008a) Recommendation E.800. Definitions of terms related to quality of serviceGoogle Scholar
  33. 33.
    ITU-T (2008b) Recommendation J.247. Objective perceptual multimedia video quality measurement in the presence of a full referenceGoogle Scholar
  34. 34.
    ITU-T (2011) Recommendation P.863. Perceptual objective listening quality assessmentGoogle Scholar
  35. 35.
    ITU-T (2014) Recommendation E.807: definitions, associated measurement methods and guidance targets of user-centric parameters for call handling in cellular mobile voice serviceGoogle Scholar
  36. 36.
    ITU-T (2016a) Recommendation P.10/G.100. Vocabulary for performance and quality of service (Amendment 5)Google Scholar
  37. 37.
    ITU-T (2016b) Recommendation. P.800.2. Mean opinion score interpretation and reportingGoogle Scholar
  38. 38.
    Ivov E, Rescorla E, Uberti J (2013) Trickle ICE: incremental provisioning of candidates for the interactive connectivity establishment (ICE) protocol. Tech. rep, IETFGoogle Scholar
  39. 39.
    Jin J, Nahrstedt K (2004) QoS specification languages for distributed multimedia applications: a survey and taxonomy. IEEE Multimed 11(3):74–87CrossRefGoogle Scholar
  40. 40.
    Johansson I, Sarker Z (2017) Self-clocked rate adaptation for multimedia. Tech. rep, IETFGoogle Scholar
  41. 41.
    Khan M (2017) WebRTCPedia! the Encyclopedia! https://www.webrtc-experiment.com/webrtcpedia/, [Online; accessed 11 June 2018]
  42. 42.
    Kilinc C, Andersson K (2014) A congestion avoidance mechanism for WebRTC interactive video sessions in LTE networks. Wirel Pers Commun 77(4):2417–2443CrossRefGoogle Scholar
  43. 43.
    Kitchenham B (2004) Procedures for performing systematic reviews. Keele UK Keele Univ 33(2004):1–26Google Scholar
  44. 44.
    Kitchenham B, Pfleeger SL (1996) Software quality: the elusive target [special issues section]. IEEE Softw 13(1):12–21CrossRefGoogle Scholar
  45. 45.
    Komperda O, Melvin H, Pota P (2016) A black box analysis of WebRTC mouth-to-ear delays. Communications pp 11–16Google Scholar
  46. 46.
    Loreto S, Romano SP (2014) Real-time communication with WebRTC: peer-to-peer in the browser. O’Reilly Media, IncGoogle Scholar
  47. 47.
    Matthews P, Mahy R, Rosenberg J (2010) RFC 5766. Traversal using relays around NAT (TURN): relay extensions to session traversal utilities for NAT (STUN). Tech. rep., IETFGoogle Scholar
  48. 48.
    Muñoz-Gea JP, Aparicio-Pardo R, Wehbe H, Simon G, Nuaymi L (2014) Optimization framework for uplink video transmission in HetNets. In: Proceedings of workshop on mobile video delivery, ACMGoogle Scholar
  49. 49.
    Pinson MH, Wolf S (2004) A new standardized method for objectively measuring video quality. IEEE Trans Broadcast 50(3):312–322CrossRefGoogle Scholar
  50. 50.
    Rix AW, Hollier MP, Hekstra AP, Beerends JG (2002) Perceptual evaluation of speech quality (PESQ) the new ITU standard for end-to-end speech quality assessment Part I-Time-delay compensation. J Audio Eng Soc 50(10):755–764Google Scholar
  51. 51.
    Rosenberg J (2010) RFC 5245. Interactive connectivity establishment (ICE): a methodology for network address translator (NAT) traversal for offer/answer protocols. Tech. rep., IETFGoogle Scholar
  52. 52.
    Rosenberg J, Schulzrinne H (2002) RFC 3264. An offer/answer model with SDP. Tech. rep, IETFGoogle Scholar
  53. 53.
    Rosenberg J, Weinberger J, Huitema C, Mahy R (2003) RFC 3489. STUN, simple traversal of user datagram protocol (UDP) through network address translators (NATs). Tech. rep., IETFGoogle Scholar
  54. 54.
    Rosenberg J, Mahy R, Matthews P, Wing D (2008) RFC 5389. Session traversal utilities for NAT (STUN)Tech. rep., IETFGoogle Scholar
  55. 55.
    Sale S, Rebbeck T (2014) Operators need to engage with WebRTC and the opportunities it presents. Analysis Mason ReportGoogle Scholar
  56. 56.
    Santos-González I, Rivero-García A, Molina-Gil J, Caballero-Gil P (2017) Implementation and analysis of real-time streaming protocols. Sensors 17(4):1–17CrossRefGoogle Scholar
  57. 57.
    Takahashi A, Hands D, Barriac V (2008) Standardization activities in the ITU for a QoE assessment of IPTV. IEEE Commun Mag 46(2):78–84CrossRefGoogle Scholar
  58. 58.
    Timmerer C, Ebrahimi T, Pereira F (2015) Toward a new assessment of quality. Computer 48(3):108–110CrossRefGoogle Scholar
  59. 59.
    Tsiaras C, Rösch M, Stiller B (2015) VoIP-based calibration of the DQX Model. In: IFIP networking conference (IFIP networking), 2015, IEEE, pp 1–9Google Scholar
  60. 60.
    Vucic D, Skorin-Kapov L (2015) The impact of mobile device factors on QoE for multi-party video conferencing via WebRTC. In: Telecommunications (ConTEL), 2015 13th international conference on, IEEE, pp 1–8Google Scholar
  61. 61.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612CrossRefGoogle Scholar
  62. 62.
    Westerlund M, Wenger S (2015) RFC 5117. RTP topologies. Tech. rep, IETFGoogle Scholar
  63. 63.
    Xiao F (2000) DCT-based video quality evaluation. Final Project for EE392JGoogle Scholar
  64. 64.
    Zhang L, Amin SO, Westphal C (2017) VR video conferencing over named data networks. In: Proceedings of the workshop on virtual reality and augmented reality network, ACM, pp 7–12Google Scholar
  65. 65.
    Zhao T, Liu Q, Chen CW (2017) QoE in video transmission: a user experience-driven strategy. IEEE Commun Surv Tutor 19(1):285–302CrossRefGoogle Scholar
  66. 66.
    Zhu X, Pan R, Ramalho MA, Mena S, Ganzhorn C, Jones PE (2015) NADA: A unified congestion control scheme for real-time media. Tech. rep, IETFGoogle Scholar

Copyright information

© Springer-Verlag GmbH Austria, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Universidad Rey Juan CarlosMadridSpain
  2. 2.Consiglio Nazionale delle RicercheRomeItaly

Personalised recommendations