QoE evaluation of dynamic adaptive streaming over HTTP (DASH) with promising transport layer protocols

Transport layer protocol performance over HTTP/2 DASH

Abstract

Recently, dynamic adaptive streaming over HTTP (DASH) has become an increasingly popular way to view video over the Internet. In particular compared to other video streaming services these DASH approaches deliver superior QoE to viewers. This is due to improved video segment selection. Generally, Transmission Control Protocol (TCP) CUBIC is the defacto transport layer protocol use by DASH. To improve the robustness of the transport layer to network congestion many other TCP variants were implemented such as Compound TCP and BBR. Nevertheless some were made to work specifically in LAN environments for example Agile-SD. However, recently another transport layer protocol User Datagram Protocol (UDP) has been used in Google’s QUIC implementation. To date no work has be found giving the performance of these transport layer protocols with DASH. In this paper we test the performance of Agile-SD, CUBIC, Compound TCP, BBR and QUIC using the BBA, MPC, Pensieve and Oboe DASH approaches. Experiments simulate congested bottleneck link conditions common at household routers where families view multiple videos at the same time. We observe Oboe and Agile-SD is the most promising combination with CUBIC and Pensieve next. However, even though QUIC was touted to have superior transport layer performance by Google it was the worst performing.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Notes

  1. 1.

    Specifically TCP.

  2. 2.

    Dummynet on FreeBSD, see https://papers.freebsd.org/1997/luigi-dummynet/.

  3. 3.

    \(\lambda _{min}\) is set to 1 and \(\lambda _{max}\) is set to 4 as in Alrshah et al. (2015).

  4. 4.

    C is set to 0.4, \(\beta\) to 0.2 as in [].

  5. 5.

    \(\beta\) to 0.5 as in Wu et al. (2009).

  6. 6.

    ProbeRTT which is triggered at most every 10 s, pacing_gain = 1.25 and 0.75 as in Hock et al. (2017).

  7. 7.

    Implementation details can be found at Wang et al. (2018).

  8. 8.

    http://www.free-codecs.com/download/x264_vfw.htm.

  9. 9.

    http://mirrors.vbi.vt.edu/linux/opensuse/repositories/packman/11.0/x86_64/.

  10. 10.

    http://www-itec.uni-klu.ac.at/ftp/datasets/DASHDataset2014/ElephantsDream/.

  11. 11.

    The throughput lower bound is obtained using harmonic mean of past 5 segments.

  12. 12.

    The prediction error is the maximum absolute percentage error of the past 5 segments.

  13. 13.

    First 30 s of streaming.

  14. 14.

    RobustMPC+Oboe and Agile-SD.

  15. 15.

    The fewer switches the better.

  16. 16.

    60 s.

  17. 17.

    Segment map represents the maximally allowable segment size according to the buffer occupancy (Huang et al. 2014).

  18. 18.

    CTCP is better than BBR and BBR is better than QUIC.

  19. 19.

    Recall we set \(\beta\) to 0.2 in our experiments.

  20. 20.

    CUBIC.

  21. 21.

    Request lower quality video segments.

  22. 22.

    Request higher quality video segments.

References

  1. Akhshabi, S., Anantakrishnan, L., Begen, A.C., Dovrolis, C.: What happens when http adaptive streaming players compete for bandwidth? In: Proceedings of the 22nd international workshop on Network and Operating System Support for Digital Audio and Video, pp. 9–14. ACM (2012)

  2. Akhtar, Z., Nam, Y.S., Govindan, R., Rao, S., Chen, J., Katz-Bassett, E., Ribeiro, B., Zhan, J., Zhang, H.: Oboe: auto-tuning video abr algorithms to network conditions. In: Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, pp. 44–58 (2018)

  3. Al Mtawa, Y., Haque, A., Bitar, B.: Does internet of things disrupt residential bandwidth consumption? In: 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall), pp. 1–5. IEEE (2018)

  4. Al-Kashoash, H.: Optimization-based hybrid congestion alleviation. In: Congestion Control for 6LoWPAN Wireless Sensor Networks: Toward the Internet of Things, pp. 135–156. Springer (2020)

  5. Allen, D.M.: Mean square error of prediction as a criterion for selecting variables. Technometrics 13(3), 469–475 (1971)

    Article  Google Scholar 

  6. Alrshah, M.A., Othman, M., Ali, B., Hanapi, Z.M.: Agile-sd: A linux-based tcp congestion control algorithm for supporting high-speed and short-distance networks. J. Netw. Comput. Appl. 55, 181–190 (2015)

    Article  Google Scholar 

  7. Bae, S., Jang, D., Park, K.: Why is http adaptive streaming so hard? In: Proceedings of the 6th Asia-Pacific Workshop on Systems, p. 12. ACM (2015)

  8. Balachandran, A., Sekar, V., Akella, A., Seshan, S., Stoica, I., Zhang, H.: Developing a predictive model of quality of experience for internet video. In: ACM SIGCOMM Computer Communication Review, vol. 43, pp. 339–350. ACM (2013)

  9. Barman, N., Zadtootaghaj, S., Schmidt, S., Martini, M.G., Möller, S.: Gamingvideoset: a dataset for gaming video streaming applications. In: 2018 16th Annual Workshop on Network and Systems Support for Games (NetGames), pp. 1–6. IEEE (2018)

  10. Barré, S., Bonaventure, O., Raiciu, C., Handley, M.: Experimenting with multipath tcp. ACM SIGCOMM Comput. Commun. Rev. 41(4), 443–444 (2011)

    Article  Google Scholar 

  11. Belshe, M., Peon, R., Thomson, M.: Hypertext transfer protocol version 2 (http/2) (2015)

  12. Berners-Lee, T., Fielding, R., Frystyk, H.: Hypertext transfer protocol–http/1.0 (1996)

  13. Bishop, M., et al.: Hypertext transfer protocol version 3 (http/3). Internet Engineering Task Force, Internet-Draft draft-ietf-quic-http-20 (2019)

  14. Brooks, P., Hestnes, B.: User measures of quality of experience: why being objective and quantitative is important. IEEE Netw. 24(2), 8–13 (2010)

    Article  Google Scholar 

  15. Cai, W., Chi, F., Wang, X., Leung, V.C.: Toward multiplayer cooperative cloud gaming. IEEE Cloud Comput. 5(5), 70–80 (2018)

    Article  Google Scholar 

  16. Callegari, C., Giordano, S., Pagano, M., Pepe, T.: Behavior analysis of tcp linux variants. Comput. Netw. 56(1), 462–476 (2012)

    Article  Google Scholar 

  17. Cardwell, N., Cheng, Y., Gunn, C.S., Yeganeh, S.H., Jacobson, V.: Bbr: congestion-based congestion control. Queue 14(5), 20–53 (2016)

    Article  Google Scholar 

  18. Carlucci, G., De Cicco, L., Mascolo, S.: Http over udp: an experimental investigation of quic. In: Proceedings of the 30th Annual ACM Symposium on Applied Computing, pp. 609–614 (2015)

  19. Chen, B., Yan, Z., Jin, H., Nahrstedt, K.: Event-driven stitching for tile-based live 360 video streaming. In: Proceedings of the 10th ACM Multimedia Systems Conference, pp. 1–12. ACM (2019)

  20. Chiu, D.M., Jain, R.: Analysis of the increase and decrease algorithms for congestion avoidance in computer networks. Comput. Netw. ISDN Syst. 17(1), 1–14 (1989)

    Article  Google Scholar 

  21. De Cicco, L., Mascolo, S.: An adaptive video streaming control system: Modeling, validation, and performance evaluation. IEEE/ACM Trans. Netw. 22(2), 526–539 (2014). 10.1109/TNET.2013.2253797. http://dx.doi.org/10.1109/TNET.2013.2253797

  22. De Cicco, L., Caldaralo, V., Palmisano, V., Mascolo, S.: Tapas: a tool for rapid prototyping of adaptive streaming algorithms. In: Proceedings of the 2014 Workshop on Design, Quality and Deployment of Adaptive Video Streaming, pp. 1–6. ACM (2014)

  23. Dobrian, F., Sekar, V., Awan, A., Stoica, I., Joseph, D., Ganjam, A., Zhan, J., Zhang, H.: Understanding the impact of video quality on user engagement. In: ACM SIGCOMM Computer Communication Review, vol. 41, pp. 362–373. ACM (2011)

  24. Egevang, K., Francis, P., et al.: The ip network address translator (nat). Tech. rep., RFC 1631, may (1994)

  25. Eggert, L., Fairhurst, G.: Unicast udp usage guidelines for application designers. Internet society (2008)

  26. Ha, S., Rhee, I., Xu, L.: Cubic: a new tcp-friendly high-speed tcp variant. ACM SIGOPS Oper. Syst. Rev. 42(5), 64–74 (2008)

    Article  Google Scholar 

  27. Hock, M., Bless, R., Zitterbart, M.: Experimental evaluation of bbr congestion control. In: 2017 IEEE 25th International Conference on Network Protocols (ICNP), pp. 1–10. IEEE (2017)

  28. Hoßfeld, T., Schatz, R., Biersack, E., Plissonneau, L.: Internet video delivery in youtube: From traffic measurements to quality of experience. In: Data Traffic Monitoring and Analysis, pp. 264–301. Springer (2013)

  29. Hoßfeld, T., Seufert, M., Hirth, M., Zinner, T., Tran-Gia, P., Schatz, R.: Quantification of youtube qoe via crowdsourcing. In: ISM, pp. 494–499 (2011)

  30. Huang, T.Y., Johari, R., McKeown, N., Trunnell, M., Watson, M.: A buffer-based approach to rate adaptation: Evidence from a large video streaming service. In: ACM SIGCOMM Computer Communication Review, vol. 44, pp. 187–198. ACM (2014)

  31. Huynh-Thu, Q., Ghanbari, M.: Scope of validity of psnr in image/video quality assessment. Electron. Lett. 44(13), 800–801 (2008a)

    Article  Google Scholar 

  32. Huynh-Thu, Q., Ghanbari, M.: Temporal aspect of perceived quality in mobile video broadcasting. IEEE Trans. Broadcast. 54(3), 641–651 (2008b)

    Article  Google Scholar 

  33. ITU-T, R.H.: 323,“. Visual Telephone Systems and Equipment for Local Area Networks which provide a Non-Guaranteed Quality of Service (1998)

  34. Jiang, J., Sekar, V., Zhang, H.: Improving fairness, efficiency, and stability in http-based adaptive video streaming with festive. IEEE/ACM Trans. Netw. (ToN) 22(1), 326–340 (2014)

    Article  Google Scholar 

  35. Johansson, I., Westerlund, M., et al.: Support for reduced-size real-time transport control protocol (rtcp): Opportunities and consequences. Tech. rep., RFC 5506, April (2009)

  36. Khan, K., Goodridge, W.: Energy aware ad-hoc on demand multipath distance vector routing. Int. J. Intell. Syst. Appl. 7(7), 50–56 (2015)

    Google Scholar 

  37. Khan, K., Goodridge, W.: B-dash: broadcast-based dynamic adaptive streaming over http. Int. J. Auto. Adapt. Commun. Syst. 12(1), 50–74 (2019a)

    Article  Google Scholar 

  38. Khan, K., Goodridge, W.: S-mdp: Streaming with markov decision processes. IEEE Trans. Multim. 21(8), 2012–2025 (2019b)

    Article  Google Scholar 

  39. Kleinrock, L.: Internet congestion control using the power metric: Keep the pipe just full, but no fuller. Ad Hoc Netw. 80, 142–157 (2018)

    Article  Google Scholar 

  40. Kleinrock, L., Lam, S.: Packet switching in a multiaccess broadcast channel: Performance evaluation. IEEE Trans. Commun. 23(4), 410–423 (1975)

    Article  Google Scholar 

  41. Laghari, K.U.R., Connelly, K.: Toward total quality of experience: a qoe model in a communication ecosystem. IEEE Commun. Mag. 50(4), 58–65 (2012)

    Article  Google Scholar 

  42. Lefelhocz, C., Lyles, B., Shenker, S., Zhang, L.: Congestion control for best-effort service: why we need a new paradigm. IEEE Netw. 10(1), 10–19 (1996)

    Article  Google Scholar 

  43. Li, Z., Zhu, X., Gahm, J., Pan, R., Hu, H., Begen, A.C., Oran, D.: Probe and adapt: Rate adaptation for http video streaming at scale. IEEE J. Select. Areas Commun. 32(4), 719–733 (2014)

    Article  Google Scholar 

  44. Liu, Y., Liu, X., Jing, Y., Zhou, S.: Adaptive backstepping h tracking control with prescribed performance for internet congestion. ISA Trans. 72, 92–99 (2018)

    Article  Google Scholar 

  45. Mao, H., Netravali, R., Alizadeh, M.: Neural adaptive video streaming with pensieve. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication, pp. 197–210. ACM (2017)

  46. McZeal Jr, A.: Multifunctional world wide walkie talkie, a tri-frequency cellular-satellite wireless instant messenger computer and network for establishing global wireless volp quality of service (qos) communications, unified messaging, and video conferencing via the internet (2004). US Patent 6,763,226

  47. Mok, R.K., Chan, E.W., Chang, R.K.: Measuring the quality of experience of http video streaming. In: Integrated Network Management, pp. 485–492 (2011)

  48. Moldovan, C., Wamser, F., Hoßfeld, T.: User behavior and engagement of a mobile video streaming user from crowdsourced measurements. In: 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX), pp. 1–3. IEEE (2019)

  49. Ni, P., Eg, R., Eichhorn, A., Griwodz, C., Halvorsen, P.: Flicker effects in adaptive video streaming to handheld devices. In: Proceedings of the 19th ACM International Conference on Multimedia, MM ’11, pp. 463–472. ACM, New York, NY, USA (2011). https://doi.org/10.1145/2072298.2072359. http://doi.acm.org/10.1145/2072298.2072359

  50. Ni, P., Eg, R., Eichhorn, A., Griwodz, C., Halvorsen, P.: Flicker effects in adaptive video streaming to handheld devices. In: Proceedings of the 19th ACM international conference on Multimedia, pp. 463–472. ACM (2011)

  51. Niu, Z., Xu, H., Han, D., Cheng, P., Xiong, Y., Chen, G., Winstein, K.: Network stack as a service in the cloud. In: Proceedings of the 16th ACM Workshop on Hot Topics in Networks, pp. 65–71 (2017)

  52. Obrist, M., Cesar, P., Geerts, D., Bartindale, T., Churchill, E.F.: Online video and interactive tv experiences. interactions 22(5), 32–37 (2015)

  53. Oyman, O., Singh, S.: Quality of experience for http adaptive streaming services. IEEE Commun. Mag. 50(4), 20–27 (2012)

    Article  Google Scholar 

  54. Postel, J.: Transmission control protocol (1981)

  55. Postel, J.: User datagram protocol. Isi (1980)

  56. Qi, Y., Dai, M.: The effect of frame freezing and frame skipping on video quality. In: 2006 international conference on intelligent information hiding and multimedia, pp. 423–426. IEEE (2006)

  57. Ramrekha, T.A., Politis, C.: An adaptive qos routing solution for manet based multimedia communications in emergency cases. In: International Conference on Mobile Lightweight Wireless Systems, pp. 74–84. Springer (2009)

  58. Recommendation, I.T.: Perceptual evaluation of speech quality (pesq): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs. Rec. ITU-T P. 862 (2001)

  59. Rizzo, L.: Dummynet: a simple approach to the evaluation of network protocols. ACM SIGCOMM Comput. Commun. Revi. 27(1), 31–41 (1997)

    Article  Google Scholar 

  60. Roos, J.P.: Audio-visual navigation and communication (2018). US Patent App. 15/817,988

  61. Roskind, J.: Quick udp internet connections: Multiplexed stream transport over udp. Adresse: https://docs. google. com/document/d/1RNHkx\_ VvKWyWg6Lr8SZ-saqsQx7rFV-ev2jRFUoVD34/edit (besucht am 05. 07. 2017) (2012)

  62. Salcedo, R.M., Espinosa, M.C.E.: Effectiveness of using smart tvs for teaching engineering. International Journal on Interactive Design and Manufacturing (IJIDeM) pp. 1–15 (2019)

  63. Sodagar, I.: The mpeg-dash standard for multimedia streaming over the internet. IEEE Multim. 18(4), 62–67 (2011)

    Article  Google Scholar 

  64. Song, K.T.J., Zhang, Q., Sridharan, M.: Compound tcp: A scalable and tcp-friendly congestion control for high-speed networks. Proceedings of PFLDnet 2006, (2006)

  65. Spiteri, K., Urgaonkar, R., Sitaraman, R.K.: Bola: Near-optimal bitrate adaptation for online videos. In: IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications, pp. 1–9. IEEE (2016)

  66. Staelens, N., De Meulenaere, J., Claeys, M., Van Wallendael, G., Van den Broeck, W., De Cock, J., Van de Walle, R., Demeester, P., De Turck, F.: Subjective quality assessment of longer duration video sequences delivered over http adaptive streaming to tablet devices. IEEE Trans. Broadcast. 60(4), 707–714 (2014)

    Article  Google Scholar 

  67. Stevens, W.R.: Tcp slow start, congestion avoidance, fast retransmit, and fast recovery algorithms (1997)

  68. Takahashi, A., Hands, D., Barriac, V.: Standardization activities in the itu for a qoe assessment of iptv. IEEE Commun. Mag. 46(2), 78–84 (2008)

    Article  Google Scholar 

  69. Varvello, M., Schomp, K., Naylor, D., Blackburn, J., Finamore, A., Papagiannaki, K.: Is the web http/2 yet? In: International Conference on Passive and Active Network Measurement, pp. 218–232. Springer (2016)

  70. Verma, S., Girdhar, A., Jha, R.R.K.: Real-time eye detection method for driver assistance system. In: Ambient Communications and Computer Systems, pp. 693–702. Springer (2018)

  71. Wang, P., Bianco, C., Riihijärvi, J., Petrova, M.: Implementation and performance evaluation of the quic protocol in linux kernel. In: Proceedings of the 21st ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, pp. 227–234 (2018)

  72. Weinberg, A., Thomas, M.: Student learning and sense-making from video lectures. Int. J. Math. Edu. Sci. Technol. 49(6), 922–943 (2018)

    Article  Google Scholar 

  73. Westerlund, M., Johansson, I., Perkins, C., O’Hanlon, P., Carlberg, K.: Explicit congestion notification (ecn) for rtp over udp. Internet Engineering Task Force (2012)

  74. Wu, X., Chan, M.C., Ananda, A.L., Ganjihal, C.: Sync-tcp: A new approach to high speed congestion control. In: 2009 17th IEEE International Conference on Network Protocols, pp. 181–192. IEEE (2009)

  75. Yin, X., Jindal, A., Sekar, V., Sinopoli, B.: A control-theoretic approach for dynamic adaptive video streaming over http. ACM SIGCOMM Comput. Commun. Rev. 45(4), 325–338 (2015)

    Article  Google Scholar 

  76. Zimmermann, T., Rüth, J., Wolters, B., Hohlfeld, O.: How http/2 pushes the web: An empirical study of http/2 server push. In: 2017 IFIP Networking Conference (IFIP Networking) and Workshops, pp. 1–9. IEEE (2017)

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Koffka Khan.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Khan, K., Goodridge, W. QoE evaluation of dynamic adaptive streaming over HTTP (DASH) with promising transport layer protocols. CCF Trans. Netw. 3, 245–260 (2020). https://doi.org/10.1007/s42045-020-00047-6

Download citation

Keywords

  • HTTP
  • DASH
  • Rate
  • Adaptation
  • Video
  • Segment
  • CUBIC
  • TCP
  • UDP
  • QUIC
  • Congestion
  • Compound
  • BBR
  • BBA
  • MPC
  • Pensieve
  • Agile-SD
  • Oboe