Multimedia Tools and Applications

, Volume 77, Issue 20, pp 27269–27300 | Cite as

Learning from users: a data-driven method of QoE evaluation for Internet video

  • Ting Yue
  • Hongbo WangEmail author
  • Shiduan Cheng


Improving quality of experience (QoE) is increasingly significant for Internet video content providers. The essential issue is how to evaluate QoE under the complex circumstance of Internet video. Based on the massive user data extracted from a large scale Video-on-Demand (VoD) provider, we present a data-driven, comprehensive and extendible study on the problems of QoE evaluation. The main works of this paper include obtaining QoE-associated features via feature engineering and building an evaluation model on features of different aspects for Internet video QoE. Firstly, for feature engineering, we propose to introduce pattern features of user viewing behaviors that interact with user-perceived video quality. A new method of frequent time series pattern mining is proposed to find typical patterns. We correlate user experience with user-perceived quality features and user behavior pattern features, and consider the impact of confounding factors by applying them as context features into modeling. Secondly, interdependency among features is challenging for QoE evaluation modeling. And the high dimension of feature vector should be considered. To address these challenges, we develop an ensemble method to model the interactions between features and their intricate relationships to user experience. Experiments demonstrate that our approach could achieve sound results in comparison with other related works.


Internet video QoE Feature engineering User behavior Time series pattern mining Ensemble method 


  1. 1.
    Agrawal R, Srikant R (1995) Mining sequential patterns. In: Proceedings of the Eleventh international conference on data engineering, 1995. IEEE, pp 3–14Google Scholar
  2. 2.
    Agrawal R, Mannila H, Srikant R, Toivonen H, Verkamo AI (1996) Fast discovery of association rules. In: Advances in knowledge discovery and data mining, pp 307–328Google Scholar
  3. 3.
    Agarwal RC, Aggarwal CC, Prasad VVV (2002) Depth first generation of long patterns. In: ACM SIGKDD International conference on knowledge discovery and data mining, pp 108–118Google Scholar
  4. 4.
    Agboma F, Liotta A (2012) Quality of experience management in mobile content delivery systems. Telecommun Syst 49(1):85–98CrossRefGoogle Scholar
  5. 5.
    Ayres J (2002) Sequential pattern mining using a bitmap representation. In: Eighth ACM SIGKDD International conference on knowledge discovery and data mining, pp 429–435Google Scholar
  6. 6.
    Balachandran A, Sekar V, Akella A, Seshan S (2013) Analyzing the potential benefits of CDN augmentation strategies for internet video workloads. In: Conference on internet measurement conference, pp 43–56Google Scholar
  7. 7.
    Balachandran A, Sekar V, Akella A, Seshan S (2013) Understanding internet video viewing behavior in the wild. In: ACM SIGMETRICS/International conference on measurement and modeling of computer systems, pp 379–380Google Scholar
  8. 8.
    Balachandran A, Sekar V, Akella A, Seshan S, Stoica I, Zhang H (2013) Developing a predictive model of quality of experience for internet video. Comput Commun Rev 43(4):339–350CrossRefGoogle Scholar
  9. 9.
    Bayardo RJ (1998) Efficiently mining long patterns from databases. ACM SIGMOD Record 27(2):85–93CrossRefGoogle Scholar
  10. 10.
    Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140zbMATHGoogle Scholar
  11. 11.
    Bryll R, Gutierrez-Osuna R, Quek F (2003) Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets. Pattern Recogn 36 (6):1291–1302CrossRefGoogle Scholar
  12. 12.
    Casas P, Schatz R, Hoßfeld T (2013) Monitoring YouTube QoE: is your mobile network delivering the right experience to your customers? In: IEEE Wireless communications and networking conference, pp 1609–1614Google Scholar
  13. 13.
    Chen L, Zhou Y, Chiu DM (2014) A study of user behavior in online VoD services. Comput Commun 46(6):66–75CrossRefGoogle Scholar
  14. 14.
    Cheng H, Yan X, Han J, Hsu CW (2007) Discriminative frequent pattern analysis for effective classification. In: IEEE International conference on data engineering, pp 716–725Google Scholar
  15. 15.
    Choi J, Reaz A, Mukherjee B (2012) A survey of user behavior in VoD service and bandwidth-saving multicast streaming schemes. Commun Surv Tutor IEEE 14(1):156–169CrossRefGoogle Scholar
  16. 16.
    Diewert E (2005) Weighted country product dummy variable regressions and index number formulae. Rev Income Wealth 51(4):561–570CrossRefGoogle Scholar
  17. 17.
    Dobrian F, Awan A, Joseph DA, Ganjam A, Zhan J, Sekar V, Stoica I, Zhang H (2013) Understanding the impact of video quality on user engagement. Commun ACM, 56(3):91–99CrossRefGoogle Scholar
  18. 18.
    Domingos P (2012) A few useful things to know about machine learning. Commun ACM 55(10):78–87CrossRefGoogle Scholar
  19. 19.
    Freudenthaler C, Schmidt-Thieme L, Rendle S (2013) Bayesian factorization machinesGoogle Scholar
  20. 20.
    Gopalakrishnan V, Jana R, Ramakrishnan KK, Swayne DF, Vaishampayan VA (2011) Understanding couch potatoes: measurement and modeling of interactive usage of IPTV at large scale. In: Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference, pp 225–242Google Scholar
  21. 21.
    Greenacre MJ (2007) Correspondence analysis in practice, 2nd edn. Crc PressGoogle Scholar
  22. 22.
    Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3(6):1157–1182zbMATHGoogle Scholar
  23. 23.
    Han J, Pei J, Yin Y (2000) Mining frequent patterns without candidate generation. ACM Sigmod Record 29(2):1–12CrossRefGoogle Scholar
  24. 24.
    Jana S, Chan A, Pande A, Mohapatra P (2016) QoE prediction model for mobile video telephony. Multimed Tools Appl 75(13):7957–7980CrossRefGoogle Scholar
  25. 25.
    Joumblatt D, Chandrashekar J, Kveton B, Taft N (2013) Predicting user dissatisfaction with Internet application performance at end-hosts. In: INFOCOM, 2013 Proceedings IEEE, pp 235–239Google Scholar
  26. 26.
    Klir GJ, Yuan B (1994) Fuzzy sets and fuzzy logic: theory and applications. Prentice-Hall, IncGoogle Scholar
  27. 27.
    Krishnan SS, Sitaraman RK (2013) Understanding the effectiveness of video ads: a measurement study. In: Conference on internet measurement conference, pp 149–162Google Scholar
  28. 28.
    Krishnan SS, Sitaraman RK (2013) Video stream quality impacts viewer behavior: inferring causality using quasi-experimental designs. IEEE/ACM Trans Network 21(6):2001–2014CrossRefGoogle Scholar
  29. 29.
    Li Z, Lin J, Akodjenou MI, Xie G, Kaafar MA, Jin Y, Peng G (2012) Watching videos from everywhere: a study of the PPTV mobile VoD system. In: ACM Conference on internet measurement conference, pp 185–198Google Scholar
  30. 30.
    Mok RKP, Chan EWW, Chang RKC (2011) Measuring the quality of experience of HTTP video streaming. In: IFIP/IEEE International symposium on integrated network management, pp 485–492Google Scholar
  31. 31.
    Mok RKP, Chan EWW, Luo X, Chang RKC (2011) Inferring the QoE of HTTP video streaming from user-viewing activities. In: ACM SIGCOMM Workshop on measurements up the stack, pp 31–36Google Scholar
  32. 32.
    Mongy S, Djeraba C, Dan AS (2007) On clustering users’ behaviors in video sessions. In: International conference on data mining, Dmin 2007. Las Vegas, pp 99–103Google Scholar
  33. 33.
    Ng A (2016) Machine learning courseraGoogle Scholar
  34. 34.
    Reiter U, Brunnström K, Moor KD, Larabi M, Pereira M, Pinheiro A, You J, Zgank A (2014) Factors influencing quality of experience, pp 55–72CrossRefGoogle Scholar
  35. 35.
    Rendle S (2012) Factorization machines with libFM. ACM Trans Intell Syst Technol 3(3):57CrossRefGoogle Scholar
  36. 36.
    Rish I (2001) An empirical study of the naive bayes classifier. J Univ Comput Sci 1(2):127Google Scholar
  37. 37.
    Robitza W, Raake A (2016) (Re-)actions speak louder than words? A novel test method for tracking user behavior in web video services. In: Eighth International conference on quality of multimedia experience, pp 1–6Google Scholar
  38. 38.
    Shenoy P, Haritsa JR, Sudarshan S, Bhalotia G, Bawa M, Shah D (2000) Turbo-charging vertical mining of large databases. ACM Sigmod Record 29(2):22–33CrossRefGoogle Scholar
  39. 39.
    Shewhart WA, Wilks SS (2005) Applied logistic regression, 2nd ednGoogle Scholar
  40. 40.
    Søgaard J, Shahid M, Pokhrel J, Brunnstrȯm K (2017) On subjective quality assessment of adaptive video streaming via crowdsourcing and laboratory based experiments. Multimed Tools Appl 76(15):16,727–16,748CrossRefGoogle Scholar
  41. 41.
    Song HH, Ge Z, Mahimkar A, Wang J, Yates J, Zhang Y, Basso A, Chen M (2011) Q-score: proactive service quality assessment in a large IPTV system. In: ACM SIGCOMM Conference on internet measurement conference, pp 195–208Google Scholar
  42. 42.
    Trouleau W, Ashkan A, Ding W, Eriksson B (2016) Just one more: modeling binge watching behavior. In: ACM SIGKDD International conference on knowledge discovery and data mining, pp 1215–1224Google Scholar
  43. 43.
    Xiao Z, Xu Y, Feng H, Yang T, Hu B, Zhou Y (2015) Modeling streaming QoE in wireless networks with large-scale measurement of user behavior. In: 2015 IEEE Global communications conference (GLOBECOM), pp 1–6Google Scholar
  44. 44.
    Yu H, Zheng D, Zhao BY, Zheng W (2006) Understanding user behavior in large-scale video-on-demand systems. In: Eurosys conference, pp 333–344Google Scholar
  45. 45.
    Yue T, Wei AM, Wang HB, Deng XD, Cheng SD (2016) A comprehensive data-driven approach to evaluating quality of experience on large-scale Internet video service. In: 2016 12th International Conference on natural computation, fuzzy systems and knowledge discovery (ICNC-FSKD). IEEE, pp 1479–1486Google Scholar
  46. 46.
    Zaki MJ, Hsiao CJ (2002) Charm: an efficient algorithm for closed itemset mining. In: Siam International conference on data mining. Arlington, pp 457–473Google Scholar
  47. 47.
    Zaki MJ, Parthasarathy S, Ogihara M, Li W et al (1997) New algorithms for fast discovery of association rules. In: KDD, vol 97, pp 283–286Google Scholar
  48. 48.
    Zhang D, Zhou D, Jin X (2017) A content-adaptive video quality assessment method for online media service. Multimed Tools Appl 76(5):7175–7195CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.State Key Laboratory of Networking and Switching TechnologyBeijing University of Posts and TelecommunicationsBeijingChina

Personalised recommendations