Skip to main content
SpringerLink
Log in

Estimation of viewers’ ratings of TV programs based on behaviors in home environments

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A system is described that can estimate a viewer’s ratings of TV programs on the basis of his/her behaviors in a home environment. A Kinect sensor, a motion-sensing device developed by Microsoft for its Xbox game console, is used to measure various behavioral parameters. The system first detects whether a viewer is present by extracting keypoint trajectories in video sequences captured by the sensor’s video camera. It then identifies whether the viewer is gazing at the TV screen or not by extracting head pose information. The extraction is carried out using two modules: a color-image-based module and a color- and depth-image-based module. The two modules share their parameters and complement each other’s characteristics. The proposed system was evaluated by having 30 participants individually spend about 2 h watching 15 TV programs in a simulated home environment, capturing video images of their behaviors, and having them rate each program on a five-point scale. Comparison of the system’s estimated ratings with the actual viewer ratings demonstrated that the system can robustly estimate a viewer’s ratings of TV programs in a home environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. AlMejrad AS (2010) Human emotions detection using brain wave signals: a challenging. Eur J Sci Res 44(4):640–659

    MATH  Google Scholar 

  2. Cai Q, Gallup D, Zhang C, Zhang Z (2010) 3D Deformable face tracking with a commodity depth camera, In: Proc. of the European conference on computer vision (ECCV), pp. 229–249

  3. Calvo RA, D’Mello S (2010) Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans Affect Comput 1(1):18–37

    Article  Google Scholar 

  4. Clippingdale S, Fujii M (2012) Video face tracking and recognition with skin region extraction and deformable template matching. Int J Multimedia Data Eng Manag 3(1):36–48

    Article  Google Scholar 

  5. Csurka G, Dance CR, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints, In: Proc. ECCV Workshop on Statistical Learning in Computer Vision, pp. 1–22

  6. Fanelli G, Weise T, Gall J, Van Gool L (2011) Real time head pose estimation from consumer depth cameras, In: Proc. Annual symposium of the German association for pattern recognition (DAGM), pp. 101–110

  7. Felfernig A, Gordea S, Jannach D, Teppan E, Zanker M (2007) A short survey of recommendation technologies in travel and tourism. OEGAI J 25(7):17–22

    Google Scholar 

  8. Funes Mora KA, Odobez J (2012) Gaze estimation from multimodal Kinect data, In: Proc. Computer vision and pattern recognition workshop (CVPRW), pp. 25–30

  9. Gajsek R, Struc V, Dobrisek S, Mihelic F (2009) Emotion recognition using linear transformations in combination with video, In: Proc. annual conference of the international speech communication association (Interspeech), pp. 1967–1970

  10. Grafsgaard JF, Fulton RM, Boyer KE, Wiebe EN, Lester JC (2012) Multimodal analysis of the implicit affective channel in computer-mediated textual communication, In: Proc. ACM international conference on multimodal interaction (ICMI), pp. 145–152

  11. Gunes H, Schuller B, Pantic M, Cowie R (2011) Emotion representation, analysis and synthesis in continuous space: a survey, In: Proc. Automatic face & gesture recognition and workshops, pp. 827–834

  12. Hernandez J, Liu Z, Hulten G, DeBarr D, Krum K, Zhang Z (2013) Measuring the engagement level of TV viewers, In: Proc. Automatic face and gesture recognition (FG), pp. 1–7

  13. Kimura A, Yonetani R, Hirayama T (2013) Computational models of human visual attention and their implementations: a survey. IEICE Trans Inf Syst E96-D(3):562–578

    Article  Google Scholar 

  14. Kinect for Windows SDK (2013) Programming guide: face tracking, doi: http://msdn.microsoft.com/en-us/library/jj130970.aspx, Microsoft MSDN

  15. Leavitt N (2006) Recommendation technology: will it boost e-commerce? IEEE Comput 39(5):13–16

    Article  Google Scholar 

  16. Lu K, Jia Y (2012) Audio-visual emotion recognition using Boltzmann Zippers, In: Proc. IEEE international conference on image processing (ICIP), pp. 2589–2592

  17. Microsoft, USA. XBOX Kinect, doi: http://www.xbox.com/kinect

  18. Mohd Zaid NH, Mohamed AM, Soliman AH (2012) Eye gesture analysis for driver Hazard awareness, World academy of science, engineering and technology (WASET) 6 (5), 1240–1246

  19. Murphy-Chutorian E (2009) Head pose estimation in computer vision: a survey. IEEE Trans Pattern Anal Mach Intel 31(4):607–626

    Article  Google Scholar 

  20. Nakano T, Yamamoto Y, Kitajo K, Takahashi T, Kitazawa S (2009) Synchronization of spontaneous eye blinks while viewing video stories. Proc R Soc B 276:3635–3644

    Article  Google Scholar 

  21. Open CV (Open source computer vision). doi: http://opencv.org/

  22. Posner MI (1980) Orienting of attention. Q J Exp Psychol 32(1):3–25

    Article  MathSciNet  Google Scholar 

  23. Shi J, Tomasi C (1994) Good features to track, In: Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 593–600

  24. Stanley D (2013) Measuring attention using Microsoft Kinect, Master’s Thesis, Rochester Institute of Technology

  25. Stiefelhagen R, Yang J, Waibel A (2002) Modeling focus of attention for meeting indexing based on multiple cues. IEEE Trans Neural Netw 13(4):928–938

    Article  Google Scholar 

  26. Takahashi M, Clippingdale S, Okuda M, Yamanouchi Y, Naemura M, Shibata M (2013) An estimator for rating video contents on the basis of a viewer’s behavior in typical home environments, In: Proc. International conference on signal-image technology & internet-based systems (SITIS), pp. 6–13

  27. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features, In: Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 511–518

  28. Wiskott L, Fellous J-M, Krüger N, von der Malsburg C (1996) Face recognition by elastic bunch graph matching, TR96-08, Institut für Neuroinformatik, Ruhr-Universität Bochum

  29. Yamamoto M, Nitta N, Babaguchi N (2008) Automatic personal preference acquisition from TV viewer’s behaviors, In: Proc. IEEE international conference on multimedia & expo (ICME), pp. 1165–1168

  30. Yasuma Y, Nakanishi M (2011) User characteristic-based information-providing service for museum with optical see-through head-mounted display: does it evoke enthusiasm? In: Proc. International conference on human-computer interaction (HCI), pp. 234–242

Download references

Acknowledgments

Part of this work was supported by the Strategic Information and Communications R&D Promotion Programme (SCOPE) of the Ministry of Internal Affairs and Communication of Japan.

Author information

Authors and Affiliations

  1. Japan Broadcasting Corporation (NHK) Science and Technology Research Laboratories, 1-10-11, Kinuta, Setagaya-ku, Tokyo, Japan

    Masaki Takahashi, Simon Clippingdale, Masahide Naemura & Masahiro Shibata

Authors
  1. Masaki Takahashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Clippingdale
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Masahide Naemura
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Masahiro Shibata
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Masaki Takahashi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Takahashi, M., Clippingdale, S., Naemura, M. et al. Estimation of viewers’ ratings of TV programs based on behaviors in home environments. Multimed Tools Appl 74, 8669–8684 (2015). https://doi.org/10.1007/s11042-014-2352-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2352-0

Keywords

Search

Navigation