Skip to main content

KINterestTV - Towards Non–invasive Measure of User Interest While Watching TV

  • Conference paper

Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT,volume 425)


Is it possible to determine only by observing the behavior of a user what are his interests for a media? The aim of this project is to develop an application that can detect whether or not a user is viewing a content on the TV and use this information to build the user profile and to make it evolve dynamically. Our approach is based on the use of a 3D sensor to study the movements of a user’s head to make an implicit analysis of his behavior. This behavior is synchronized with the TV content (media fragments) and other user interactions (clicks, gestural interaction) to further infer viewer’s interest. Our approach is tested during an experiment simulating the attention changes of a user in a scenario involving second screen (tablet) interaction, a behavior that has become common for spectators and a typical source of attention switches.


  • user tracking
  • face detection
  • face direction
  • face tracking
  • visual attention
  • interest
  • TV
  • gesture


  1. Linkedtv project,

  2. Node js,

  3. Abe, K., Makikawa, M.: Spatial setting of visual attention and its appearance in head-movement. IFMBE Proceedings 25/4, 1063–1066 (2010),

    CrossRef  Google Scholar 

  4. Aldoma, A.: 3D face detection and pose estimation in pcl (September 2012)

    Google Scholar 

  5. Bailly, G., Vo, D.B., Lecolinet, E., Guiard, Y.: Gesture-aware remote controls: Guidelines and interaction technique. In: Proceedings of the 13th International Conference on Multimodal Interfaces, ICMI 2011, pp. 263–270. ACM, New York (2011),

    Google Scholar 

  6. Bettens, F., Todoroff, T.: Real-time dtw-based gesture recognition external object for max/msp and puredata. In: Proc. SMC 2009, pp. 30–35 (2009)

    Google Scholar 

  7. Bevilacqua, F., Guédy, F., Schnell, N., Fléty, E., Leroy, N.: Wireless sensor interface and gesture-follower for music pedagogy. In: Proceedings of the 7th International Conference on New Interfaces for Musical Expression, NIME 2007, pp. 124–129. ACM, New York (2007),

    Google Scholar 

  8. Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001),

    CrossRef  MATH  Google Scholar 

  9. Daugman, J.G.: Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. J. Opt. Soc. Am. A 2(7), 1160–1169 (1985),

    CrossRef  Google Scholar 

  10. Fanelli, G., Gall, J., Van Gool, L.: Real time head pose estimation with random regression forests. Cvpr 2011, 617–624 (2011),

    Google Scholar 

  11. Fanelli, G., Gall, J., Van Gool, L.: Real time 3d head pose estimation: Recent achievements and future challenges. In: 2012 5th International Symposium on Communications Control and Signal Processing (ISCCSP), pp. 1–4 (2012)

    Google Scholar 

  12. Fanelli, G., Dantone, M., Gall, J., Fossati, A., Gool, L.: Random Forests for Real Time 3D Face Analysis. International Journal of Computer Vision 101(3), 437–458 (2012),

    CrossRef  Google Scholar 

  13. Fanelli, G., Dantone, M., Gall, J., Fossati, A., Gool, L.: Random forests for real time 3d face analysis. International Journal of Computer Vision 101, 437–458 (2013),

    CrossRef  Google Scholar 

  14. Fanelli, G., Weise, T., Gall, J., Van Gool, L.: Real time head pose estimation from consumer depth cameras. In: Mester, R., Felsberg, M. (eds.) DAGM 2011. LNCS, vol. 6835, pp. 101–110. Springer, Heidelberg (2011)

    CrossRef  Google Scholar 

  15. Venturini, F., Marshall, C., Di Alberto, E.: Hearts, minds and wallets winning the battle for consumer trust accenture video-over-internet consumer survey (2012)

    Google Scholar 

  16. Frisson, C., Keyaerts, G., Grisard, F., Dupont, S., Ravet, T., Zajga, F., Colmenares-Guerra, L., Todoroff, T., Dutoit, T.: Mashtacycle: On-stage improvised audio collage by contentbased similarity and gesture recognition. In: 5th International Conference on Intelligent Technologies for Interactive Entertainment, INTETAIN (2013)

    Google Scholar 

  17. Gaschler, A., Jentzsch, S., Giuliani, M., Huth, K., de Ruiter, J., Knoll, A.: Social behavior recognition using body posture and head pose for human-robot interaction. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2128–2133 (October 2012),

  18. Kistler, F.: Fubi- full body interaction framework (2011),

  19. Kistler, F., Endrass, B., Damian, I., Dang, C., Andr, E.: Natural interaction with culturally adaptive virtual characters. Journal on Multimodal User Interfaces, 1–9, , doi:10.1007/s12193-011-0087-z

  20. Kistler, F., Sollfrank, D., Bee, N., André, E.: Full body gestures enhancing a game book for interactive story telling. In: Si, M., Thue, D., André, E., Lester, J.C., Tanenbaum, J., Zammitto, V. (eds.) ICIDS 2011. LNCS, vol. 7069, pp. 207–218. Springer, Heidelberg (2011)

    CrossRef  Google Scholar 

  21. Kuchař, J., Kliegr, T.: Gain: Web service for user tracking and preference learning - a smart tv use case. In: Proceedings of the 7th ACM Conference on Recommender Systems, RecSys 2013, pp. 467–468. ACM, New York (2013),

    CrossRef  Google Scholar 

  22. Leroy, J., Rocca, F., Mancaş, M., Gosselin, B.: 3D head pose estimation for tv setups. In: Mancas, M., d’ Alessandro, N., Siebert, X., Gosselin, B., Valderrama, C., Dutoit, T. (eds.) Intetain. LNICST, vol. 124, pp. 55–64. Springer, Heidelberg (2013)

    CrossRef  Google Scholar 

  23. Leroy, J., Rocca, F., Mancas, M., Gosselin, B.: Second screen interaction: An approach to infer tv watcher’s interest using 3d head pose estimation. In: Proceedings of the 22nd International Conference on World Wide Web Companion, WWW 2013 Companion, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, pp. 465–468 (2013)

    Google Scholar 

  24. Microsoft: Kinect sensor,

  25. Murphy-Chutorian, E., Trivedi, M.M.: Head pose estimation in computer vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(4), 607–626 (2009),

    CrossRef  Google Scholar 

  26. PrimeSense: Capri sensor,

  27. Riche, N., Mancas, M., Duvinage, M., Mibulumukini, M., Gosselin, B., Dutoit, T.: Rare2012: A multi-scale rarity-based saliency detection with its comparative statistical analysis. Signal Processing: Image Communication 28(6), 642–658 (2013),

    Google Scholar 

  28. Vatavu, R.: A comparative study of user-defined handheld vs. freehand gestures for home entertainment environments. Journal of Ambient Intelligence and Smart Environments

    Google Scholar 

  29. Vatavu, R.D.: User-defined gestures for free-hand tv control. In: Proceedings of the 10th European Conference on Interactive Tv and Video, EuroiTV 2012, pp. 45–48. ACM, New York (2012),

    Google Scholar 

  30. Vinciarelli, A., Pantic, M., Bourlard, H.: Social signal processing: Survey of an emerging domain. Image and Vision Computing 27(12), 1743–1759 (2009),

    CrossRef  Google Scholar 

  31. Wobbrock, J.O., Morris, M.R., Wilson, A.D.: User-defined gestures for surface computing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2009, pp. 1083–1092. ACM, New York (2009),

    Google Scholar 

Download references

Author information

Authors and Affiliations


Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 IFIP International Federation for Information Processing

About this paper

Cite this paper

Leroy, J. et al. (2014). KINterestTV - Towards Non–invasive Measure of User Interest While Watching TV. In: Rybarczyk, Y., Cardoso, T., Rosas, J., Camarinha-Matos, L.M. (eds) Innovative and Creative Developments in Multimodal Interaction Systems. eNTERFACE 2013. IFIP Advances in Information and Communication Technology, vol 425. Springer, Berlin, Heidelberg.

Download citation

  • DOI:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-55142-0

  • Online ISBN: 978-3-642-55143-7

  • eBook Packages: Computer ScienceComputer Science (R0)