Skip to main content
SpringerLink
Log in
Book cover

Social Media Retrieval pp 217–237Cite as

Highlight Detection in Movie Scenes Through Inter-users, Physiological Linkage

  • Chapter
  • First Online:

Part of the book series: Computer Communications and Networks ((CCN))

Abstract

Automatic summarization techniques facilitate multimedia indexing and access by reducing the content of a given item to its essential parts. However, novel approaches for summarization should be developed since existing methods cannot offer a general and unobtrusive solution. Considering that the consumption of multimedia data is more and more social, we propose to use a physiological index of social interaction, namely, physiological linkage, to determine general highlights of videos. The proposed method detects highlights which are relevant to the majority of viewers without requiring them any conscious effort. Experimental testing has demonstrated the validity of the proposed system which obtained a classification accuracy of up to 78.2%.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Pantic, M., Vinciarelli, A.: Implicit human centered tagging. IEEE Signal Process. Mag. 26, 11 (2009)

    Article  Google Scholar 

  2. Gross, J.J., Levenson, R.W.: Emotion elicitation using films. Cognit. Emot. 9(1), 87–108 (1995)

    Article  Google Scholar 

  3. Hamann, S.: Cognitive and neural mechanisms of emotional memory. Trends Cognit. Sci. 5(9), 394–400 (2001)

    Article  Google Scholar 

  4. Picard, R.W.: Affective computing. M.I.T media laboratory perceptual computing section technical report, 321 (1995)

    Google Scholar 

  5. Zeng, Z.H., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2009)

    Article  Google Scholar 

  6. Chanel, G., Kierkels, J.J.M., Soleymani, M., Pun, T.: Short-term emotion assessment in a recall paradigm. Int. J. Hum. Comput. Stud. 67(8), 607–627 (2009)

    Article  Google Scholar 

  7. Pantic, M., Rothkrantz, L.J.M.: Toward an affect-sensitive multimodal human-computer interaction. Proc. IEEE 91(9), 1370–1390 (2003)

    Article  Google Scholar 

  8. Gottman, J.M.: Detecting cyclicity in social-interaction. Psychol. Bull. 86(2), 338–348 (1979)

    Article  Google Scholar 

  9. Money, A.G., Agius, H.: Analysing user physiological responses for affective video summarisation. Displays 30(2), 59–70 (2009)

    Article  Google Scholar 

  10. Levenson, R.W., Gottman, J.M.: Marital interaction: physiological linkage and affective exchange. J. Personal. Soc. Psychol. 45(3), 587–597 (1983)

    Article  Google Scholar 

  11. Golder, S.A., Huberman, B.A.: Usage patterns of collaborative tagging systems. J. Inf. Sci. 32, 198–208 (2006)

    Article  Google Scholar 

  12. Golder, S., Huberman, B.A.: The structure of collaborative tagging systems (2005)

    Google Scholar 

  13. Nov, O., Naaman, M., Ye, C.: What drives content tagging: the case of photos on flickr. In: Proceedings of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems, CHI ’08, pp. 1097–1100. ACM, New York(2008)

    Google Scholar 

  14. Ames, M., Naaman, M.: Why we tag: motivations for annotation in mobile and online media. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’07, pp. 971–980. ACM, New York (2007)

    Google Scholar 

  15. Jiao, J., Pantic, M.: Implicit image tagging via facial information. In: Proceedings of the 2nd International Workshop on Social Signal Processing, SSPW ’10, pp. 59–64. ACM, New York (2010)

    Google Scholar 

  16. Koelstra, S., Mühl, C., Patras, I.: Eeg analysis for implicit tagging of video data. In: Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, ACII 2009, pp. 27–32. IEEE, Los Alamitos (2009)

    Google Scholar 

  17. Yazdani, A., Lee, J.S., Ebrahimi, T.: Implicit emotional tagging of multimedia using eeg signals and brain computer interface. In: Proceedings of the First SIGMM Workshop on Social Media, WSM ’09, pp. 81–88. ACM, New York (2009)

    Google Scholar 

  18. Soleymani, M., Chanel, G., Kierkels, J.J.M., Pun, T.: Affective characterization of movie scenes based on multimedia content analysis and user’s physiological emotional responses. In: Proceedings of the Tenth IEEE International Symposium on Multimedia, vol. 5, pp. 228–235. IEEE, Piscataway (2008)

    Google Scholar 

  19. Turing, A.M.: Computing Machinery and Intelligence, pp. 11–35. MIT, Cambridge (1995)

    Google Scholar 

  20. Lisetti, C., Lerouge, C.: Affective computing and tele-home health. In: Proceedings of the 37th Hawaii International Conference on System Sciences, pp. 148–155. Los IEEE, Alamitos (2004)

    Google Scholar 

  21. Paiva, A., Prada, R., Chaves, R., Vala, M., Bullock, A., Andersson, G., Höök, K.: Towards tangibility in gameplay: building a tangible affective interface for a computer game. In: Proceedings of the 5th International Conference on Multimodal Interfaces, ICMI ’03, pp. 60–67. ACM, New York (2003)

    Google Scholar 

  22. Chanel, G., Rebetez, C., Betrancourt, M., Pun, T.: Emotion assessment from physiological signals for adaptation of games difficulty. IEEE Trans. Syst. Man Cybern. Part A 41(6), 1052–1063 (2011)

    Article  Google Scholar 

  23. Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23, 1175–1191 (2001)

    Article  Google Scholar 

  24. Kim, K., Bang, S., Kim, S.: Emotion recognition system using short-term monitoring of physiological signals. Med. Biol. Eng. Comput. 42, 419–427 (2004). doi:10.1007/BF02344719

    Article  Google Scholar 

  25. Scheirer, J., Fernandez, R., Klein, J., Picard, R.W.: Frustrating the user on purpose: a step toward building an affective computer. Interact. Comput. 14(2), 93–118 (2002)

    Article  Google Scholar 

  26. Katsis, C.D., Katertsidis, N., Ganiatras, G., Fotiadis, D.I.: Toward emotion recognition in car racing drivers: a biosignal processing approach. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 38(3), 502–512 (2008)

    Article  Google Scholar 

  27. Rani, P., Liu, C., Sarkar, N.: An empirical study of machine learning techniques for affect recognition in human-robot interaction. Pattern Anal. Appl. 9, 58–69 (2006)

    Article  Google Scholar 

  28. Varni, G., Volpe, G., Camurri, A.: A system for real-time multimodal analysis of nonverbal affective social interaction in user-centric media. IEEE Trans. Multimed. 12(6), 576–590 (2010)

    Article  Google Scholar 

  29. Hatfield, E., Cacioppo, J.T., Rapson, R.L.: Emotional contagion. Curr. Dir. Psychol. Sci. 2, 96–100 (1993)

    Article  Google Scholar 

  30. Jakobs, E., Fischer, A.H., Manstead, A.S.R.: Emotional experience as a function of social context: the role of the other. J. Nonverbal Behav. 21(2), 103–130 (1997)

    Article  Google Scholar 

  31. Pentland, A.: Social signal processing [exploratory dsp]. Signal Process. Mag. IEEE 24(4), 108–111 (2007)

    Article  Google Scholar 

  32. Vinciarelli, A., Pantic, M., Bourlard, H.: Social signal processing: survey of an emerging domain. Image Vis. Comput. 27(12), 1743–1759 (2009). Visual and multimodal analysis of human spontaneous behaviour

    Google Scholar 

  33. Ekman, I., Chanel, G., Kivikangas, J.M., Salminen, M., Järvelä, S., Ravaja, N.: Social interaction in games: measuring physiological linkage and social presence. Simul. Gaming (2011, in press)

    Google Scholar 

  34. Ekman, P., Levenson, R.W., Friesen, W.V.: Autonomic nervous-system activity distinguishes among emotions. Science 221(4616), 1208–1210 (1983)

    Article  Google Scholar 

  35. Coplan, A.: Catching characters’ emotions: emotional contagion responses to narrative fiction film. Film Stud. 8, 26–38 (2006)

    Google Scholar 

  36. Stam, C.J.: Nonlinear dynamical analysis of EEG and MEG: review of an emerging field. Clin. Neurophysiol. 116(10), 2266–2301 (2005)

    Article  Google Scholar 

  37. Levenson, R.W., Gottman, J.M.: Marital interaction: physiological linkage and affective exchange. J. Personal. Soc. Psychol. 45(3), 587–597 (1983)

    Article  Google Scholar 

  38. Levenson, R.W., Ruef, A.M.: Empathy: a physiological substrate. J. Personal. Soc. Psychol. 63(2), 234–246 (1992)

    Article  Google Scholar 

  39. Henning, R.A., Boucsein, W., Gil, M.C.: Social-physiological compliance as a determinant of team performance. Int. J. Psychophysiol. Off. J. Int. Organ. Psychophysiol. 40(3), 221–232 (2001)

    Article  Google Scholar 

  40. Henning, R.A., Armstead, A.G., Ferris, J.K.: Social psychophysiological compliance in a four-person research team. Appl. Ergon. 40(6), 1004–1010 (2009)

    Article  Google Scholar 

  41. Dumas, G., Nadel, J., Soussignan, R., Martinerie, J., Garnero, L.: Inter-brain synchronization during social interaction. PloS one 5(8), 1–10 (2010)

    Google Scholar 

  42. Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., Malach, R.: Intersubject synchronization of cortical activity during natural vision. Science (New York) 303(5664), 1634–1640 (2004)

    Google Scholar 

  43. Hasson, U., Ghazanfar, A.A., Galantucci, B., Garrod, S., Keysers, C.: Brain-to-brain coupling: a mechanism for creating and sharing a social world. Trends in Cognit. Sci. 16, 114–121 (2012)

    Article  Google Scholar 

  44. Ngo, C.-W., Ma, Y.-F., Zhang, H.-J.: Automatic video summarization by graph modeling. IEEE Int. Conf. Comput. Vis. 1, 104 (2003)

    Google Scholar 

  45. Rasheed, Z., Shah, M.: Detection and representation of scenes in videos. IEEE Trans. Multimed. 7, 1097–1105 (2005)

    Article  Google Scholar 

  46. Money, A.G., Agius, H.: Video summarisation: a conceptual framework and survey of the state of the art. J. Vis. Commun. Image Represent. 19(2), 121–143 (2008)

    Article  Google Scholar 

  47. Money, A.G., Agius, H.: Elvis: entertainment-led video summaries. ACM Trans. Multimed. Comput. Commun. Appl. 6, 17:1–17:30 (2010)

    Google Scholar 

  48. Ekin, A., Tekalp, A.M., Mehrotra, R.: Automatic soccer video analysis and summarization. IEEE Trans. Image Process. 12(7), 796–807 (2003)

    Article  Google Scholar 

  49. Wang, J., Xu, C., Chng, E., Tian, Q.: Sports highlight detection from keyword sequences using hmm. In: Proceedings of the IEEE ICME, Taipei, pp. 27–30 (2004)

    Google Scholar 

  50. Li, J., Wang, T., Hu, W., Sun, M., Zhang, Y.: Soccer highlight detection using two-dependence bayesian network. IEEE Int. Conf. Multimed. Expo 0, 1625–1628 (2006)

    Google Scholar 

  51. Joho, H., Staiano, J., Sebe, N., Jose, J.M.: Looking at the viewer: analysing facial activity to detect personal highlights of multimedia contents. Multimed. Tools Appl. 51, 505–523 (2011)

    Article  Google Scholar 

  52. Aizawa, K., Tancharoen, D., Kawasaki, S., Yamasaki, T.: Efficient retrieval of life log based on context and content. In: Proceedings of the the 1st ACM workshop on Continuous Archival and Retrieval of Personal Experiences, CARPE’04, pp. 22–31. ACM, New York (2004)

    Google Scholar 

  53. Jaimes, A., Jaimes, R., Echigo, T., Teraguchi, M., Satoh, F.: Learning personalized video highlights from detailed mpeg-7 metadata. In: MPEG-7 Metadata, in Proceedings of the ICIP, p. 2002. IEEE, Piscataway (2002)

    Google Scholar 

  54. Takahashi, Y., Nitta, N., Babaguchi, N.: Video summarization for large sports video archives. In: Proceedings of the IEEE International Conference on Multimedia and Expo, ICME 2005, Amsterdam, pp. 1170–1173 (2005)

    Google Scholar 

  55. Soleymani, M., Chanel, G., Kierkels, J.J.M., Pun, T.: Affective characterization of movie scenes based on multimedia content analysis and user’s physiological emotional responses. In: Proceedings of the 2008 Tenth IEEE International Symposium on Multimedia, ISM ’08, pp. 228–235. IEEE, Washington, DC (2008)

    Google Scholar 

  56. Fridlund, A.J., Cacioppo, J.T.: Guidelines for human electromyographic research. Psychophysiology 23(5), 567–589 (1986)

    Article  Google Scholar 

  57. Cacioppo, J.T., Tassinary, L.G., Berntson, G.G.: Handbook of Psychophysiology. Cambridge University Press, Cambridge/New York (2000)

    Google Scholar 

  58. Dawson, M.E., Schell, A.M., Filion, D.L.: The electrodermal response system. In: Cacioppo, J.T., Tassinary, L.G. (eds.) Principles of Psychophysiology: Physical, Social and Inferential Elements, pp. 295–324. Cambridge University Press, Cambridge (1990)

    Google Scholar 

  59. Joho, H., Jose, J.M., Valenti, R., Sebe, N.: Exploiting facial expressions for affective video summarisation. In: Proceedings of the ACM International Conference on Image and Video Retrieval, CIVR ’09, pp. 31:1–31:8. ACM, New York (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, Computer Vision and Multimedia Laboratory (CVML), Geneva, Switzerland

    Christophe Chênes, Mohammad Soleymani & Thierry Pun

  2. University of Geneva, Battelle Campus, Building A, 7 route de Drize, CH-1227, Carouge, Geneva, Switzerland

    Christophe Chênes, Mohammad Soleymani & Thierry Pun

  3. CVML and Swiss Center for Affective Sciences, 7 rue des Battoirs, CH-1205, Geneva, Switzerland

    Guillaume Chanel

Authors
  1. Christophe Chênes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guillaume Chanel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Soleymani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thierry Pun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christophe Chênes .

Editor information

Editors and Affiliations

  1. Department of Electronic Engineering, Queen Mary University of London, Mile End Road, London, E1 4NS, United Kingdom

    Naeem Ramzan

  2. Director of Product Innovation, Search, Netflix, Winchester Circle 100, Los Gatos, 95032, California, USA

    Roelof van Zwol

  3. School of Integrated Technology, Yonsei University, 162-1 Songdo-dong, Yeonsu-gu, Incheon, 406-840, Korea, Republic of (South Korea)

    Jong-Seok Lee

  4. Institut für Telekommunikationssysteme, Technische Universität Berlin, Einsteinufer 17, Berlin, 10587, Germany

    Kai Clüver

  5. Media Computing Group, Microsoft Research, 555 108th Ave NE, Bellevue, 98004, Washington, USA

    Xian-Sheng Hua

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag London

About this chapter

Cite this chapter

Chênes, C., Chanel, G., Soleymani, M., Pun, T. (2013). Highlight Detection in Movie Scenes Through Inter-users, Physiological Linkage. In: Ramzan, N., van Zwol, R., Lee, JS., Clüver, K., Hua, XS. (eds) Social Media Retrieval. Computer Communications and Networks. Springer, London. https://doi.org/10.1007/978-1-4471-4555-4_10

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-4555-4_10

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-4554-7

  • Online ISBN: 978-1-4471-4555-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation