Advertisement

Cognitive, Affective, & Behavioral Neuroscience

, Volume 13, Issue 3, pp 533–553 | Cite as

Probabilistic models of expectation violation predict psychophysiological emotional responses to live concert music

  • Hauke Egermann
  • Marcus T. Pearce
  • Geraint A. Wiggins
  • Stephen McAdams
Article

Abstract

We present the results of a study testing the often-theorized role of musical expectations in inducing listeners’ emotions in a live flute concert experiment with 50 participants. Using an audience response system developed for this purpose, we measured subjective experience and peripheral psychophysiological changes continuously. To confirm the existence of the link between expectation and emotion, we used a threefold approach. (1) On the basis of an information-theoretic cognitive model, melodic pitch expectations were predicted by analyzing the musical stimuli used (six pieces of solo flute music). (2) A continuous rating scale was used by half of the audience to measure their experience of unexpectedness toward the music heard. (3) Emotional reactions were measured using a multicomponent approach: subjective feeling (valence and arousal rated continuously by the other half of the audience members), expressive behavior (facial EMG), and peripheral arousal (the latter two being measured in all 50 participants). Results confirmed the predicted relationship between high-information-content musical events, the violation of musical expectations (in corresponding ratings), and emotional reactions (psychologically and physiologically). Musical structures leading to expectation reactions were manifested in emotional reactions at different emotion component levels (increases in subjective arousal and autonomic nervous system activations). These results emphasize the role of musical structure in emotion induction, leading to a further understanding of the frequently experienced emotional effects of music.

Keywords

Emotion Music Expectation Statistical learning Computational modeling Psychophysiology 

Notes

Author Notes

H.E. and S.M.’s work was partially funded by the Canadian Social Sciences and Humanities Research Council through a grant to S.M. (#410-2009-2201), as well as S.M.'s Canada Research Chair. The CIRMMT Audience Response System was funded by a grant from the Canada Foundation for Innovation. M.T.P. and G.A.W.’s contribution was funded by EPSRC research grant EP/H01294X, “Information and neural dynamics in the perception of musical structure”. We would like to thank all participants, members of the Music Perception and Cognition Laboratory, and the technical team of the Centre for Interdisciplinary Research in Music Media and Technology for being very supportive in carrying out this study.

References

  1. Abdallah, S. A., & Plumbley, M. D. (2009). Information dynamics: Patterns of expectation and surprise in the perception of music. Connection Science, 21(2), 89–117.CrossRefGoogle Scholar
  2. Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412.CrossRefGoogle Scholar
  3. Bates, D., Maechler, M., & Bolker, B. (2011). lme4: Linear mixed-effects models using S4 classes, R package version 0.999375-39 [Computer Software].Google Scholar
  4. Bharucha, J. J., & Stoeckig, K. (1986). Reaction time and musical expectancy: Priming of chords. Journal of Experimental Psychology. Human Perception and Performance, 12(4), 403–410.PubMedCrossRefGoogle Scholar
  5. Bigand, E., & Pineau, M. (1997). Global context effects on musical expectancy. Perception & Psychophysics, 59(7), 1098–1107.CrossRefGoogle Scholar
  6. Boucsein, W. (2001). Physiologische Grundlagen und Meßmethoden der dermalen Aktivität [Physiological Bases and Measurement Methods for Electrodermal Activity]. In F. Rösler (Ed.), Enzyklopädie der Psychologie, Bereich Psychophysiologie: Vol. 1. Grundlagen und Methoden der Psychophysiologie [Encyclopedia of psychology, area psychophysiology: Vol. 1. Basics and methods of psychophysiology] (pp. 551–623). Hogrefe: Göttingen.Google Scholar
  7. Bradley, M. M., Moulder, M., & Lang, P. J. (2005). When good things go bad: The reflex physiology of defense. Psychological Science, 16, 468–473.PubMedGoogle Scholar
  8. Cacioppo, J. T., Petty, R. E., Losch, M. E., & Kim, H. S. (1986). Electromyographic activity over facial muscle regions can differentiate the valence and intensity of affective reactions. Journal of Personality and Social Psychology, 50(2), 260–268.PubMedCrossRefGoogle Scholar
  9. Cannam, Landone, & Sandler, (2010). Sonic visualiser: An open source application for viewing, analysing, and annotating music audio files [Computer Software]. MM’10, October 25–29, 2010, Firenze, Italy.Google Scholar
  10. Carlsen, J. C. (1981). Some factors which influence melodic expectancy. Psychomusicology, 1, 12–29.CrossRefGoogle Scholar
  11. Carrión, R. E., & Bly, B. M. (2008). The effects of learning on event-related potential correlates of musical expectancy. Psychophysiology, 45(5), 759–775.PubMedCrossRefGoogle Scholar
  12. Castellano, M. A., Bharucha, J. J., & Krumhansl, C. L. (1984). Tonal hierarchies in the music of North India. Journal of Experimental Psychology. General, 113(3), 394–412.PubMedCrossRefGoogle Scholar
  13. Cochrane, T. (2010). Music, emotions and the influence of the cognitive sciences. Philosophy Compass, 11, 978–988.CrossRefGoogle Scholar
  14. Colombetti, G. (2005). Appraising valence. Journal of Consciousness Studies, 12(8), 103–126.Google Scholar
  15. Conklin, D., & Witten, I. H. (1995). Multiple viewpoint systems for music prediction. Journal of New Music Research, 24, 51–73.CrossRefGoogle Scholar
  16. Cuddy, L., & Lunney, C. A. (1995). Expectancies generated by melodic intervals: Perceptual judgments of melodic continuity. Attention, Perception, & Psychophysics, 57(6), 451–462.Google Scholar
  17. Eerola, T. (2004). Data-driven influences on melodic expectancy: Continuations in North Sami Yoiks rated by South African traditional healers. In S. D. Libscomb, R. Ashley, R. O. Gjerdingen, & P. Webster (Eds.), Proceedings of the 8th International Conference on Music Perception & Cognition, Evanston, IL, 2004 (pp. 83–87). Adelaide, Australia: Causal Productions.Google Scholar
  18. Egermann, H., Grewe, O., Kopiez, R., & Altenmüller, E. (2009). Social feedback influences musically induced emotions. The Neurosciences and Music III: Disorders and plasticity: Annals of the New York Academy of Sciences, 1169, 346–350.CrossRefGoogle Scholar
  19. Egermann, H., Nagel, F., Altenmüller, E., & Kopiez, R. (2009). Continuous measurement of musically-induced emotion: A web experiment. International Journal of Internet Science, 4(1), 4–20.Google Scholar
  20. Egermann, H., Sutherland, M. E., Grewe, O., Nagel, F., Kopiez, R., & Altenmüller, E. (2011). Does music listening in a social context alter experience? A physiological and psychological perspective on emotion. Musicae Scientiae, 15(3), 307–323.CrossRefGoogle Scholar
  21. Grewe, O., Kopiez, R., & Altenmueller, E. (2009). The chill parameter: Goose bumps and shivers as promising measures in emotion research. Music Perception, 27(1), 61–74.CrossRefGoogle Scholar
  22. Grewe, O., Nagel, F., Altenmüller, E., & Kopiez, R. (2009–2010). Individual emotional reactions towards music: Evolutionary-based universals? Musicae Scientiae, Special Issue, 261–287.Google Scholar
  23. Grewe, O., Nagel, F., Kopiez, R., & Altenmüller, E. (2007). Emotions over time: Synchronicity and development of subjective, physiological, and facial affective reactions to music. Emotion, 7(4), 774–788.PubMedCrossRefGoogle Scholar
  24. Huron, D. (2006). Sweet anticipation: Music and the psychology of expectation. Cambridge: MIT Press.Google Scholar
  25. Janata, P. (1995). ERP measures assay the degree of expectancy violation of harmonic contexts in music. Journal of Cognitive Neuroscience, 7(2), 153.PubMedCrossRefGoogle Scholar
  26. Juslin, P. N., & Västfjäll, D. (2008). Emotional responses to music: The need to consider underlying mechanisms. The Behavioral and Brain Sciences, 31(5), 559–575. Discussion 575–621.PubMedGoogle Scholar
  27. Kessler, E. J., Hansen, C., & Shepard, R. N. (1984). Tonal schemata in the perception of music in Bali and the West. Music Perception, 2(2), 131–165.CrossRefGoogle Scholar
  28. Kivy, P. (1990). Music alone: Philosophical reflections on the purely musical experience. Ithaca, NY: Cornell University Press.Google Scholar
  29. Koelsch, S., Fritz, T., & Schlaug, G. (2008). Amygdala activity can be modulated by unexpected chord functions during music listening. Neuroreport, 19(18), 1815.PubMedCrossRefGoogle Scholar
  30. Koelsch, S., Kilches, S., Steinbeis, N., & Schelinski, S. (2008). Effects of unexpected chords and of performer's expression on brain responses and electrodermal activity. PLoS One, 3(7), e2631.PubMedCrossRefGoogle Scholar
  31. Konecni, V. J. (2008). Does music induce emotion? A theoretical and methodological analysis. Psychology of Aesthetics, Creativity, and the Arts, 2(2), 115–129.CrossRefGoogle Scholar
  32. Krumhansl, C. L. (1990). Cognitive foundations of musical pitch. Oxford: Oxford University Press.Google Scholar
  33. Krumhansl, C. L. (1996). A perceptual analysis of Mozart’s Piano Sonata K. 282: Segmentation, tension, and musical ideas. Music Perception, 13(3), 401–432.CrossRefGoogle Scholar
  34. Krumhansl, C. L. (2002). Music: A link between cognition and emotion. Current Directions in Psychological Science, 11(2), 45–50.CrossRefGoogle Scholar
  35. Krumhansl, C. L., Louhivuori, J., Toiviainen, P., Järvinen, T., & Eerola, T. (1999). Melodic expectation in Finnish spiritual hymns: Convergence of statistical, behavioral and computational approaches. Music Perception, 17, 151–195.CrossRefGoogle Scholar
  36. Krumhansl, C. L., Toivanen, P., Eerola, T., Toiviainen, P., Jarvinen, T., & Louhivuori, J. (2000). Cross-cultural music cognition: Cognitive methodology applied to North Sami yoiks. Cognition, 76(1), 13–58.PubMedCrossRefGoogle Scholar
  37. Ladinig, O., Honing, H., Háden, G., & Winkler, I. (2009). Probing attentive and preattentive emergent meter in adult listeners without extensive music training. Music Perception, 26(4), 377–386.CrossRefGoogle Scholar
  38. Lang, P. J., Bradley, M. M., & Cuthbert, M. M. (1997). Motivated attention: Affect, activation and action. In P. J. Lang, R. F. Simons, & M. T. Balaban (Eds.), Attention and orienting: Sensory and motivational processes (pp. 97–136). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.Google Scholar
  39. Larson, S. (2004). Musical forces and melodic expectations: Comparing computer models and experimental results. Music Perception, 21(4), 457–498.CrossRefGoogle Scholar
  40. Liljeström, S., Juslin, P. N., & Vastfjall, D. (2012). Experimental evidence of the roles of music choice, social context, and listener personality in emotional reactions to music. Psychology of Music. doi: 10.1177/0305735612440615 Google Scholar
  41. Lundqvist, L.-O., Carlsson, F., Hilmersson, P., & Juslin, P. N. (2008). Emotional responses to music: Experience, expression, and physiology. Psychology of Music, 37(1), 61–90.CrossRefGoogle Scholar
  42. MacKay, D. J. C. (2003). Information theory, inference, and learning algorithms. Cambridge: Cambridge University Press.Google Scholar
  43. Manning, C. D., & Schütze, H. (1999). Foundations of statistical natural language processing. Cambridge: MIT Press.Google Scholar
  44. Manzara, L. C., Witten, I. H., & James, M. (1992). On the entropy of music: An experiment with Bach chorale melodies. Leonardo, 2, 81–88.CrossRefGoogle Scholar
  45. Margulis, E. H. (2005). A model of melodic expectation. Music Perception, 22(4), 663–714.CrossRefGoogle Scholar
  46. Margulis, E. H., & Levine, W. (2006). Timbre priming effects and expectation in melody. Journal of New Music Research, 35(2), 175–182.CrossRefGoogle Scholar
  47. McAdams, S., Vines, B. W., Vieillard, S., Smith, B. K., & Reynolds, R. (2004). Influences of large-scale form on continuous ratings in response to a contemporary piece in a live concert setting. Music Perception, 22(2), 297–350.CrossRefGoogle Scholar
  48. Meyer, L. B. (1956). Emotion and meaning in music. Chicago: University of Chicago Press.Google Scholar
  49. Meyer, L. B. (1957). Meaning in music and information theory. Journal of Aesthetics and Art Criticism, 15(4), 412–424.CrossRefGoogle Scholar
  50. Nagel, F., Kopiez, R., Grewe, O., & Altenmüller, E. (2007). EMuJoy: Software for continuous measurement of perceived emotions in music. Behavior Research Methods, 39(2), 283–290.PubMedCrossRefGoogle Scholar
  51. Narmour, E. (1990). The analysis and cognition of basic melodic structures. Chicago: University of Chicago Press.Google Scholar
  52. Narmour, E. (1992). The analysis and cognition of melodic complexity. Chicago: University of Chicago Press.Google Scholar
  53. Ockelford, A. (2006). Implication and expectation in music: A zygonic model. Psychology of Music, 34(1), 81–142.CrossRefGoogle Scholar
  54. Omigie, D., Pearce, M. T., & Stewart, L. (2012). Tracking of pitch probabilities in congenital amusia. Neuropsychologia, 50, 1483–1493.PubMedCrossRefGoogle Scholar
  55. Oram, N., & Cuddy, L. L. (1995). Responsiveness of Western adults to pitch- distributional information in melodic sequences. Psychological Research, 57(2), 103–118.PubMedGoogle Scholar
  56. Pearce, M.T. (2005). The construction and evaluation of statistical models of melodic structure in music perception and composition. PhD thesis, London, UK: Department of Computing, City University.Google Scholar
  57. Pearce, M. T., Conklin, D., & Wiggins, G. A. (2005). Methods for combining statistical models of music. In U. K. Wiil (Ed.), Computer music modelling and retrieval (pp. 295–312). Berlin: Springer.CrossRefGoogle Scholar
  58. Pearce, M. T., Müllensiefen, D., & Wiggins, G. (2010). The role of expectation and probabilistic learning in auditory boundary perception: A model comparison. Perception, 39(10), 1365–1389.PubMedCrossRefGoogle Scholar
  59. Pearce, M. T., Ruiz, M. H., Kapasi, S., Wiggins, G., & Bhattacharya, J. (2010). Unsupervised statistical learning underpins computational, behavioural, and neural manifestations of musical expectation. NeuroImage, 50(1), 302–313.PubMedCrossRefGoogle Scholar
  60. Pearce, M. T., & Wiggins, G. A. (2004). Improved methods for statistical modelling of monophonic music. Journal of New Music Research, 33(4), 367–385.CrossRefGoogle Scholar
  61. Pearce, M. T., & Wiggins, G. A. (2006). Expectation in melody: The influence of context and learning. Music Perception, 23(5), 377–405.CrossRefGoogle Scholar
  62. Ravaja, N., Turpeinen, M., Saari, T., Puttonen, S., & Keltikangas-Järvinen, L. (2008). The psychophysiology of James Bond: Phasic emotional responses to violent video game events. Emotion, 8(1), 114–120.PubMedCrossRefGoogle Scholar
  63. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178.CrossRefGoogle Scholar
  64. Saffran, J. R., Johnson, E. K., Aslin, R. N., & Newport, E. L. (1999). Statistical learning of tone sequences by human infants and adults. Cognition, 70(1), 27–52.PubMedCrossRefGoogle Scholar
  65. Salimpoor, V. N., Benovoy, M., Larcher, K., Dagher, A., & Zatorre, R. J. (2011). Anatomically distinct dopamine release during anticipation and experience of peak emotion to music. Nature Neuroscience, 14, 257–262.PubMedCrossRefGoogle Scholar
  66. Salimpoor, V. N., Benovoy, M., Longo, G., Cooperstock, J. R., & Zatorre, R. J. (2009). The rewarding aspects of music listening are related to degree of emotional arousal. PloS One, 4(10), e7487.PubMedCrossRefGoogle Scholar
  67. Schellenberg, E. G. (1996). Expectancy in melody: Tests of the implication-realization model. Cognition, 58(1), 75–125.PubMedCrossRefGoogle Scholar
  68. Schellenberg, E. G. (1997). Simplifying the implication-realisation model of melodic expectancy. Music Perception, 14, 295–318.CrossRefGoogle Scholar
  69. Schellenberg, E. G., Adachi, M., Purdy, K. T., & McKinnon, M. C. (2002). Expectancy in melody: Tests of children and adults. Journal of Experimental Psychology. General, 131(4), 511–537.PubMedCrossRefGoogle Scholar
  70. Scherer, K. (2004). Which emotions can be induced by music? What are the underlying mechanisms? And how can we measure them? Journal of New Music Research, 33(3), 239–251.CrossRefGoogle Scholar
  71. Scherer, K. R. (2005). What are emotions? And how can they be measured? Social Science Information, 44(4), 695–729.CrossRefGoogle Scholar
  72. Scherer, K. R., & Zentner, M. R. (2001). Emotional effects of music: Production rules. In P. N. Juslin & J. A. Sloboda (Eds.), Music and emotion: Theory and research (pp. 361–392). Oxford: Oxford University Press.Google Scholar
  73. Schmuckler, M. A., & Boltz, M. (1994). Harmonic and rhythmic influences on musical expectancy. Perception & Psychophysics, 56(3), 313–325.CrossRefGoogle Scholar
  74. Schubert, E. (1999). Measuring emotion continuously: Validity and reliability of the two dimensional emotion space. Australian Journal of Psychology, 51, 154–165.CrossRefGoogle Scholar
  75. Sloboda, J. A. (1991). Music structure and emotional response: Some empirical findings. Psychology of Music, 19(2), 110–120.CrossRefGoogle Scholar
  76. Steinbeis, N., Koelsch, S., & Sloboda, J. A. (2006). The role of harmonic expectancy violations in musical emotions: Evidence from subjective, physiological, and neural responses. Journal of Cognitive Neuroscience, 18(8), 1380–1393.PubMedCrossRefGoogle Scholar
  77. Stevens, C. J., Schubert, E., Morris, R. H., Frear, M., Chen, J., Healey, S., et al. (2009). Cognition and the temporal arts: Investigating audience response to dance using PDAs that record continuous data during live performance. International Journal of Human Computer Studies, 67(9), 800–813.CrossRefGoogle Scholar
  78. Thompson, W. F., & Stainton, M. (1998). Expectancy in Bohemian folk song melodies: Evaluation of implicative principles for implicative and closural intervals. Music Perception, 15, 231–252.CrossRefGoogle Scholar
  79. Tillmann, B., Bharucha, J. J., & Bigand, E. (2000). Implicit learning of tonality: A self-organizing approach. Psychological Review, 107(4), 885–913.PubMedCrossRefGoogle Scholar
  80. Tillmann, B., Bigand, E., & Pineau, M. (1998). Effects of global and local contexts on harmonic expectancy. Music Perception, 16(1), 99–117.CrossRefGoogle Scholar
  81. Tremblay, A. (2011). A suite of functions to back-fit fixed effects and forward-fit random effects, as well as other miscellaneous functions. R package version 1.6 [Computer Software].Google Scholar
  82. Vaitl, D., Vehrs, W., & Sternagel, S. (1993). Prompts—leitmotif—emotion: Play it again, Richard Wagner. In N. Birnbaumer & A. Öhman (Eds.), The structure of emotion: Psychophysiological, cognitive, and clinical aspects (pp. 169–189). Göttingen: Hogrefe & Huber.Google Scholar
  83. West, B. T., Welch, K. B., & Galecki, A. T. (2007). Linear mixed models: A practical guide using statistical software. Boca Raton: Chapman & Hall/CRC Press.Google Scholar
  84. Zanto, T. P., Snyder, J. S., & Large, E. W. (2006). Neural correlates of rhythmic expectancy. Advances in Cognitive Psychology, 2(2), 221–231.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2013

Authors and Affiliations

  • Hauke Egermann
    • 1
    • 2
  • Marcus T. Pearce
    • 3
  • Geraint A. Wiggins
    • 3
  • Stephen McAdams
    • 1
  1. 1.McGill UniversityMontrealCanada
  2. 2.Audio Communication GroupTechnische Universität BerlinBerlinGermany
  3. 3.Queen Mary University of LondonLondonUK

Personalised recommendations