Skip to main content

Probabilistic models of expectation violation predict psychophysiological emotional responses to live concert music

Abstract

We present the results of a study testing the often-theorized role of musical expectations in inducing listeners’ emotions in a live flute concert experiment with 50 participants. Using an audience response system developed for this purpose, we measured subjective experience and peripheral psychophysiological changes continuously. To confirm the existence of the link between expectation and emotion, we used a threefold approach. (1) On the basis of an information-theoretic cognitive model, melodic pitch expectations were predicted by analyzing the musical stimuli used (six pieces of solo flute music). (2) A continuous rating scale was used by half of the audience to measure their experience of unexpectedness toward the music heard. (3) Emotional reactions were measured using a multicomponent approach: subjective feeling (valence and arousal rated continuously by the other half of the audience members), expressive behavior (facial EMG), and peripheral arousal (the latter two being measured in all 50 participants). Results confirmed the predicted relationship between high-information-content musical events, the violation of musical expectations (in corresponding ratings), and emotional reactions (psychologically and physiologically). Musical structures leading to expectation reactions were manifested in emotional reactions at different emotion component levels (increases in subjective arousal and autonomic nervous system activations). These results emphasize the role of musical structure in emotion induction, leading to a further understanding of the frequently experienced emotional effects of music.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Abdallah, S. A., & Plumbley, M. D. (2009). Information dynamics: Patterns of expectation and surprise in the perception of music. Connection Science, 21(2), 89–117.

    Article  Google Scholar 

  2. Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412.

    Article  Google Scholar 

  3. Bates, D., Maechler, M., & Bolker, B. (2011). lme4: Linear mixed-effects models using S4 classes, R package version 0.999375-39 [Computer Software].

  4. Bharucha, J. J., & Stoeckig, K. (1986). Reaction time and musical expectancy: Priming of chords. Journal of Experimental Psychology. Human Perception and Performance, 12(4), 403–410.

    PubMed  Article  Google Scholar 

  5. Bigand, E., & Pineau, M. (1997). Global context effects on musical expectancy. Perception & Psychophysics, 59(7), 1098–1107.

    Article  Google Scholar 

  6. Boucsein, W. (2001). Physiologische Grundlagen und Meßmethoden der dermalen Aktivität [Physiological Bases and Measurement Methods for Electrodermal Activity]. In F. Rösler (Ed.), Enzyklopädie der Psychologie, Bereich Psychophysiologie: Vol. 1. Grundlagen und Methoden der Psychophysiologie [Encyclopedia of psychology, area psychophysiology: Vol. 1. Basics and methods of psychophysiology] (pp. 551–623). Hogrefe: Göttingen.

    Google Scholar 

  7. Bradley, M. M., Moulder, M., & Lang, P. J. (2005). When good things go bad: The reflex physiology of defense. Psychological Science, 16, 468–473.

    PubMed  Google Scholar 

  8. Cacioppo, J. T., Petty, R. E., Losch, M. E., & Kim, H. S. (1986). Electromyographic activity over facial muscle regions can differentiate the valence and intensity of affective reactions. Journal of Personality and Social Psychology, 50(2), 260–268.

    PubMed  Article  Google Scholar 

  9. Cannam, Landone, & Sandler, (2010). Sonic visualiser: An open source application for viewing, analysing, and annotating music audio files [Computer Software]. MM’10, October 25–29, 2010, Firenze, Italy.

  10. Carlsen, J. C. (1981). Some factors which influence melodic expectancy. Psychomusicology, 1, 12–29.

    Article  Google Scholar 

  11. Carrión, R. E., & Bly, B. M. (2008). The effects of learning on event-related potential correlates of musical expectancy. Psychophysiology, 45(5), 759–775.

    PubMed  Article  Google Scholar 

  12. Castellano, M. A., Bharucha, J. J., & Krumhansl, C. L. (1984). Tonal hierarchies in the music of North India. Journal of Experimental Psychology. General, 113(3), 394–412.

    PubMed  Article  Google Scholar 

  13. Cochrane, T. (2010). Music, emotions and the influence of the cognitive sciences. Philosophy Compass, 11, 978–988.

    Article  Google Scholar 

  14. Colombetti, G. (2005). Appraising valence. Journal of Consciousness Studies, 12(8), 103–126.

    Google Scholar 

  15. Conklin, D., & Witten, I. H. (1995). Multiple viewpoint systems for music prediction. Journal of New Music Research, 24, 51–73.

    Article  Google Scholar 

  16. Cuddy, L., & Lunney, C. A. (1995). Expectancies generated by melodic intervals: Perceptual judgments of melodic continuity. Attention, Perception, & Psychophysics, 57(6), 451–462.

    Google Scholar 

  17. Eerola, T. (2004). Data-driven influences on melodic expectancy: Continuations in North Sami Yoiks rated by South African traditional healers. In S. D. Libscomb, R. Ashley, R. O. Gjerdingen, & P. Webster (Eds.), Proceedings of the 8th International Conference on Music Perception & Cognition, Evanston, IL, 2004 (pp. 83–87). Adelaide, Australia: Causal Productions.

    Google Scholar 

  18. Egermann, H., Grewe, O., Kopiez, R., & Altenmüller, E. (2009). Social feedback influences musically induced emotions. The Neurosciences and Music III: Disorders and plasticity: Annals of the New York Academy of Sciences, 1169, 346–350.

    Article  Google Scholar 

  19. Egermann, H., Nagel, F., Altenmüller, E., & Kopiez, R. (2009). Continuous measurement of musically-induced emotion: A web experiment. International Journal of Internet Science, 4(1), 4–20.

    Google Scholar 

  20. Egermann, H., Sutherland, M. E., Grewe, O., Nagel, F., Kopiez, R., & Altenmüller, E. (2011). Does music listening in a social context alter experience? A physiological and psychological perspective on emotion. Musicae Scientiae, 15(3), 307–323.

    Article  Google Scholar 

  21. Grewe, O., Kopiez, R., & Altenmueller, E. (2009). The chill parameter: Goose bumps and shivers as promising measures in emotion research. Music Perception, 27(1), 61–74.

    Article  Google Scholar 

  22. Grewe, O., Nagel, F., Altenmüller, E., & Kopiez, R. (2009–2010). Individual emotional reactions towards music: Evolutionary-based universals? Musicae Scientiae, Special Issue, 261–287.

  23. Grewe, O., Nagel, F., Kopiez, R., & Altenmüller, E. (2007). Emotions over time: Synchronicity and development of subjective, physiological, and facial affective reactions to music. Emotion, 7(4), 774–788.

    PubMed  Article  Google Scholar 

  24. Huron, D. (2006). Sweet anticipation: Music and the psychology of expectation. Cambridge: MIT Press.

    Google Scholar 

  25. Janata, P. (1995). ERP measures assay the degree of expectancy violation of harmonic contexts in music. Journal of Cognitive Neuroscience, 7(2), 153.

    PubMed  Article  Google Scholar 

  26. Juslin, P. N., & Västfjäll, D. (2008). Emotional responses to music: The need to consider underlying mechanisms. The Behavioral and Brain Sciences, 31(5), 559–575. Discussion 575–621.

    PubMed  Google Scholar 

  27. Kessler, E. J., Hansen, C., & Shepard, R. N. (1984). Tonal schemata in the perception of music in Bali and the West. Music Perception, 2(2), 131–165.

    Article  Google Scholar 

  28. Kivy, P. (1990). Music alone: Philosophical reflections on the purely musical experience. Ithaca, NY: Cornell University Press.

    Google Scholar 

  29. Koelsch, S., Fritz, T., & Schlaug, G. (2008). Amygdala activity can be modulated by unexpected chord functions during music listening. Neuroreport, 19(18), 1815.

    PubMed  Article  Google Scholar 

  30. Koelsch, S., Kilches, S., Steinbeis, N., & Schelinski, S. (2008). Effects of unexpected chords and of performer's expression on brain responses and electrodermal activity. PLoS One, 3(7), e2631.

    PubMed  Article  Google Scholar 

  31. Konecni, V. J. (2008). Does music induce emotion? A theoretical and methodological analysis. Psychology of Aesthetics, Creativity, and the Arts, 2(2), 115–129.

    Article  Google Scholar 

  32. Krumhansl, C. L. (1990). Cognitive foundations of musical pitch. Oxford: Oxford University Press.

    Google Scholar 

  33. Krumhansl, C. L. (1996). A perceptual analysis of Mozart’s Piano Sonata K. 282: Segmentation, tension, and musical ideas. Music Perception, 13(3), 401–432.

    Article  Google Scholar 

  34. Krumhansl, C. L. (2002). Music: A link between cognition and emotion. Current Directions in Psychological Science, 11(2), 45–50.

    Article  Google Scholar 

  35. Krumhansl, C. L., Louhivuori, J., Toiviainen, P., Järvinen, T., & Eerola, T. (1999). Melodic expectation in Finnish spiritual hymns: Convergence of statistical, behavioral and computational approaches. Music Perception, 17, 151–195.

    Article  Google Scholar 

  36. Krumhansl, C. L., Toivanen, P., Eerola, T., Toiviainen, P., Jarvinen, T., & Louhivuori, J. (2000). Cross-cultural music cognition: Cognitive methodology applied to North Sami yoiks. Cognition, 76(1), 13–58.

    PubMed  Article  Google Scholar 

  37. Ladinig, O., Honing, H., Háden, G., & Winkler, I. (2009). Probing attentive and preattentive emergent meter in adult listeners without extensive music training. Music Perception, 26(4), 377–386.

    Article  Google Scholar 

  38. Lang, P. J., Bradley, M. M., & Cuthbert, M. M. (1997). Motivated attention: Affect, activation and action. In P. J. Lang, R. F. Simons, & M. T. Balaban (Eds.), Attention and orienting: Sensory and motivational processes (pp. 97–136). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

    Google Scholar 

  39. Larson, S. (2004). Musical forces and melodic expectations: Comparing computer models and experimental results. Music Perception, 21(4), 457–498.

    Article  Google Scholar 

  40. Liljeström, S., Juslin, P. N., & Vastfjall, D. (2012). Experimental evidence of the roles of music choice, social context, and listener personality in emotional reactions to music. Psychology of Music. doi:10.1177/0305735612440615

    Google Scholar 

  41. Lundqvist, L.-O., Carlsson, F., Hilmersson, P., & Juslin, P. N. (2008). Emotional responses to music: Experience, expression, and physiology. Psychology of Music, 37(1), 61–90.

    Article  Google Scholar 

  42. MacKay, D. J. C. (2003). Information theory, inference, and learning algorithms. Cambridge: Cambridge University Press.

    Google Scholar 

  43. Manning, C. D., & Schütze, H. (1999). Foundations of statistical natural language processing. Cambridge: MIT Press.

    Google Scholar 

  44. Manzara, L. C., Witten, I. H., & James, M. (1992). On the entropy of music: An experiment with Bach chorale melodies. Leonardo, 2, 81–88.

    Article  Google Scholar 

  45. Margulis, E. H. (2005). A model of melodic expectation. Music Perception, 22(4), 663–714.

    Article  Google Scholar 

  46. Margulis, E. H., & Levine, W. (2006). Timbre priming effects and expectation in melody. Journal of New Music Research, 35(2), 175–182.

    Article  Google Scholar 

  47. McAdams, S., Vines, B. W., Vieillard, S., Smith, B. K., & Reynolds, R. (2004). Influences of large-scale form on continuous ratings in response to a contemporary piece in a live concert setting. Music Perception, 22(2), 297–350.

    Article  Google Scholar 

  48. Meyer, L. B. (1956). Emotion and meaning in music. Chicago: University of Chicago Press.

    Google Scholar 

  49. Meyer, L. B. (1957). Meaning in music and information theory. Journal of Aesthetics and Art Criticism, 15(4), 412–424.

    Article  Google Scholar 

  50. Nagel, F., Kopiez, R., Grewe, O., & Altenmüller, E. (2007). EMuJoy: Software for continuous measurement of perceived emotions in music. Behavior Research Methods, 39(2), 283–290.

    PubMed  Article  Google Scholar 

  51. Narmour, E. (1990). The analysis and cognition of basic melodic structures. Chicago: University of Chicago Press.

    Google Scholar 

  52. Narmour, E. (1992). The analysis and cognition of melodic complexity. Chicago: University of Chicago Press.

    Google Scholar 

  53. Ockelford, A. (2006). Implication and expectation in music: A zygonic model. Psychology of Music, 34(1), 81–142.

    Article  Google Scholar 

  54. Omigie, D., Pearce, M. T., & Stewart, L. (2012). Tracking of pitch probabilities in congenital amusia. Neuropsychologia, 50, 1483–1493.

    PubMed  Article  Google Scholar 

  55. Oram, N., & Cuddy, L. L. (1995). Responsiveness of Western adults to pitch- distributional information in melodic sequences. Psychological Research, 57(2), 103–118.

    PubMed  Google Scholar 

  56. Pearce, M.T. (2005). The construction and evaluation of statistical models of melodic structure in music perception and composition. PhD thesis, London, UK: Department of Computing, City University.

  57. Pearce, M. T., Conklin, D., & Wiggins, G. A. (2005). Methods for combining statistical models of music. In U. K. Wiil (Ed.), Computer music modelling and retrieval (pp. 295–312). Berlin: Springer.

    Chapter  Google Scholar 

  58. Pearce, M. T., Müllensiefen, D., & Wiggins, G. (2010). The role of expectation and probabilistic learning in auditory boundary perception: A model comparison. Perception, 39(10), 1365–1389.

    PubMed  Article  Google Scholar 

  59. Pearce, M. T., Ruiz, M. H., Kapasi, S., Wiggins, G., & Bhattacharya, J. (2010). Unsupervised statistical learning underpins computational, behavioural, and neural manifestations of musical expectation. NeuroImage, 50(1), 302–313.

    PubMed  Article  Google Scholar 

  60. Pearce, M. T., & Wiggins, G. A. (2004). Improved methods for statistical modelling of monophonic music. Journal of New Music Research, 33(4), 367–385.

    Article  Google Scholar 

  61. Pearce, M. T., & Wiggins, G. A. (2006). Expectation in melody: The influence of context and learning. Music Perception, 23(5), 377–405.

    Article  Google Scholar 

  62. Ravaja, N., Turpeinen, M., Saari, T., Puttonen, S., & Keltikangas-Järvinen, L. (2008). The psychophysiology of James Bond: Phasic emotional responses to violent video game events. Emotion, 8(1), 114–120.

    PubMed  Article  Google Scholar 

  63. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178.

    Article  Google Scholar 

  64. Saffran, J. R., Johnson, E. K., Aslin, R. N., & Newport, E. L. (1999). Statistical learning of tone sequences by human infants and adults. Cognition, 70(1), 27–52.

    PubMed  Article  Google Scholar 

  65. Salimpoor, V. N., Benovoy, M., Larcher, K., Dagher, A., & Zatorre, R. J. (2011). Anatomically distinct dopamine release during anticipation and experience of peak emotion to music. Nature Neuroscience, 14, 257–262.

    PubMed  Article  Google Scholar 

  66. Salimpoor, V. N., Benovoy, M., Longo, G., Cooperstock, J. R., & Zatorre, R. J. (2009). The rewarding aspects of music listening are related to degree of emotional arousal. PloS One, 4(10), e7487.

    PubMed  Article  Google Scholar 

  67. Schellenberg, E. G. (1996). Expectancy in melody: Tests of the implication-realization model. Cognition, 58(1), 75–125.

    PubMed  Article  Google Scholar 

  68. Schellenberg, E. G. (1997). Simplifying the implication-realisation model of melodic expectancy. Music Perception, 14, 295–318.

    Article  Google Scholar 

  69. Schellenberg, E. G., Adachi, M., Purdy, K. T., & McKinnon, M. C. (2002). Expectancy in melody: Tests of children and adults. Journal of Experimental Psychology. General, 131(4), 511–537.

    PubMed  Article  Google Scholar 

  70. Scherer, K. (2004). Which emotions can be induced by music? What are the underlying mechanisms? And how can we measure them? Journal of New Music Research, 33(3), 239–251.

    Article  Google Scholar 

  71. Scherer, K. R. (2005). What are emotions? And how can they be measured? Social Science Information, 44(4), 695–729.

    Article  Google Scholar 

  72. Scherer, K. R., & Zentner, M. R. (2001). Emotional effects of music: Production rules. In P. N. Juslin & J. A. Sloboda (Eds.), Music and emotion: Theory and research (pp. 361–392). Oxford: Oxford University Press.

    Google Scholar 

  73. Schmuckler, M. A., & Boltz, M. (1994). Harmonic and rhythmic influences on musical expectancy. Perception & Psychophysics, 56(3), 313–325.

    Article  Google Scholar 

  74. Schubert, E. (1999). Measuring emotion continuously: Validity and reliability of the two dimensional emotion space. Australian Journal of Psychology, 51, 154–165.

    Article  Google Scholar 

  75. Sloboda, J. A. (1991). Music structure and emotional response: Some empirical findings. Psychology of Music, 19(2), 110–120.

    Article  Google Scholar 

  76. Steinbeis, N., Koelsch, S., & Sloboda, J. A. (2006). The role of harmonic expectancy violations in musical emotions: Evidence from subjective, physiological, and neural responses. Journal of Cognitive Neuroscience, 18(8), 1380–1393.

    PubMed  Article  Google Scholar 

  77. Stevens, C. J., Schubert, E., Morris, R. H., Frear, M., Chen, J., Healey, S., et al. (2009). Cognition and the temporal arts: Investigating audience response to dance using PDAs that record continuous data during live performance. International Journal of Human Computer Studies, 67(9), 800–813.

    Article  Google Scholar 

  78. Thompson, W. F., & Stainton, M. (1998). Expectancy in Bohemian folk song melodies: Evaluation of implicative principles for implicative and closural intervals. Music Perception, 15, 231–252.

    Article  Google Scholar 

  79. Tillmann, B., Bharucha, J. J., & Bigand, E. (2000). Implicit learning of tonality: A self-organizing approach. Psychological Review, 107(4), 885–913.

    PubMed  Article  Google Scholar 

  80. Tillmann, B., Bigand, E., & Pineau, M. (1998). Effects of global and local contexts on harmonic expectancy. Music Perception, 16(1), 99–117.

    Article  Google Scholar 

  81. Tremblay, A. (2011). A suite of functions to back-fit fixed effects and forward-fit random effects, as well as other miscellaneous functions. R package version 1.6 [Computer Software].

  82. Vaitl, D., Vehrs, W., & Sternagel, S. (1993). Prompts—leitmotif—emotion: Play it again, Richard Wagner. In N. Birnbaumer & A. Öhman (Eds.), The structure of emotion: Psychophysiological, cognitive, and clinical aspects (pp. 169–189). Göttingen: Hogrefe & Huber.

    Google Scholar 

  83. West, B. T., Welch, K. B., & Galecki, A. T. (2007). Linear mixed models: A practical guide using statistical software. Boca Raton: Chapman & Hall/CRC Press.

    Google Scholar 

  84. Zanto, T. P., Snyder, J. S., & Large, E. W. (2006). Neural correlates of rhythmic expectancy. Advances in Cognitive Psychology, 2(2), 221–231.

    Article  Google Scholar 

Download references

Author Notes

H.E. and S.M.’s work was partially funded by the Canadian Social Sciences and Humanities Research Council through a grant to S.M. (#410-2009-2201), as well as S.M.'s Canada Research Chair. The CIRMMT Audience Response System was funded by a grant from the Canada Foundation for Innovation. M.T.P. and G.A.W.’s contribution was funded by EPSRC research grant EP/H01294X, “Information and neural dynamics in the perception of musical structure”. We would like to thank all participants, members of the Music Perception and Cognition Laboratory, and the technical team of the Centre for Interdisciplinary Research in Music Media and Technology for being very supportive in carrying out this study.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hauke Egermann.

Appendices

Appendix

figurea

Rating instructions

… By moving your finger from left to right you can indicate how pleasant the music is to you (left = negative and unpleasant; right = positive and pleasant). By moving your finger from top to bottom you can indicate your degree of emotional arousal during listening to the music (top = excited; bottom = calm). You should try to rate what your current emotional state is along both dimensions simultaneously. The position of your finger should reflect at each moment your emotional response to the piece as you are listening. …

… By moving your finger from top to bottom you can indicate how unexpected the music events you are hearing are (top= very unexpected; bottom = very expected). The position of your finger should reflect at each moment the unexpectedness of the events as you are listening. You need to constantly monitor your expectations for every musical event in order to keep your finger at the corresponding position. ….

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Egermann, H., Pearce, M.T., Wiggins, G.A. et al. Probabilistic models of expectation violation predict psychophysiological emotional responses to live concert music. Cogn Affect Behav Neurosci 13, 533–553 (2013). https://doi.org/10.3758/s13415-013-0161-y

Download citation

Keywords

  • Emotion
  • Music
  • Expectation
  • Statistical learning
  • Computational modeling
  • Psychophysiology