Skip to main content

Predictive Modeling of Expressed Emotions in Music Using Pairwise Comparisons

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNISA,volume 7900)

Abstract

We introduce a two-alternative forced-choice (2AFC) experimental paradigm to quantify expressed emotions in music using the arousal and valence (AV) dimensions. A wide range of well-known audio features are investigated for predicting the expressed emotions in music using learning curves and essential baselines. We furthermore investigate the scalability issues of using 2AFC in quantifying emotions expressed in music on large-scale music databases. The possibility of dividing the annotation task between multiple individuals, while pooling individuals’ comparisons is investigated by looking at the subjective differences of ranking emotion in the AV space. We find this to be problematic due to the large variation in subjects’ rankings of excerpts. Finally, solving scalability issues by reducing the number of pairwise comparisons is analyzed. We compare two active learning schemes to selecting comparisons at random by using learning curves. We show that a suitable predictive model of expressed valence in music can be achieved from only 15% of the total number of comparisons when using the Expected Value of Information (EVOI) active learning scheme. For the arousal dimension we require 9% of the total number of comparisons.

Keywords

  • expressed emotion
  • pairwise comparison
  • Gaussian process
  • active learning

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-41248-6_14
  • Chapter length: 25 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-642-41248-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Barthet, M., Fazekas, G., Sandler, M.: Multidisciplinary perspectives on music emotion recognition: Implications for content and context-based models. In: 9th International Symposium on Computer Music Modeling and Retrieval (CMMR) Music and Emotions, pp. 19–22 (June 2012)

    Google Scholar 

  2. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006)

    Google Scholar 

  3. Chaloner, K., Verdinelli, I.: Bayesian experimental design: A review. Statistical Science 10(3), 273–304 (1995)

    MathSciNet  CrossRef  MATH  Google Scholar 

  4. Chu, W., Ghahramani, Z.: Preference learning with Gaussian Processes. In: ICML 2005 - Proceedings of the 22nd International Conference on Machine Learning, pp. 137–144 (2005)

    Google Scholar 

  5. Cover, T., Thomas, J.: Elements of information theory. Wiley (1991)

    Google Scholar 

  6. Cross, I.: The nature of music and its evolution. In: Oxford Handbook of Music Psychology, pp. 3–13. Oxford University Press (2009)

    Google Scholar 

  7. Gelman, A.: Prior distributions for variance parameters in hierarchical models. Bayesian Analysis 1(3), 515–533 (2006)

    MathSciNet  Google Scholar 

  8. Hevner, K.: Experimental studies of the elements of expression in music. American Journal of Psychology 48(2), 246–268 (1936)

    CrossRef  Google Scholar 

  9. Hodges, D.A.: Psychophysiology measures. In: Music and Emotion: Theory, Research, Applications, pp. 279–312. Oxford University Press, New York (2010)

    Google Scholar 

  10. Houlsby, N., Hernandez-Lobato, J.M., Huszar, F., Ghahramani, Z.: Collaborative Gaussian processes for preference learning. In: Bartlett, P., Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 2105–2113 (2012)

    Google Scholar 

  11. Huq, A., Bello, J.P., Rowe, R.: Automated Music Emotion Recognition: A Systematic Evaluation. Journal of New Music Research 39(3), 227–244 (2010)

    CrossRef  Google Scholar 

  12. Juslin, P.N., Vastfjall, D.: Emotional response to music: The need to consider underlying mechanism. Behavioral and Brain Sciences 31, 559–621 (2008)

    Google Scholar 

  13. Juslin, P.N., Sloboda, J.A. (eds.): Music and Emotion: theory, research, applications. Oxford University Press, New York (2010)

    Google Scholar 

  14. Kim, Y., Schmidt, E., Migneco, R., Morton, B., Richardson, P., Scott, J., Speck, J., Turnbull, D.: Music emotion recognition: A state of the art review. In: 11th International Conference on Music Information Retrieval (ISMIR), pp. 255–266 (2010)

    Google Scholar 

  15. Koelsch, S., Siebel, W.A., Fritz, T.: Functional neuroimaging. In: Music and Emotion: Theory, Research, Applications, pp. 313–346. Oxford University Press, New York (2010)

    Google Scholar 

  16. Lartillot, O., Eerola, T., Toiviainen, P., Fornari, J.: Multi-feature modeling of pulse clarity: Design, validation, and optimization. In: 9th International Conference on Music Information Retrieval (ISMIR), pp. 521–526 (2008)

    Google Scholar 

  17. Lartillot, O., Toiviainen, P., Eerola, T.: A matlab toolbox for music information retrieval. In: Preisach, C., Burkhardt, H., Schmidt-Thieme, L., Decker, R. (eds.) Data Analysis, Machine Learning and Applications, Studies in Classification, Data Analysis, and Knowledge Organization. Springer (2008)

    Google Scholar 

  18. Lindley, D.V.: On a measure of the information provided by an experiment. The Annals of Mathematical Statistics 27(4), 986–1005 (1956)

    MathSciNet  CrossRef  MATH  Google Scholar 

  19. Madsen, J.: Modeling of Emotions expressed in Music using Audio features. DTU Informatics, Master Thesis (2011), http://www2.imm.dtu.dk/pubdb/views/publicationtextunderscoredetails.php?id=6036

  20. Madsen, J., Jensen, B.S., Larsen, J., Nielsen, J.B.: Towards predicting expressed emotion in music from pairwise comparisons. In: 9th Sound and Music Computing Conference (SMC) Illusions (July 2012)

    Google Scholar 

  21. Madsen, J., Nielsen, J.B., Jensen, B.S., Larsen, J.: Modeling expressed emotions in music using pairwise comparisons. In: 9th International Symposium on Computer Music Modeling and Retrieval (CMMR) Music and Emotions (June 2012)

    Google Scholar 

  22. Mathieu, B., Essid, S., Fillon, T., Prado, J., Richard, G.: An easy to use and efficient audio feature extraction software. In: 11th International Conference on Music Information Retrieval, ISMIR (2010)

    Google Scholar 

  23. Müller, M., Ewert, S.: Chroma Toolbox: MATLAB implementations for extracting variants of chroma-based audio features. In: 12th International Conference on Music Information Retrieval (ISMIR), Miami, USA (2011)

    Google Scholar 

  24. Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press (2006)

    Google Scholar 

  25. Russell, J.: A circumplex model of affect. Journal of Personality and Social Psychology 39(6), 1161 (1980)

    CrossRef  Google Scholar 

  26. Schubert, E.: Measurement and time series analysis of emotion in music. Ph.D. thesis, University of New South Wales (1999)

    Google Scholar 

  27. Thurstone, L.L.: A law of comparative judgement. Psychological Review 34 (1927)

    Google Scholar 

  28. Train, K.: Discrete Choice Methods with Simulation. Cambridge University Press (2009)

    Google Scholar 

  29. Västfjäll, D.: Indirect perceptual, cognitive, and behavioral measures. In: Music and Emotion: Theory, Research, Applications, pp. 255–278. Oxford University Press, New York (2010)

    Google Scholar 

  30. Yang, Y.H., Chen, H.: Ranking-Based Emotion Recognition for Music Organization and Retrieval. IEEE Transactions on Audio, Speech, and Language Processing 19(4), 762–774 (2011)

    CrossRef  Google Scholar 

  31. Zentner, M., Eerola, T.: Self-report measures and models. In: Music and Emotion: Theory, Research, Applications, pp. 187–222. Oxford University Press, New York (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Madsen, J., Jensen, B.S., Larsen, J. (2013). Predictive Modeling of Expressed Emotions in Music Using Pairwise Comparisons. In: Aramaki, M., Barthet, M., Kronland-Martinet, R., Ystad, S. (eds) From Sounds to Music and Emotions. CMMR 2012. Lecture Notes in Computer Science, vol 7900. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41248-6_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-41248-6_14

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-41247-9

  • Online ISBN: 978-3-642-41248-6

  • eBook Packages: Computer ScienceComputer Science (R0)