Advertisement

Sonifying Twitter’s Emotions Through Music

  • Mariana SeiçaEmail author
  • Rui (Buga) Lopes
  • Pedro Martins
  • F. Amílcar Cardoso
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11265)

Abstract

Sonification is a scientific field that seeks to explore the potential of sound as an instrument to convey and interpret data. Its techniques have been developing significantly with the growth of technology and supporting hardware and software, which have spread in our daily environment. This allowed the establishment of new communication tools to share information, opinion and feelings as part of our daily routine.

The aim of this project was to unite the social media phenomena with sonification, using Twitter data to extract user’s emotions and translate them into musical compositions. The focus was to explore the potential of music in translating data as personal and subjective as human emotions, developing a musically complex and captivating mapping based on the rules of Western Music. The music is accompanied by a simple visualization, which results in emotions being heard and seen with the corresponding tweets, in a multimodal experience that represents Twitter’s emotional reality. The mapping was tested through an online survey, and despite a few misunderstandings, the results were generally positive, expressing the efficiency and impact of the developed system.

Keywords

Musical sonification Emotion detection Twitter Algorithmic composition Sound design 

References

  1. 1.
    Aldwell, E., Schachter, C.: Harmony & Voice Leading. Thomson-Schirmer, Belmont (2003)Google Scholar
  2. 2.
    Bulley, J., Jones, D.: Living Symphonies (2014). http://www.livingsymphonies.com/
  3. 3.
    Coelho, A., Martins, P., Cardoso, A.: A Portuguese Epopee seen through sound. In: 5th Conference on Computation, Communication, Aesthetics & X (2017)Google Scholar
  4. 4.
    Foo, B.: Rhapsody in grey (2016). https://datadrivendj.com/tracks/brain
  5. 5.
    Gabrielsson, A., Lindström, E.: The role of structure in the musical expression of emotions. In: Juslin, P.N., Sloboda, J. (eds.) Handbook of Music and Emotion: Theory, Research, Applications. Oxford University Press, New York (2010)CrossRefGoogle Scholar
  6. 6.
    Han, B., Cook, P., Baldwin, T.: Automatically constructing a normalisation dictionary for microblogs. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Jeju Island, Korea (2012)Google Scholar
  7. 7.
    Harris, J., Kamvar, S.: We Feel Fine (2006). http://number27.org/wefeelfine
  8. 8.
    Hermann, T., Hunt, A., Neuhoff, J.G.: The Sonification Handbook. Logos Verlag, Berlin (2011)Google Scholar
  9. 9.
    Hermann, T., Nehls, A.V., Eitel, F., Barri, T., Gammenl, M.: Tweetscapes - Real-time sonification of Twitter data stream for radio broadcasting. In: Proceedings of the 18th International Conference on Auditory Display, Atlanta, GA, USA (2012)Google Scholar
  10. 10.
    Hevner, K.: Experimental studies of the elements of expression in music. Am. J. Psychol. 48(2), 246–268 (1936)CrossRefGoogle Scholar
  11. 11.
    Juslin, P., Västfjäll, D.: Emotional responses to music: the need to consider underlying mechanisms. Behav. Brain Sci. 31, 559–621 (2008)CrossRefGoogle Scholar
  12. 12.
    Kramer, G., et al.: Sonification report: status of the field and research agenda. Faculty Publications, Department of Psychology. Paper 444 (2010). http://digitalcommons.unl.edu/psychfacpub/444
  13. 13.
    Ligon, B.: Jazz Theory Resources: Tonal, Harmonic, Melodic and Rhythmic Organization of Jazz. Houston Publishing (2003)Google Scholar
  14. 14.
    Liu, F., Weng, F., Jiang, X.: A broad-coverage normalization system for social media language. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, pp. 1035–1044 (2012)Google Scholar
  15. 15.
    Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S.J., McClosky, D.: The Stanford CoreNLP natural language processing toolkit. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60 (2014)Google Scholar
  16. 16.
    Miller, G.M.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)CrossRefGoogle Scholar
  17. 17.
    Mohammad, S.M., Kiritchenjko, S.: Using hashtags to capture fine emotion categories from tweets. Comput. Intell. 31(2), 301–326 (2013)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Morreale, F., Miniukovich, A., De Angeli, A.: TwitterRadio: Translating Tweets into Music. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto, ON, Canada (2014)Google Scholar
  19. 19.
    Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Retr. 2(1–2), 1–135 (2008)CrossRefGoogle Scholar
  20. 20.
    Plutchik, R.: The nature of emotions. Am. Scientist. 89, 344–350 (2001)CrossRefGoogle Scholar
  21. 21.
    Quinn, M.: Research set to music: the climate symphony and other sonifications of ice core, radar, DNA, seismic and solar wind data. In: Proceedings of the International Conference on Auditory Display, Espoo, Finland (2001)Google Scholar
  22. 22.
    Reynolds, C.W.: Flocks, herds and schools: a distributed behavioral model. In: Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pp. 25–34 (1987)Google Scholar
  23. 23.
    Russell, J.: A circumplex model of affect. J. Pers. Soc. Psychology. 39(6), 1161–1178 (1980)CrossRefGoogle Scholar
  24. 24.
    Song, Y., Dixon, S., Pearce, M.T., Halpern, A.R.: Perceived and induced emotion responses to popular music: categorical and dimensional models. Music. Percept.: Interdiscip. J. 33(4), 472–492 (2016)CrossRefGoogle Scholar
  25. 25.
  26. 26.
    Waite, B.: Modern jazz piano: a study in harmony and improvisation. Wise-Publications (Book Sales Limited) (1987)Google Scholar
  27. 27.
    Winters, R.M., Gresham-Lancaster, S.: Sonification of Emotion (2015). https://creativedisturbance.org/podcast/mike-winters-sonification-of-emotion/
  28. 28.
    Winters, R.M., Hattwick, I., Wanderley, M.M.: Emotional data in music performance: two environments for the emotional imaging composer. In: Proceedings of the 3rd International Conference of Music & Emotion (ICM3), Jyväskylä, Finalnd (2013)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Mariana Seiça
    • 1
    Email author
  • Rui (Buga) Lopes
    • 2
  • Pedro Martins
    • 1
  • F. Amílcar Cardoso
    • 1
  1. 1.CISUC, Informatics Engineering DepartmentUniversity of Coimbra, DEICoimbraPortugal
  2. 2.CoimbraPortugal

Personalised recommendations