Multimedia Tools and Applications

, Volume 78, Issue 23, pp 32967–32998 | Cite as

Learning the harmonic analysis: is visualization an effective approach?

  • Delfina MalandrinoEmail author
  • Donato Pirozzi
  • Rocco Zaccagnino


Understanding the structure of music compositions requires an ability built over time, through the study of the music theory and the application of countless hours of practice. In particular for beginner learners, it can be a time-consuming and a tedious task due to the steep learning curve, especially for classical music. In this work we focus on a specific type of classical music composition, that is music in chorale style. Composing such type of music requires the study of rules that are related to many structural aspects of music, such as melodic and mainly harmonic aspects. To overcome these difficulties, interdisciplinary techniques could be exploited to understand whether extra (visual) information, provided through a specific software tool, could be useful to improve learning in a quick and effective way. We introduce therefore VisualHarmony, a tool that allows users to perform the harmonic analysis of music compositions by exploiting visual clues superimposed on the music scores. Since the harmonic analysis requires to identify similar tonalities and relevant degrees, the visualization approach proposed uses closest colors to represent similar tonalities and degrees. To assess the effectiveness of our idea, we performed an evaluation study involving 60 participants among experts (with a conservatory degree) and music students from conservatory classes. We derived interesting results about the overall learning capabilities (when using visualization in supporting learning) and music information retention when using VisualHarmony in a first phase to study rules, and then move on to the classic way of performing harmonization. This result allowed us to further demonstrate the effectiveness of visualization to learn classic music rules. Finally, we also obtained positive feedback about the system usability and the satisfaction of the users with regard to the easiness and the usefulness of the tested tool.


Music visualization Harmonic analysis visualization Tool Learning Evaluation 



  1. 1.
    Al-Musawi M, Ledesma A, Nieminen H, Korhonen I (2016) Implementation and user testing of a system for visualizing continuous health data and events. In: International conference on biomedical and health informatics (BHI), pp 156–159Google Scholar
  2. 2.
    Bergstrom T, Karahalios K, Hart JC (2007) Isochords: visualizing structure in music. In: Proceedings of graphics interface, pp 297–304Google Scholar
  3. 3.
    Blake A, Stapleton G, Rodgers P, Howse J (2014) How should we use colour in euler diagrams? In: Proceedings of the 7th international symposium on visual information communication and interaction, pp 149:149–149:158Google Scholar
  4. 4.
    Chan W, Qu H, Mak W (2010) Visualizing the semantic structure in classical music works. IEEE Trans Vis Comput Graph 16(1):161–173CrossRefGoogle Scholar
  5. 5.
    Ciuha P, Klemenc B, Solina F (2010) Visualization of concurrent tones in music with colours. In: Proceedings of the international conference on multimedia, pp 1677–1680Google Scholar
  6. 6.
    Conati C, Carenini G, Hoque E, Steichen B, Toker D (2014) Evaluating the impact of user characteristics and different layouts on an interactive visualization for decision making. Comput. Graph Forum, 33(3)CrossRefGoogle Scholar
  7. 7.
    De Felice C, De Prisco R, Malandrino D, Zaccagnino G, Zaccagnino R, Zizza R (2017) Splicing music composition. Inf Sci 385:196–212CrossRefGoogle Scholar
  8. 8.
    De Haas W, Magalhães J, Wiering F, Veltkamp CR (2013) Automatic functional harmonic analysis. Comput Music J 37(4):37–53CrossRefGoogle Scholar
  9. 9.
    De Prisco R, Zaccagnino G, Zaccagnino R (2010) EvoBassComposer: a multi-objective genetic algorithm for 4-voice compositions. In: Proceedings of the 12th annual conference on genetic and evolutionary computation, GECCO ’10, pp 817–818Google Scholar
  10. 10.
    De Prisco R, Zaccagnino G, Zaccagnino R (2011) A multi-objective differential evolution algorithm for 4-voice compositions. In: 2011 IEEE Symposium on differential evolution (SDE), pp 1–8Google Scholar
  11. 11.
    De Prisco R, Lettieri N, Malandrino D, Pirozzi D, Zaccagnino G, Zaccagnino R (2016) Visualization of music plagiarism: analysis and evaluation. In: 20th International conference information visualisation. Lisbon, Portugal, July 19-22, pp 177–182Google Scholar
  12. 12.
    De Prisco R, Malandrino D, Pirozzi D, Zaccagnino G, Zaccagnino R (2017) Understanding the structure of musical compositions: is visualization an effective approach? Inf Vis 16(2):139–152CrossRefGoogle Scholar
  13. 13.
    De Prisco R, Malandrino D, Pirozzi D, Zaccagnino G, Zaccagnino R (2018) Evaluation study of visualisations for harmonic analysis of 4-part music. In: 22th International conference information visualisation, IV’2018 (10-13 July 2018)Google Scholar
  14. 14.
    Doolittle PE, Hicks D (2003) Constructivism as a theoretical foundation for the use of technology in social studies. Theory Res Soc Educ 31(1):72–104CrossRefGoogle Scholar
  15. 15.
    Erra U, Iaccarino G, Malandrino D, Scarano V (2007) Personalizable edge services for Web accessibility. Univ Access Inf Soc 6(3):285–306CrossRefGoogle Scholar
  16. 16.
    Erra U, Malandrino D, Pepe L (2018) A methodological evaluation of natural user interfaces for immersive 3d graph explorations. J Vis Lang Comput 44:13–27CrossRefGoogle Scholar
  17. 17.
    Fonteles JH, Rodrigues MAF, Basso VED (2013) Creating and evaluating a particle system for music visualization. J Vis Lang Comput 24(6):472–482CrossRefGoogle Scholar
  18. 18.
    Iaccarino G, Malandrino D, Scarano V (2006) Personalizable edge services for web accessibility. In: Proceedings of the 2006 international cross-disciplinary workshop on web accessibility (W4A): building the mobile web: rediscovering accessibility?, W4A ’06, pp 23–32Google Scholar
  19. 19.
    Koelle D Music programming for Java and JVM Languages.
  20. 20.
    Kroger P, Passos R, Sampaio M, Cidra GD (2008) Rameau: a system for automatic harmonic analysis. In: Proceedings of the 2008 international computer music conference, pp 273–281Google Scholar
  21. 21.
    Lazar J, Feng JH, Hochheiser H (2010) Research methods in human-computer interactionGoogle Scholar
  22. 22.
    Leon P, Ur B, Shay R, Wang Y, Balebako R, Cranor L (2012) Why Johnny can’t opt out: a usability evaluation of tools to limit online behavioral advertising. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 589–598Google Scholar
  23. 23.
    Lewis JR (1995) IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum-Comput Interact 7(1):57–78CrossRefGoogle Scholar
  24. 24.
    Malandrino D, Manno I, Palmieri G, Scarano V, Tateo L, Casola D, Ferrante I, Foresta F (2015) Tailorable infrastructure to enhance mobile seamless learning. IEEE Trans Learn Technol 8(1):18–30CrossRefGoogle Scholar
  25. 25.
    Malandrino D, Pirozzi D, Zaccagnino G, Zaccagnino R (2015) A color-based visualization approach to understand harmonic structures of musical compositions. In: 19th International conference on information visualisation, IV 2015, Barcelona, Spain, July 22-24, 2015, pp 56–61Google Scholar
  26. 26.
    Malandrino D, Pirozzi D, Zaccagnino R (2018) Visualization and music harmony: design, implementation, and evaluation. In: 22th International conference information visualisation, IV’2018 (10-13 July 2018)Google Scholar
  27. 27.
    Malinowki S The music animation machine — “music worth watching”.
  28. 28.
    Mardirossian A, Chew E (2007) Visualizing music: tonal progressions and distributions. In: Proceedings of the 8th international conference on music information retrieval, pp 189–194Google Scholar
  29. 29.
    Maxwell SE, Delaney HD, Kelley K (2018) Designing experiments and analyzing data: a model comparison perspective, 3rd ed. RoutledgeGoogle Scholar
  30. 30.
    Miyazaki R, Fujishiro I, Hiraga R (2003) Exploring MIDI datasets. In: ACM SIGGRAPH 2003 sketches and applications, pp 1–1Google Scholar
  31. 31.
    Muelder C, Provan T, Ma KL (2010) Content based graph visualization of audio data for music library navigation. In: Int. sym. on multimedia (ISM), pp 129–136Google Scholar
  32. 32.
    Ono JP, Corrêa D, Ferreira M, Mello R, Nonato LG (2015) Similarity graph: visual exploration of song collections. In: SIBGRAPIGoogle Scholar
  33. 33.
    Piston W (1987) Harmony: 5th edn. W. W. Norton & Company IncGoogle Scholar
  34. 34.
    Piston W, DeVoto M (1987) Harmony. NortonGoogle Scholar
  35. 35.
    Prisco RD, Malandrino D, Zaccagnino G, Zaccagnino R, Zizza R (2017) A kind of bio-inspired learning of music style. In: Computational intelligence in music, sound, art and design - 6th international conference, EvoMUSART 2017, Amsterdam, The Netherlands, April 19-21, 2017, Proceedings, pp 97–113Google Scholar
  36. 36.
    Purchase H (2012) Experimental human computer interaction: a practical guide with visual examples. Cambridge University PressGoogle Scholar
  37. 37.
    Ramirez R, Canepa C, Ghisio S, Kolykhalova K, Mancini M, Volta E, Volpe G, Giraldo S, Mayor O, Perez A et al (2018) Enhancing music learning with smart technologies. In: Proceedings of the 5th international conference on movement and computing. ACM, p 49Google Scholar
  38. 38.
    Rentfrow PJ (2012) The role of music in everyday life: current directions in the social psychology of music. Soc Person Psychol Compass 6(5):402–416CrossRefGoogle Scholar
  39. 39.
    Riche NH, Dwyer T (2010) Untangling euler diagrams. IEEE Trans Vis Comput Graph 16(6):1090–1099CrossRefGoogle Scholar
  40. 40.
    Roediger HL, Butler AC (2011) The critical role of retrieval practice in long-term retention. Trends Cogn Sci 15(1):20–27CrossRefGoogle Scholar
  41. 41.
    Roediger HL, Karpicke JD (2006) The power of testing memory: basic research and implications for educational practice. Perspect Psychol Sci 1(3):181–210CrossRefGoogle Scholar
  42. 42.
    Sapp CS (2005) Visual hierarchical key analysis. Comput Entertain 3(4):1–19CrossRefGoogle Scholar
  43. 43.
    Shapiro SS, Wilk MB (1965) An analysis of variance test for normality (complete samples). Biometrika 52(3/4):591–611MathSciNetCrossRefGoogle Scholar
  44. 44.
    Sidorov KA, Jones A, Marshall AD (2014) Music analysis as a smallest grammar problem. In: Proceedings of the 15th international society for music information retrieval conference, ISMIR 2014, Taipei, Taiwan, October 27-31, pp 301–306Google Scholar
  45. 45.
    Smith S, Williams G (1997) A visualization of music. In: Proceedings of the 8th conference on visualization, pp 499–503Google Scholar
  46. 46.
    Snydal J, Hearst M (2005) ImproViz: visual explorations of jazz improvisations. In: Human factors in computing systems, CHI EA ’05, pp 1805–1808Google Scholar
  47. 47.
    Soriano A, Paulovich F, Nonato LG, Oliveira MCF (2014) Visualization of music collections based on structural content similarity. In: 2014 27th SIBGRAPI conference on graphics, patterns and images, pp 25–32Google Scholar
  48. 48.
    Van Der Linden J, Schoonderwaldt E, Bird J, Johnson R (2011) Musicjacket—combining motion capture and vibrotactile feedback to teach violin bowing. IEEE Trans Instrum Meas 60(1):104–113CrossRefGoogle Scholar
  49. 49.
    Wattenberg M (2002) Arc diagrams: visualizing structure in strings. In: IEEE Symposium on information visualization (INFOVIS), pp 110–116Google Scholar
  50. 50.
    Wattenberg M The shape of song.

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Dipartimento di InformaticaUniversità di SalernoFisciano (SA)Italy

Personalised recommendations