Interactive Melodic Analysis

  • David RizoEmail author
  • Plácido R. Illescas
  • José M. Iñesta


In a harmonic analysis task, melodic analysis determines the importance and role of each note in a particular harmonic context. Thus, a note is classified as a harmonic tone when it belongs to the underlying chord, and as a non-harmonic tone otherwise, with a number of categories in this latter case. Automatic systems for fully solving this task without errors are still far from being available, so it must be assumed that, in a practical scenario in which the melodic analysis is the system’s final output, the human expert must make corrections to the output in order to achieve the final result. Interactive systems allow for turning the user into a source of high-quality and high-confidence ground-truth data, so online machine learning and interactive pattern recognition provide tools that have proven to be very convenient in this context. Experimental evidence will be presented showing that this seems to be a suitable way to approach melodic analysis.


Tonal Function Music Information Retrieval Music Theory Musical Work Pitch Class 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Barthelemy, J. and Bonardi, A. (2001). Figured bass and tonality recognition. In Proceedings of the Second International Symposium on Music Information Retrieval (ISMIR 2001), pages 129-136, Bloomington, IN.Google Scholar
  2. Chew, E. and Francois, A. (2003). MuSA.RT: Music on the spiral array. Real-time. In Proceedings of the Eleventh ACM International Conference on Multimedia, pages 448-449, New York, NY.Google Scholar
  3. Choi, A. (2011). Jazz harmonic analysis as optimal tonality segmentation. Computer Music Journal, 35(2):49-66.Google Scholar
  4. Chuan, C.-H. and Chew, E. (2007). A hybrid system for automatic generation of style- specific accompaniment. In Proceedings of the 4th International Joint Workshop on Computational Creativity, pages 57-64, London, UK.Google Scholar
  5. Chuan, C.-H. and Chew, E. (2010). Quantifying the benefits of using an interactive decision support tool for creating musical accompaniment in a particular style. In Proceedings of the Eleventh International Society for Music Information Retrieval Conference (ISMIR 2010), pages 471-476, Utrecht, The Netherlands.Google Scholar
  6. Chuan, C.-H. and Chew, E. (2011). Generating and evaluating musical harmonizations that emulate style. Computer Music Journal, 35(4):64-82.Google Scholar
  7. Cohen, W. W. (1995). Fast effective rule induction. In Proceedings of the 12th International Conference on Machine Learning (ICML), pages 115-123.Google Scholar
  8. de Haas, W. B. (2012). Music information retrieval based on tonal harmony. PhD thesis, Utrecht University.Google Scholar
  9. Ebcioğlu, K. (1986). An expert system for chorale harmonization. In Proceedings of the American Association for Artificial Intelligence, AAAI, pages 784-788.Google Scholar
  10. Feng, Y., Chen, K., and Liu, X.-B. (2011). Harmonizing melody with meta-structure of piano accompaniment figure. Journal of Computer Science and Technology, 26(6):1041-1060.Google Scholar
  11. Forgács, R. (2007). Gallus Dressler’s Praecepta Musicae Poeticae. University of Illinois Press.Google Scholar
  12. Forte, A. (1967). Music and computing: The present situation. In AFIPS Proceedings of the Fall Joint Computer Conference, pages 327-329, Anaheim, CA.Google Scholar
  13. Granroth-Wilding, M. (2013). Harmonic analysis of music using combinatory categorial grammar. PhD thesis, University of Edinburgh, UK.Google Scholar
  14. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. (2009). The WEKA data mining software: an update. SIGKDD Explorations, 11(1):10-18.Google Scholar
  15. Hoffman, T. and Birmingham, W. (2000). A constraint satisfaction approach to tonal harmonic analysis. Technical report, Electrical Engineering and Computer Science Department, University of Michigan.Google Scholar
  16. Hulten, G., Spencer, L., and Domingos, P. (2001). Mining time-changing data streams. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 97-106. ACM Press.Google Scholar
  17. Illescas, P. R., Rizo, D., and Iñesta, J. M. (2007). Harmonic, melodic, and functional automatic analysis. In Proceedings of the 2007 International Computer Music Conference (ICMC), pages 165-168, Copenhagen, Denmark.Google Scholar
  18. Illescas, P. R., Rizo, D., and Iñesta, J. M. (2008). Learning to analyse tonal music. In Proceedings of the International Workshop on Machine Learning and Music (MML 2008), pages 25-26, Helsinki, Finland.Google Scholar
  19. Illescas, P. R., Rizo, D., Iñesta, J. M., and Ramírez, R. (2011). Learning melodic analysis rules. In Proceedings of the International Workshop on Music and Machine Learning (MML 2011). Google Scholar
  20. Iñesta, J. and Pérez-Sancho, C. (2013). Interactive multimodal music transcription. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2013), pages 211-215, Vancouver, Canada.Google Scholar
  21. Kaliakatsos-Papakostas, M. (2014). Probabilistic harmonisation with fixed intermediate chord constraints. In Proceedings of the 11th Sound and Music Computing Conference (SMC) and 40th International Computer Music Conference (ICMC), Athens, Greece.Google Scholar
  22. Kirlin, P. B. (2009). Using harmonic and melodic analyses to automate the initial stages of Schenkerian analysis. In Proceedings of the 10th International Conference on Music Information Retrieval (ISMIR 2009), pages 423-428, Kobe, Japan.Google Scholar
  23. Kröger, P., Passos, A., and Sampaio, M. (2010). A survey of automated harmonic analysis techniques. Technical Report 1, Universidade Federal da Bahia.Google Scholar
  24. Lidy, T., Rauber, A., Pertusa, A., and Iñesta, J. M. (2007). Improving genre classification by combination of audio and symbolic descriptors using a transcription system. InProceedings of the 8th International Conference on Music Information Retrieval (ISMIR 2007), pages 61-66, Vienna, Austria. Google Scholar
  25. Margineantu, D. D. and Dietterich, T. G. (2003). Improved class probability estimates from decision tree models. In Nonlinear Estimation and Classification, pages 173-188. Springer.Google Scholar
  26. Marsden, A. (2010). Schenkerian analysis by computer: A proof of concept. Journal of New Music Research, 39(3):269-289.Google Scholar
  27. Martin, J. G. (1972). Rhythmic (hierarchical) versus serial structure in speech and other behavior. Psychological Review, 79(6):487-509.Google Scholar
  28. Maxwell, H. J. (1984). An artificial intelligence approach to computer-implemented analysis of harmony in tonal music. PhD thesis, Indiana University.Google Scholar
  29. Mearns, L. (2013). The computational analysis of harmony in Western art music. PhD thesis, School of Electronic Engineering and Computer Science, Queen Mary University of London.Google Scholar
  30. Meredith, D. (1993). Computer-aided comparison of syntax systems in three piano pieces by Debussy. Contemporary Music Review, 9(1-2):285-304.Google Scholar
  31. Meredith, D. (2007). Computing pitch names in tonal music: A comparative analysis of pitch spelling algorithms. PhD thesis, Oxford University.Google Scholar
  32. Mouton, R. and Pachet, F. (1995). The symbolic vs. numeric controversy in automatic analysis of music. In Proceedings of the Workshop on Artificial Intelligence and Music (IJCAI1995), pages 32-39.Google Scholar
  33. Pachet, F. (1991). A meta-level architecture applied to the analysis of jazz chord sequences. In Proceedings of the International Computer Music Conference, pages 1-4, Montreal, Canada.Google Scholar
  34. Pachet, F. and Roy, P. (2000). Musical harmonization with constraints: A survey. Constraints, 6(1):7-19.Google Scholar
  35. Pardo, B. and Birmingham, W. (2002). Algorithms for chordal analysis. Computer Music Journal, 26(2):27-49.Google Scholar
  36. Pardo, B. and Birmingham, W. P. (2000). Automated partitioning of tonal music. FLAIRS Conference, pages 23-27.Google Scholar
  37. Passos, A., Sampaio, M., Kröger, P., and de Cidra, G. (2009). Functional harmonic analysis and computational musicology in Rameau. In Proceedings of the 12th Brazilian Symposium on Computer Music (SBMC). Google Scholar
  38. Pérez-García, T., Iñesta, J. M., Ponce de León, P. J., and Pertusa, A. (2011). A multimodal music transcription prototype. In Proceedings of ACM International Conference on Multimodal Interaction (ICMI 2011), pages 315-318, Alicante, Spain.Google Scholar
  39. Perez-Sancho, C., Rizo, D., and Iñesta, J. M. (2009). Genre classification using chords and stochastic language models. Connection Science, 21(2,3):145-159.Google Scholar
  40. Phon-Amnuaisuk, S., Smaill, A., and Wiggins, G. (2006). Chorale harmonization: A view from a search control perspective. Journal of New Music Research, 35(4):279- 305.Google Scholar
  41. Piston, W. (1987). Harmony. W. W. Norton & Company. Revised and expanded by Mark DeVoto.Google Scholar
  42. Prather, R. (1996). Harmonic analysis from the computer representation of a musical score. Communications of the ACM, 39(12es):239-255.Google Scholar
  43. Quinlan, J. R. (2014). C4.5: Programs for Machine Learning. Elsevier.Google Scholar
  44. Raczyński, S., Fukayama, S., and Vincent, E. (2013). Melody harmonization with interpolated probabilistic models. Journal of New Music Research, 42(3):223-235.Google Scholar
  45. Radicioni, D. and Esposito, R. (2007). Tonal harmony analysis: A supervised sequential learning approach. In Basili, R. and Pazienza, M. T., editors, AI*IA 2007: Artificial Intelligence and Human-Oriented Computing, volume 4733 of Lecture Notes in Artificial Intelligence, pages 638-649. Springer.Google Scholar
  46. Rameau, J.-P. (1722). Traite de l’harmonie, reduite a sesprincipes naturels. Jean- Baptiste-Christophe Ballard.Google Scholar
  47. Ramirez, R., Perez, A., Kersten, S., Rizo, D., Roman, P., and Iñesta, J. M. (2010). Modeling violin performances using inductive logic programming. Intelligent Data Analysis, 14(5):573-585.Google Scholar
  48. Raphael, C. and Nichols, E. (2008). Training music sequence recognizers with linear dynamic programming. In Proceedings of the International Workshop on Machine Learning and Music (MML), Helsinki, Finland.Google Scholar
  49. Raphael, C. and Stoddard, J. (2004). Functional harmonic analysis using probabilistic models. Computer Music Journal, 28(3):45-52.Google Scholar
  50. Rizo, D. (2010). Symbolic music comparison with tree data structures. PhD thesis, Universidad de Alicante.Google Scholar
  51. Rohrmeier, M. (2007). A generative grammar approach to diatonic harmonic structure. In Proceedings of the 4th Sound and Music Computing Conference, pages 97-100.Google Scholar
  52. Rohrmeier, M. (2011). Towards a generative syntax of tonal harmony. Journal of Mathematics and Music, 5(1):35-53.Google Scholar
  53. Rothgeb, J. E. (1968). Harmonizing the unfigured bass: A computational study. PhD thesis, Yale University.Google Scholar
  54. Sabater, J., Arcos, J., and López de Mántaras, R. (1998). Using rules to support case-based reasoning for harmonizing melodies. In Freuder, E., editor, Multimodal Reasoning: Papers from the 1998 AAAI Spring Symposium (Technical Report, SS-98-04), pages 147-151, Menlo Park, CA. AAAI Press.Google Scholar
  55. Sapp, C. (2007). Computational chord-root identification in symbolic musical data: Rationale, methods, and applications. Computing in Musicology, 15:99-119.Google Scholar
  56. Sapp, C. (2011). Computational methods for the analysis of musical structure. PhD thesis, Stanford University.Google Scholar
  57. Scarborough, D. L., Miller, B. O., and Jones, J. A. (1989). Connectionist models for tonal analysis. Computer Music Journal, 13(3):49.Google Scholar
  58. Scholz, R., Dantas, V., and Ramalho, G. (2005). Automating functional harmonic analysis: The Funchal system. In Seventh IEEE International Symposium on Multimedia, Irvine, CA.Google Scholar
  59. Simon, I., Morris, D., and Basu, S. (2008). MySong: Automatic accompaniment generation for vocal melodies. In CHI ‘08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 725-734, New York, NY.Google Scholar
  60. Suzuki, S. and Kitahara, T. (2014). Four-part harmonization using Bayesian networks: Pros and cons of introducing chord nodes. Journal of New Music Research, 43(3):331-353.Google Scholar
  61. Taube, H. (1999). Automatic tonal analysis: Toward the implementation of a music theory workbench. Computer Music Journal, 23(4):18-32.Google Scholar
  62. Taube, H. and Burnson, W. A. (2008). Software for teaching music theory. Technical report, University of Illinois at Urbana-Champaign.Google Scholar
  63. Temperley, D. (1997). An algorithm for harmonic analysis. Music Perception, 15(1):31-68.Google Scholar
  64. Temperley, D. (2001). The Cognition of Basic Musical Structures. The MIT Press.Google Scholar
  65. Temperley, D. (2004). Bayesian models of musical structure and cognition. Musicae Scientiae, 8(2):175-205.Google Scholar
  66. Temperley, D. and Sleator, D. (1999). Modeling meter and harmony: A preference- rule approach. Computer Music Journal, 23(1):10-27.Google Scholar
  67. Tojo, S., Oka, Y., and Nishida, M. (2006). Analysis of chord progression by HPSG. In AIA’06: Proceedings of the 24th IASTED international conference on Artificial intelligence and applications. Google Scholar
  68. Toselli, A. H., Vidal, E., and Casacuberta, F. (2011). Multimodal Interactive Pattern Recognition and Applications. Springer.Google Scholar
  69. Tracy, M. S. (2013). Bach in Beta: Modeling Bach chorales with Markov Chains. PhD thesis, Harvard University.Google Scholar
  70. Tsui, W. (2002). Harmonic analysis using neural networks. Master’s thesis, University of Toronto.Google Scholar
  71. Ulrich, J. W. (1977). The analysis and synthesis of jazz by computer. In Proceedings of the 5th International Joint Conference on Artificial intelligence (IJCAI). Google Scholar
  72. Willingham, T. J. (2013). The harmonic implications of the non-harmonic tones in the four-part chorales ofJohann Sebastian Bach. PhD thesis, Liberty University.Google Scholar
  73. Winograd,T. (1968). Linguistics and the computer analysis of tonal harmony. Journal of Music Theory, 12(1):2-49.Google Scholar
  74. Yi, L. and Goldsmith, J. (2007). Automatic generation of four-part harmony. In Proceedings of the Fifth UAI Bayesian Modeling Applications Workshop (UAI-AW 2007, BMA’07, Vancouver, Canada.Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • David Rizo
    • 1
    • 2
    Email author
  • Plácido R. Illescas
    • 1
  • José M. Iñesta
    • 1
  1. 1.Universidad de AlicanteAlicanteSpain
  2. 2.Instituto Superior de Enseñanzas Artísticas de la Comunidad Valenciana (ISEA.CV), EASDAlicanteSpain

Personalised recommendations