Segregating Musical Chords for Automatic Music Transcription: A LSTM-RNN Approach

  • Himadri MukherjeeEmail author
  • Ankita Dhar
  • Sk. Md. Obaidullah
  • K. C. Santosh
  • Santanu Phadikar
  • Kaushik Roy
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11942)


Notating or transcribing a music piece is very important for musicians. It not only helps them to communicate among each other but also helps in understanding a piece. This is very much essential for improvisations and performances. This makes automatic music transcription systems extremely important. Every music piece can be broadly categorized into two parts namely the lead section and the accompaniment section or background music (BGM). The BGM is very important in a piece as it sets the mood and makes a piece complete. Thus it is very much important to notate the BGM for properly understanding and performing a piece. One of the key components of BGM is known as chord which is constituted of two or more musical notes. Every composition is accompanied with a chord chart. In this paper, a long short term memory-recurrent neural network (LSTM-RNN)- based approach is presented for segregating musical chords from clips of short durations which can aid in automatic transcription. Experiments were performed on over 46800 clips and a highest accuracy of 99.91% has been obtained for the proposed system.


Chord identification Music signal LSTM-RNN 



The authors would like to thank Mr. Soukhin Bhattacherjee, Mr. Debajyoti Bose of Department of Electrical, Power & Energy, University of Petroleum and Energy Studies for their help during the entire course of this work. They also thank for the block diagram template.


  1. 1.
    Benetos, E., Dixon, S., Duan, Z., Ewert, S.: Automatic music transcription: an overview. IEEE Sig. Process. Mag. 36(1), 20–30 (2018)CrossRefGoogle Scholar
  2. 2.
    Benetos, E., Dixon, S., Giannoulis, D., Kirchhoff, H., Klapuri, A.: Automatic music transcription: challenges and future directions. J. Intell. Inf. Syst. 41(3), 407–434 (2013)CrossRefGoogle Scholar
  3. 3.
    Bereket, M., Shi, K.: An AI approach to automatic natural music transcription (2017)Google Scholar
  4. 4.
    Cheng, H.T., Yang, Y.H., Lin, Y.C., Liao, I.B., Chen, H.H.: Automatic chord recognition for music classification and retrieval. In: 2008 IEEE International Conference on Multimedia and Expo, pp. 1505–1508. IEEE (2008)Google Scholar
  5. 5.
    Costantini, G., Casali, D.: Recognition of musical chord notes. WSEAS Trans. Acoust. Music 1(1), 17–20 (2004)Google Scholar
  6. 6.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. ACM SIGKDD Explor. Newsl. 11(1), 10–18 (2009)CrossRefGoogle Scholar
  7. 7.
    Kroher, N., Gómez, E.: Automatic transcription of flamenco singing from polyphonic music recordings. IEEE/ACM Trans. Audio Speech Lang. Process. (TASLP) 24(5), 901–913 (2016)CrossRefGoogle Scholar
  8. 8.
    Li, J., Mohamed, A., Zweig, G., Gong, Y.: LSTM time and frequency recurrence for automatic speech recognition. In: 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pp. 187–191. IEEE (2015)Google Scholar
  9. 9.
    Lipton, Z.C., Berkowitz, J., Elkan, C.: A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:1506.00019 (2015)
  10. 10.
    Mukherjee, H., Dutta, M., Obaidullah, S.M., Santosh, K.C., Phadikar, S., Roy, K.: Lazy learning based segregation of Top-3 south indian languages with LSF-a feature. In: Santosh, K.C., Hegadi, R.S. (eds.) RTIP2R 2018. CCIS, vol. 1035, pp. 449–459. Springer, Singapore (2019). Scholar
  11. 11.
    Mukherjee, H., Obaidullah, S.M., Santosh, K., Phadikar, S., Roy, K.: Line spectral frequency-based features and extreme learning machine for voice activity detection from audio signal. Int. J. Speech Technol. 21(4), 753–760 (2018)CrossRefGoogle Scholar
  12. 12.
    Muludi, K., Loupatty, A.F.S., et al.: Chord identification using pitch class profile method with fast fourier transform feature extraction. Int. J. Comput. Sci. Issues (IJCSI) 11(3), 139 (2014)Google Scholar
  13. 13.
    Osmalsky, J., Embrechts, J.J., Van Droogenbroeck, M., Pierard, S.: Neural networks for musical chords recognition. In: Journees d’informatique Musicale, pp. 39–46 (2012)Google Scholar
  14. 14.
    Quenneville, D.: Automatic Music Transcription. Ph.D. thesis, Middlebury College (2018)Google Scholar
  15. 15.
    Rajparkur, P., Girardeau, B., Migimatsu, T.: A supervised approach to musical chord recognition (2015)Google Scholar
  16. 16.
  17. 17.
    Wats, N., Patra, S.: Automatic music transcription using accelerated multiplicative update for non-negative spectrogram factorization. In: 2017 International Conference on Intelligent Computing and Control (I2C2), pp. 1–5. IEEE (2017)Google Scholar
  18. 18.
    Zhou, X., Lerch, A.: Chord detection using deep learning. In: Proceedings of the 16th ISMIR Conference, vol. 53 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Himadri Mukherjee
    • 1
    Email author
  • Ankita Dhar
    • 1
  • Sk. Md. Obaidullah
    • 2
  • K. C. Santosh
    • 3
  • Santanu Phadikar
    • 4
  • Kaushik Roy
    • 1
  1. 1.Department of Computer ScienceWest Bengal State UniversityKolkataIndia
  2. 2.Department of Computer Science and EngineeringAliah UniversityKolkataIndia
  3. 3.Department of Computer ScienceThe University of South DakotaVermillionUSA
  4. 4.Department of Computer Science and EngineeringMaulana Abul Kalam Azad University of TechnologyKolkataIndia

Personalised recommendations