MIRAI: Multi-hierarchical, FS-Tree Based Music Information Retrieval System

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4585)


With the fast booming of online music repositories, there is a need for content-based automatic indexing which will help users to find their favorite music objects in real time. Recently, numerous successful approaches on musical data feature extraction and selection have been proposed for instrument recognition in monophonic sounds. Unfortunately, none of these methods can be successfully applied to polyphonic sounds. Identification of music instruments in polyphonic sounds is still difficult and challenging, especially when harmonic partials are overlapping with each other. This has stimulated the research on music sound separation and new features development for content-based automatic music information retrieval. Our goal is to build a cooperative query answering system (QAS), for a musical database, retrieving from it all objects satisfying queries like ”find all musical pieces in pentatonic scale with a viola and piano where viola is playing for minimum 20 seconds and piano for minimum 10 seconds”. We use the database of musical sounds, containing almost 4000 sounds taken from the MUMs (McGill University Master Samples), as a vehicle to construct several classifiers for automatic instrument recognition. Classifiers showing the best performance are adopted for automatic indexing of musical pieces by instruments. Our musical database has an FS-tree (Frame Segment Tree) structure representation. The cooperativeness of QAS is driven by several hierarchical structures used for classifying musical instruments.


Musical Instrument Decision Attribute Audio Feature Musical Piece Music Information Retrieval 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ben-Tal, O., Berger, J., Cook, B., Daniels, M., Scavone, G., Cook, P.: SONART: The Sonification Application Research Toolbox. In: Proceedings of the 2002 International Conference on Auditory Display, Kyoto, Japan (July 2002)Google Scholar
  2. 2.
    Bregman, A.S.: Auditory scene analysis, the perceptual organization of sound. MIT Press, Cambridge (1990)Google Scholar
  3. 3.
    Brown, J.C., Houix, O., McAdams, S.: Feature dependence in the automatic identification of musical woodwind instruments. J. Acoust. Soc. of America 109, 1064–1072 (2001)CrossRefGoogle Scholar
  4. 4.
    Cardoso, J.F., Comon, P.: Independent Component Analysis, a Survey of Some Algebraic methods. In: Proc. ISCAS Conference, Atlanta, May 1996, vol. 2, pp. 93–96 (1996)Google Scholar
  5. 5.
    Flanagan, J.L.: Speech Analysis, Synthesis and Perception. Springer, New York (1972)Google Scholar
  6. 6.
    Fujinaga, I., McMillan, K.: Real time Recognition of Orchestral Instruments. In: International Computer Music Conference, pp. 141–143 (2000)Google Scholar
  7. 7.
    Gaasterland, T.: Cooperative answering through controlled query relaxation. IEEE Expert 12(5), 48–59 (1997)CrossRefGoogle Scholar
  8. 8.
    Godfrey, P.: Minimization in cooperative response to failing database queries. International Journal of Cooperative Information Systems 6(2), 95–149 (1993)CrossRefGoogle Scholar
  9. 9.
    Goodwin, M.M.: Adaptive Signal Models: Theory, Algorithms, and Audio Applications, Ph.D. dissertation, University of California, Berkeley (1997)Google Scholar
  10. 10.
    Herrera, P., Amatriain, X., Batlle, E., Serra, X.: Towards instrument segmentation for music content description: a critical review of instrument classification techniques. In: ISMIR 2000. Proc. of International Symposium on Music Information Retrieval, Plymouth, MA (2000)Google Scholar
  11. 11.
    Hornbostel, E.M.V., Sachs, C.: Systematik der Musikinstrumente. Ein Versuch. Zeitschrift fur Ethnologie 46(4-5), 553–590 (1914), available at Google Scholar
  12. 12.
    Kaminskyj, I.: Multi-feature Musical Instrument Classifier, MikroPolyphonie 6, 2000, online journal at
  13. 13.
    Kostek, B., Czyzewski, A.: Representing Musical Instrument Sounds for Their Automatic Classification. J. Audio Eng. Soc. 49(9), 768–785 (2001)Google Scholar
  14. 14.
    Kostek, B., Wieczorkowska, A.: Parametric Representation of Musical Sounds. Archive of Acoustics 22(1), 3–26 (1997)Google Scholar
  15. 15.
    Lewis, R., Zhang, X., Ras, Z.W.: Blind Signal Separation of Similar Pitches and Instruments in a Noisy Polyphonic Domain. In: Esposito, F., Raś, Z.W., Malerba, D., Semeraro, G. (eds.) ISMIS 2006. LNCS (LNAI), vol. 4203, pp. 228–237. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  16. 16.
    Manjunath, B.S., Salembier, P., Sikora, T. (eds.): Introduction to MPEG-7. Multimedia Content Description Interface. J. Wiley and Sons, New York (2002)Google Scholar
  17. 17.
    Martin, K.D., Kim, Y.E.: Musical instrument identification: a pattern-recognition approach. In: Proceedings of 136th Meeting of the Acoustical Society of America, Norfolk, VA (October 1998)Google Scholar
  18. 18.
    Meier, U., Stiefelhagen, R., Yang, J., Waibel, A.: Towards Unrestricted Lip Reading. International Journal of Pattern Recognition and Artificial Intelligence 14(5), 571–586 (2000)CrossRefGoogle Scholar
  19. 19.
    Opolko, F., Wapnick, J.: MUMS - McGill University Master Samples, CD’s (1987)Google Scholar
  20. 20.
    Pollard, H.F., Jansson, E.V.: A Tristimulus Method for the spectificaiton of Musical Timbre. Acustica (51), 162–171 (1982)Google Scholar
  21. 21.
    Popovic, I., Coifman, R., Berger, J.: Aspects of Pitch-Tracking and Timbre Separation: Feature Detection in Digital Audio Using Adapted Local Trigonometric Bases and Wavelet Packets Center for Studies in Music Technology, Yale University, Research Abstract (June 1995)Google Scholar
  22. 22.
    Rabiner, L., Schafer, R.: Digital Processing of Speech Signals. Prentice-Hall, Englewood Cliffs, New Jersey (1978)Google Scholar
  23. 23.
    Wieczorkowska, A.: Musical Sound Classification based on Wavelet Analysis. Fundamenta Informaticae Journal 47(1), 175–188 (2001)zbMATHGoogle Scholar
  24. 24.
    Wieczorkowska, A.: The recognition efficiency of musical instrument sounds depending on parameterization and type of a classifier, PhD. thesis (in Polish), Technical University of Gdansk, Poland (1999)Google Scholar
  25. 25.
    Wieczorkowska, A., Raś, Z.W., Zhang, X., Lewis, R.: Multi-way Hierarchic Classification of Musical Instrument Sounds (will appear). In: MUE 2007. Proceedings of the IEEE CS International Conference on Multimedia and Ubiquitous Engineering, Seoul, Korea, April 26-28, 2007, IEEE Computer Society Press, Los Alamitos (2007)Google Scholar
  26. 26.
    Zhang, X., Raś, Z.W.: Differentiated Harmonic Feature Analysis on Music Information Retrieval For Instrument Recognition. In: IEEE GrC, 2006. Proceedings of IEEE International Conference on Granular Computing, Atlanta, Georgia, May 10-12, 2006, pp. 578–581. IEEE Computer Society Press, Los Alamitos (2006)CrossRefGoogle Scholar
  27. 27.
    Zhang, X., Marasek, K., Raś, Z.W.: Maximum Likelihood Study for Sound Pattern Separation and Recognition. In: MUE 2007. Proceedings of the IEEE CS International Conference on Multimedia and Ubiquitous Engineering, Seoul, Korea, April 26-28, 2007, IEEE Computer Society Press, Los Alamitos (2007) (will appear)Google Scholar
  28. 28.
    Zhang, X., Raś, Z.W.: Analysis of Sound Features for Music Timbre Recognition. In: MUE 2007. Proceedings of the IEEE CS International Conference on Multimedia and Ubiquitous Engineering, April 26-28, 2007, IEEE Computer Society Press, Los Alamitos, in Seoul, Korea (will appear)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  1. 1.University of North Carolina, Dept. of Comp. Science, Charlotte, N.C. 28223USA
  2. 2.Polish-Japanese Institute of Information Technology, 02-008 WarsawPoland

Personalised recommendations