Optical recognition of music symbols

A comparative study
  • A. RebeloEmail author
  • G. Capela
  • Jaime S. Cardoso
Original Paper


Many musical works produced in the past are still currently available only as original manuscripts or as photocopies. The preservation of these works requires their digitalization and transformation into a machine-readable format. However, and despite the many research activities on optical music recognition (OMR), the results for handwritten musical scores are far from ideal. Each of the proposed methods lays the emphasis on different properties and therefore makes it difficult to evaluate the efficiency of a proposed method. We present in this article a comparative study of several recognition algorithms of music symbols. After a review of the most common procedures used in this context, their respective performances are compared using both real and synthetic scores. The database of scores was augmented with replicas of the existing patterns, transformed according to an elastic deformation technique. Such transformations aim to introduce invariances in the prediction with respect to the known variability in the symbols, particularly relevant on handwritten works. The following study and the adopted databases can constitute a reference scheme for any researcher who wants to confront a new OMR algorithm face to well-known ones.


Music Performance evaluation Symbol recognition Document image processing Off-line recognition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arica N., Yarman-Vural F.: An overview of character recognition focused on off-line handwriting. IEEE Trans. Syst., Man, Cybern., Part C: Applica. Rev. 31(2), 216–233 (2001). doi: 10.1109/5326.941845 CrossRefGoogle Scholar
  2. 2.
    Bainbridge, D.: An extensible optical music recognition system. In: Nineteenth Australasian Computer Science Conference, pp. 308–317 (1997)Google Scholar
  3. 3.
    Baird, H.: Document image defect models and their uses. pp. 62–67 (1993). doi: 10.1109/ICDAR.1993.395781
  4. 4.
    Bellini, P., Bruno, I., Nesi, P.: Optical music sheet segmentation. In: Proceedings of the 1st International Conference on Web Delivering of Music, pp. 183–190 (2001)Google Scholar
  5. 5.
    Blostein D., Baird H.S.: A critical survey of music image analysis. In: Baird Bunke, Y. (eds) Structured Document Image Analysis, pp. 405–434. Springer, Heidelberg (1992)Google Scholar
  6. 6.
    Bojovic, M., Savic, M.D.: Training of hidden Markov models for cursive handwritten word recognition. In: ICPR ’00: Proceedings of the International Conference on Pattern Recognition, p. 1973. IEEE Computer Society, Washington, DC, USA (2000)Google Scholar
  7. 7.
    Capela, A., Rebelo, A., Cardoso, J.S., Guedes, C.: Staff line detection and removal with stable paths. In: Proceedings of the International Conference on Signal Processing and Multimedia Applications (SIGMAP 2008), pp. 263–270 (2008).
  8. 8.
    Cardoso, J.S., Capela, A., Rebelo, A., Guedes, C.: A connected path approach for staff detection on a music score. In: Proceedings of the International Conference on Image Processing (ICIP 2008), pp. 1005–1008 (2008)Google Scholar
  9. 9.
    Cardoso J.S., Capela A., Rebelo A., Guedes C., da Costa J.P.: Staff detection with stable paths. IEEE Trans. Pattern Anal. Mach. Intell. 31(6), 1134–1139 (2009). doi: 10.1109/TPAMI.2009.34 CrossRefGoogle Scholar
  10. 10.
    Coüasnon, B.: Segmentation et reconnaissance de documents guidées par la connaissance a priori: application aux partitions musicales. Ph.D. thesis, Université de Rennes (1996)Google Scholar
  11. 11.
    Coüasnon, B., Camillerapp, J.: Using grammars to segment and recognize music scores. In: Proceedings of DAS-94: International Association for Pattern Recognition Workshop on Document Analysis Systems, pp. 15–27. Kaiserslautern (1993)Google Scholar
  12. 12.
    Dalitz, C., Droettboom, M., Czerwinski, B., Fujigana, I.: Staff removal toolkit for gamera (2005–2007).
  13. 13.
    Dalitz C., Droettboom M., Czerwinski B., Fujigana I.: A comparative study of staff removal algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 30, 753–766 (2008)CrossRefGoogle Scholar
  14. 14.
    Duda R.O., Hart P.E., Stork D.G.: Pattern Classification (2nd Edn.). Wiley, New York (2000)Google Scholar
  15. 15.
    Fornés, A., Lladós, J., Sánchez, G.: Primitive segmentation in old handwritten music scores. In: Liu, W., Lladós, J. (eds.) GREC, Lecture Notes in Computer Science, vol. 3926, pp. 279–290. Springer (2005).
  16. 16.
    Fujinaga I.: Staff detection and removal. In: George, S. (eds) Visual Perception of Music Notation: On-Line and Off-Line Recognition, pp. 1–39. Idea Group Inc, Hershey (2004)Google Scholar
  17. 17.
    Gonzalez R.C., Woods R.E., Eddins S.L.: In: Digital Image processing using MATLAB, Pearson/Prentice-Hall, Upper Saddle River (2004)Google Scholar
  18. 18.
    Haykin, S.: Neural Networks: A Comprehensive Foundation (2nd edn.). Prentice Hall (1998).
  19. 19.
    Jain A.K., Zhong Y., Lakshmanan S.: Object matching using deformable templates. IEEE Trans. Pattern Anal. Mach. Intell. 18(3), 267–278 (1996). doi: 10.1109/34.485555 CrossRefGoogle Scholar
  20. 20.
    Jain A.K., Zongker D.: Representation and recognition of handwritten digits using deformable templates. IEEE Trans. Pattern Anal. Mach. Intell. 19(12), 1386–1391 (1997). doi: 10.1109/34.643899 CrossRefGoogle Scholar
  21. 21.
    Kanungo, T.: Document degradation models and a methodology for degradation model validation. Ph.D. thesis, Seattle, WA, USA (1996)Google Scholar
  22. 22.
    Kanungo T., Haralick R., Baird H., Stuezle W., Madigan D.: A statistical, nonparametric methodology for document degradation model validation. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1209–1223 (2000). doi: 10.1109/34.888707 CrossRefGoogle Scholar
  23. 23.
    Kopec, G.E., Parc, P.A.C., Maltzcarnegie, D.A.: Markov source model for printed music decoding. J Electron Imaging, pp. 7–14 (1996)Google Scholar
  24. 24.
    Lam L., Suen C.Y.: Structural classification and relaxation matching of totally unconstrained handwritten zip-code numbers. Pattern Recognit. 21(1), 19–32 (1988). doi: 10.1016/0031-3203(88)90068-4 CrossRefGoogle Scholar
  25. 25.
    Mitobe, Y., Miyao, H., Maruyama, M.: A fast HMM algorithm based on stroke lengths for on-line recognition of handwritten music scores. In: IWFHR ’04: Proceedings of the Ninth International Workshop on Frontiers in Handwriting Recognition, pp. 521–526. IEEE Computer Society, Washington (2004). doi: 10.1109/IWFHR.2004.2
  26. 26.
    Miyao H., Nakano Y.: Note symbol extraction for printed piano scores using neural networks. IEICE Trans. Inf. Syst. E79–D, 548–554 (1996)Google Scholar
  27. 27.
    Miyao H., Okamoto M.: Stave extraction for printed music scores using DP matching. J Adv. Comput. Intell. Intell. Inform. 8, 208–215 (2007)Google Scholar
  28. 28.
    Ng K.: Optical music analysis for printed music score and handwritten music manuscript. In: George, S. (eds) Visual Perception of Music Notation: On-Line and Off-Line Recognition, pp. 108–127. Idea Group Inc, Hershey (2004)Google Scholar
  29. 29.
    Nishida H.: A structural model of shape deformation. Pattern Recognit. 28(10), 1611–1620 (1995)CrossRefGoogle Scholar
  30. 30.
    Pugin, L.: Optical music recognition of early typographic prints using hidden Markov models. In: ISMIR, pp. 53–56 (2006)Google Scholar
  31. 31.
    Randriamahefa, R., Cocquerez, J., Fluhr, C., Pepin, F., Philipp, S.: Printed music recognition. In: Proceedings of the Second International Conference on Document Analysis and Recognition, pp. 898–901 (1993). doi: 10.1109/ICDAR.1993.395592
  32. 32.
    Reed K.T., Parker J.R.: Automatic computer recognition of printed music. Proc. 13th Int. Conf. Pattern Recognit. 3, 803–807 (1996). doi: 10.1109/ICPR.1996.547279 CrossRefGoogle Scholar
  33. 33.
    Rossant F., Bloch I.: Robust and adaptive omr system including fuzzy modeling, fusion of musical rules, and possible error detection. EURASIP J. Adv. Signal Process. 2007(1), 160–160 (2007). doi: 10.1155/2007/81541 Google Scholar
  34. 34.
    Szwoch M.: A robust detector for distorted music staves. In: Computer Analysis of Images and Patterns, pp. 701–708. Springer, Heidelberg (2005)Google Scholar
  35. 35.
    Toyama, F., Shoji, K., Miyamichi, J.: Symbol recognition of printed piano scores with touching symbols. pp. 480–483 (2006). doi: 10.1109/ICPR.2006.1099
  36. 36.
    Vapnik, V.N.: Statistical Learning Theory. Wiley (1998).
  37. 37.
    Wakahara T.: Shape matching using LAT and its application to handwritten numeral recognition. IEEE Trans. Pattern Anal. Mach. Intell. 16(6), 618–629 (1994). doi: 10.1109/34.295906 CrossRefGoogle Scholar
  38. 38.
    Wang, Y.K., Fan, K.C., Juang, Y.T., Chen, T.H.: Using hidden Markov model for chinese business card recognition. In: ICIP (1), pp. 1106–1109 (2001)Google Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  1. 1.INESC Porto, Faculdade de CiênciasUniversidade do PortoPortoPortugal
  2. 2.INESC Porto, Faculdade de EngenhariaUniversidade do PortoPortoPortugal

Personalised recommendations