Music staff removal with supervised pixel classification

  • Jorge Calvo-Zaragoza
  • Luisa Micó
  • Jose Oncina
Original Paper


This work presents a novel approach to tackle the music staff removal. This task is devoted to removing the staff lines from an image of a music score while maintaining the symbol information. It represents a key step in the performance of most optical music recognition systems. In the literature, staff removal is usually solved by means of image processing procedures based on the intrinsics of music scores. However, we propose to model the problem as a supervised learning classification task. Surprisingly, although there is a strong background and a vast amount of research concerning machine learning, the classification approach has remained unexplored for this purpose. In this context, each foreground pixel is labelled as either staff or symbol. We use pairs of scores with and without staff lines to train classification algorithms. We test our proposal with several well-known classification techniques. Moreover, in our experiments no attempt of tuning the classification algorithms has been made, but the parameters were set to the default setting provided by the classification software libraries. The aim of this choice is to show that, even with this straightforward procedure, results are competitive with state-of-the-art algorithms. In addition, we also discuss several advantages of this approach for which conventional methods are not applicable such as its high adaptability to any type of music score.


Music staff removal Optical music recognition Pixel classification Supervised learning 


  1. 1.
    Bainbridge, D., Bell, T.: The challenge of optical music recognition. Comput. Humanit. 35(2), 95–121 (2001). doi: 10.1023/A:1002485918032 CrossRefGoogle Scholar
  2. 2.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001). doi: 10.1023/A:1010933404324 MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Calvo-Zaragoza, J., Barbancho, I., Tardón, L.J., Barbancho, A.M.: Avoiding staff removal stage in optical music recognition: application to scores written in white mensural notation. Pattern Anal. Appl. 18(4), 933–943 (2015)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Carter, N.P.: Segmentation and preliminary recognition of madrigals notated in white mensural notation. Mach. Vis. Appl. 5(3), 223–229 (1992). doi: 10.1007/BF02627000 CrossRefGoogle Scholar
  5. 5.
    Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3642–3649. IEEE (2012)Google Scholar
  6. 6.
    Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13(1), 21–27 (1967). doi: 10.1109/TIT.1967.1053964 CrossRefzbMATHGoogle Scholar
  7. 7.
    Dalitz, C., Droettboom, M., Pranzas, B., Fujinaga, I.: A comparative study of staff removal algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 30(5), 753–766 (2008). doi: 10.1109/TPAMI.2007.70749 CrossRefGoogle Scholar
  8. 8.
    Diethe, T., Girolami, M.: Online learning with multiple kernels: a review. Neural Comput. 25(3), 567–625 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York (2001)zbMATHGoogle Scholar
  10. 10.
    Dutta, A., Pal, U., Fornes, A., Llados, J.: An efficient staff removal approach from printed musical documents. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 1965–1968 (2010)Google Scholar
  11. 11.
    Fornés, A., Dutta, A., Gordo, A., Lladós, J.: CVC-MUSCIMA: a ground truth of handwritten music score images for writer identification and staff removal. Int. J. Doc. Anal. Recogn. 15(3), 243–251 (2012)CrossRefGoogle Scholar
  12. 12.
    Fornés, A., Kieu, V.C., Visani, M., Journet, N., Dutta, A.: The ICDAR/GREC 2013 music scores competition: Staff removal. In: 10th International Workshop on Graphics Recognition, Current Trends and Challenges GREC 2013, Bethlehem, PA, USA, August 20–21, 2013, Revised Selected Papers, pp. 207–220 (2013)Google Scholar
  13. 13.
    Géraud, T.: A morphological method for music score staff removal. In: Proceedings of the 21st International Conference on Image Processing (ICIP), pp. 2599–2603. Paris, France (2014)Google Scholar
  14. 14.
    Gosselin, P., Cord, M.: Active learning methods for interactive image retrieval. IEEE Trans. Image Process. 17(7), 1200–1211 (2008)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explor. Newsl. 11(1), 10–18 (2009). doi: 10.1145/1656274.1656278 CrossRefGoogle Scholar
  16. 16.
    Hart, P.: The condensed nearest neighbor rule (corresp.). IEEE Trans. Inf. Theory 14(3), 515–516 (1968). doi: 10.1109/TIT.1968.1054155 CrossRefGoogle Scholar
  17. 17.
    Hirata, N.S.T.: Multilevel training of binary morphological operators. IEEE Trans. Pattern Anal. Mach. Intell. 31(4), 707–720 (2009)CrossRefGoogle Scholar
  18. 18.
    Khan, S.S., Madden, M.G.: One-class classification: taxonomy of study and review of techniques. Knowl. Eng. Rev. 29, 345–374 (2014)CrossRefGoogle Scholar
  19. 19.
    Montagner, I.d.S., Hirata, R., Hirata, N.S.: A machine learning based method for staff removal. In: Pattern Recognition (ICPR), 2014 22nd International Conference on, pp. 3162–3167 (2014). doi: 10.1109/ICPR.2014.545
  20. 20.
    Patel, V.M., Gopalan, R., Li, R., Chellappa, R.: Visual domain adaptation: a survey of recent advances. IEEE Signal Process. Mag. 32(3), 53–69 (2015)CrossRefGoogle Scholar
  21. 21.
    Piatkowska, W., Nowak, L., Pawlowski, M., Ogorzalek, M.: Stafflines pattern detection using the swarm intelligence algorithm. In: Bolc, L., Tadeusiewicz, R., Chmielewski, L.J., Wojciechowski, K. (eds.) Computer Vision and Graphics. Lecture Notes in Computer Science, vol. 7594, pp. 557–564. Springer, Berlin Heidelberg (2012)Google Scholar
  22. 22.
    Pinto, J.R.C., Vieira, P., Ramalho, M., Mengucci, M., Pina, P.,Muge, F.: Ancient music recovery for digital libraries. In:Proceedings of the 4th European Conference on Research and Advanced Technology for Digital Libraries, ECDL ’00, pp. 24–34.Springer, London, UK, UK (2000)Google Scholar
  23. 23.
    Pruslin, D.: Automatic recognition of sheet music. Sc.d. dissertation, Massachusetts Institute of Technology, UK (1966)Google Scholar
  24. 24.
    Pugin, L.: Optical music recognition of early typographic prints using hidden Markov models. In: ISMIR 2006, 7th International Conference on Music Information Retrieval, Victoria, Canada, 8–12 October 2006, Proceedings, pp. 53–56 (2006)Google Scholar
  25. 25.
    Ramirez, C., Ohya, J.: Automatic recognition of square notation symbols in western plainchant manuscripts. J. New Music Res. 43(4), 390–399 (2014)CrossRefGoogle Scholar
  26. 26.
    Rebelo, A., Capela, G., Cardoso, J.S.: Optical recognition of music symbols. Int. J. Doc. Anal. Recogn. 13(1), 19–31 (2010)CrossRefGoogle Scholar
  27. 27.
    Rebelo, A., Cardoso, J.: Staff line detection and removal in the grayscale domain. In: 2013 12th International Conference on Document Analysis and Recognition (ICDAR), pp. 57–61 (2013). doi: 10.1109/ICDAR.2013.20
  28. 28.
    Rebelo, A., Fujinaga, I., Paszkiewicz, F., Marçal, A.R.S., Guedes, C., Cardoso, J.S.: Optical music recognition: state-of-the-art and open issues. IJMIR 1(3), 173–190 (2012). doi: 10.1007/s13735-012-0004-6 Google Scholar
  29. 29.
    Rossant, F., Bloch, I.: Robust and adaptive OMR system including fuzzy modeling, fusion of musical rules, and possible error detection. EURASIP J. Appl. Signal Process. 2007(1), 160–160 (2007)CrossRefzbMATHGoogle Scholar
  30. 30.
    dos Santos Cardoso, J., Capela, A., Rebelo, A., Guedes, C., Pinto da Costa, J.: Staff detection with stable paths. IEEE Trans. Pattern Anal. Mach. Intell. 31(6), 1134–1139 (2009)CrossRefGoogle Scholar
  31. 31.
    Su, B., Lu, S., Pal, U., Tan, C.: An effective staff detection and removal technique for musical documents. In: 2012 10th IAPR International Workshop on Document Analysis Systems (DAS), pp. 160–164 (2012). doi: 10.1109/DAS.2012.16
  32. 32.
    Tardón, L.J., Sammartino, S., Barbancho, I., Gómez, V., Oliver, A.: Optical music recognition for scores written in white mensural notation. EURASIP J. Image Video Process. 2009 (2009). doi: 10.1155/2009/843401
  33. 33.
    Vapnik, V.N.: Statistical Learning Theory, 1st edn. Wiley, Hoboken (1998)zbMATHGoogle Scholar
  34. 34.
    Visani, M., Kieu, V., Fornes, A., Journet, N.: ICDAR 2013 Music Scores Competition: Staff Removal. In: 2013 12th International Conference on Document Analysis and Recognition (ICDAR), pp. 1407–1411 (2013). doi: 10.1109/ICDAR.2013.284

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Jorge Calvo-Zaragoza
    • 1
  • Luisa Micó
    • 1
  • Jose Oncina
    • 1
  1. 1.Departamento de Lenguajes y Sistemas InformáticosUniversidad de AlicanteAlicanteSpain

Personalised recommendations