Skip to main content
SpringerLink
Log in

Recognition of Music Scores with Non-Linear Distortions in Mobile Devices

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Optical music recognition (OMR), when the input music score is captured by a handheld or a mobile phone camera, suffers from severe degradation in the image quality and distortions caused by non-planar document curvature and perspective projection. Hence the binarization of the input often fails to preserve the details of the original music score, leading to a poor performance in recognition of music symbols. This paper addresses the issue of staff line detection, which is the most important step in OMR, in the presence of nonlinear distortions and describes how to cope with severe degradations in recognition of music symbols. First, a RANSAC-based detection of curved staff lines is presented and staves are segmented into sub-areas for the rectification with bi-quadratic transformation. Then, run length coding is used to recognize music symbols such as stem, note head, flag, and beam. The proposed system is implemented on smart phones, and it shows promising results with music score images captured in the mobile environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. Demonstration of user interface: https://www.youtube.com/watch?v=lh5c9z-JaZg

References

  1. Audiveris (2013) https://github.com/audiveris

  2. Bellini P, Bruno I, Nesi P (2001) Optical music sheet segmentation. In: International Conference on WEB Delivering of Music. IEEE, Florence, p 183–190

  3. Blostein D, Baird HS (1992) A critical survey of music image analysis. In: Structured Document Image Analysis, p 405–434

  4. Bolles RC, Fischler MA (1981) A RANSAC-based approach to model fitting and its application to finding cylinders in range data. In: International Joint Conference on Artificial Intelligence, Vancouver, Canada, vol 2, p 637–643

  5. Calvo-Zaragoza J, Vigliensoni G, Fujinaga I (2017) Pixel-wise Binarization of Musical Documents with Convolutional Neural Networks. In: IAPR Conference on Machine Vision Applications

  6. Cardoso JS, Capela A, Rebelo A, Guedes C, Pinto da Costa J (2009) Staff Detection with Stable Paths. IEEE Trans Pattern Anal Mach Intell 31(6):1134–1139

    Article  Google Scholar 

  7. Carter NP, Bacon RA (1992) Automatic Recognition of Printed Music. Structured Document Image Analysis, p 454–465

  8. Cormarck AM (1963) Representation of a function by its line integral, with some radiological applications. J Appl Phys 34(9):2722–2727

    Article  Google Scholar 

  9. Fujinaga I (2004) Staff Detection and Removal. Visual Perception of Music Notation, p 1–39

  10. Hsu YN, Arsenault HH, April G (1982) Rotation-invariant digital pattern recognition using circular harmonic expansion. Appl Opt 21(22):4012–4015

    Article  Google Scholar 

  11. ICDAR / GREC (2013) Competition on Music Scores. http://dag.cvc.uab.es/muscima/?page_id=11

  12. Jacobi L (2011) SmartScore X Pro (review). PC World, London Accessed August 2013

    Google Scholar 

  13. Johnson M (2008) Finale 2008 Power. Penolope Press, New York, p 288

    Google Scholar 

  14. Kahan S, Pavlidis T, Baird HS (1987) On the recognition of printed characters of any font and size. IEEE Trans Pattern Anal Machine Intell 9(2):274–288

    Article  Google Scholar 

  15. Kato H, Inokuchi S (1992) A recognition system for printed piano music using musical knowledge and constraints. In: Structured Document Image Analysis, p 435–455

  16. Martin P, Bellissant C (1991) Low-Level Analysis of Music Drawing Images. Proc. First Int’l Conf. Document Analysis and Recognition, p 417–425

  17. Nicholl, Grudzinski (2007) Music Notation: Preparing Scores and Parts, 1st edn. Berklee Press, Boston, p 110

    Google Scholar 

  18. Pinto T, Rebelo A, Giraldi G, Cardoso JS (2011) Music Score Binarization Based on Domain Knowledge. Lecture Notes in Computer Science book series (LNCS, volume 6669)

  19. Randriamahefa R, Cocquerez JP, Fluhr C, Pepin F, Philipp S (1993) Printed music recognition. Proceedings of the second international conference on document analysis and recognition, p 898–901

  20. Rebelo FI, Paszkiewicz F, Marcal ARS, Guedes C, Cardoso JS (2012) Optical music recognition: state-of-the-art and open issues. Int J Multimed Inf Retr 1:173–190

    Article  Google Scholar 

  21. Reed KT, Parker JR (1996) Automatic computer recognition of printed music. In: International Conference on Pattern Recognition, vol 3. IEEE, Vienna, p 803–807

  22. Sayood K (2002) Lossless Compression Handbook. Academic Press, Oxford

    Google Scholar 

  23. Soille P (1999) Morphological Image Analysis: Principles and Applications. Springer-Verlag, New York, pp 173–174

    Book  MATH  Google Scholar 

  24. Spenillo C (2013) MakeMusic Releases Finale 2014. Business Wire, San Francisco

    Google Scholar 

  25. Su B, Lu S, Pal U, Tan CL (2012) An Effective Staff Detection and Removal Technique for Musical Documents. Proceedings of the 2012 10th IAPR International Workshop on Document Analysis Systems, p 160–164

  26. Tang YY, Suen CY (1993) Image transformation approach to nonlinear shape restoration. IEEE Trans Syst Man Cybernet 23(1):155–172

    Article  MATH  Google Scholar 

  27. Tang YY, Li ZC, Suen CY, Bui TD (1988) Conversion of Chinese characters by transformation models. In: Proc. Int. Conf Computer Processing of Chinese and Oriental Languages, p 293–297

  28. Tard’on LJ, Sammartino S, Barbancho I, G’omez V, Oliver A (2009) Optical Music Recognition for Scores Written in White Mensural Notation, EURASIP Journal on Image and Video Processing 2009

  29. VanDerBosch K (2009) MakeMusic, Inc. Releases Finale 2009: Finale’s 20th anniversary is celebrated with major workflow improvements (Press release)

  30. Vo QN, Nguyen T, Kim SH, Yang HJ, Lee GS (2014) Distorted Music Score Recognition without Staffline Removal. In: Proc. Int. Conf on Pattern Recognition, p 2956–2960

Download references

Acknowledgements

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by MEST (NRF-2015R1D1A1A01060172 and NRF-2017R1A4A1015559).

Author information

Authors and Affiliations

  1. School of Electronics & Computer Engineering, Chonnam National University, Gwangju, 500-757, South Korea

    QuangNhat Vo, GueeSang Lee, SooHyung Kim & HyungJeong Yang

Authors
  1. QuangNhat Vo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. GueeSang Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. SooHyung Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. HyungJeong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to GueeSang Lee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vo, Q., Lee, G., Kim, S. et al. Recognition of Music Scores with Non-Linear Distortions in Mobile Devices. Multimed Tools Appl 77, 15951–15969 (2018). https://doi.org/10.1007/s11042-017-5169-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5169-9

Keywords

Search

Navigation