Computers and the Humanities

, Volume 35, Issue 2, pp 95–121 | Cite as

The Challenge of Optical Music Recognition

  • David Bainbridge
  • Tim Bell

Abstract

This article describes the challenges posed by optical musicrecognition – a topic in computer science that aims to convert scannedpages of music into an on-line format. First, the problem is described;then a generalised framework for software is presented that emphasises keystages that must be solved: staff line identification, musical objectlocation, musical feature classification, and musical semantics. Next,significant research projects in the area are reviewed, showing how eachfits the generalised framework. The article concludes by discussingperhaps the most open question in the field: how to compare the accuracy and success of rival systems, highlighting certain steps thathelp ease the task.

optical music recognition musical data acquisition document image analysis pattern recognition 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alphonce, B., B. Pennycook, I. Fujinaga and N. Boisvert. “Optical Music Recognition: A Progress Report”. In Proceedings of the Small Computers in the Arts, 1988, pp. 8–12.Google Scholar
  2. Anstince, J. A Pen-Input Computer Music Editing System. M.Sc. thesis, Department of Computer Science, University of Canterbury, Christchurch, NZ, 1996.Google Scholar
  3. Bainbridge, D. and T. Bell. “An Extensible Optical Music Recognition System”. In Proceedings of the Nineteenth Australasian Computer Science Conference. Melbourne, 1996, pp. 308–317.Google Scholar
  4. Bainbridge, D. and T. Bell. “Dealing with Superimposed Objects in Optical Music recognition”. In Proceedings of the Sixth International Conference on Image Processing and Its Applications. Dublin, 1997, pp. 756–760.Google Scholar
  5. Bainbridge, D. and N. Carter. “Automatic Reading of Music Notation”. In Handbook on Optical Character Recognition and Document Image Analysis. Ed. H. Bunke and P.S.P. Wang. Singapore, World Scientific, 1997, pp. 583–603.Google Scholar
  6. Bainbridge, D. and S. Inglis. “Music Image Compression”. In Proceedings of the IEEE Data Compression Conference. Ed J. Storer and M. Cohn. Snowbird, Utah: IEEE Computer Society Press, pp. 208–218.Google Scholar
  7. Bainbridge, D. A Complete Optical Music Recognition System: Looking to the Future. Technical report tr-cosc 06.94. Christchurch, NZ: Department of Computer Science, University of Canterbury, March 1994.Google Scholar
  8. Bainbridge, D. Optical Music Recognition: Progress Report 1. Technical report tr-cosc 05.94. Christchurch, NZ: Department of Computer Science, University of Canterbury, November 1994.Google Scholar
  9. Bainbridge, D. “Playing Musical Computers”. New Zealand Science Monthly, 5 (November 1994), 10–11.Google Scholar
  10. Bainbridge, D. “Optical Music Recognition: A Generalised Approach”. In Proceedings of the Second New Zealand Computer Science Research Students' Conference. Waikato, NZ, April 1995, pp. 23–30.Google Scholar
  11. Bainbridge, D. Optical Music Recognition: Progress Report 2. Technical report tr-cosc 02.95. Christchurch, NZ: University of Canterbury, January 1994.Google Scholar
  12. Bainbridge, D. Extensible Optical Music Recognition. Ph.D. thesis, Department of Computer Science, University of Canterbury, Christchurch, NZ, 1997.Google Scholar
  13. Baird, H.S., H. Bunke and K. Yamamoto, Ed. Structured Document Image Analysis. Berlin: Springer-Verlag, 1992.Google Scholar
  14. Baumann, S. and A. Dengel. “Transforming Printed Piano Music into MIDI”. In Advances in Structural and Syntactic Pattern Recognition (Proceedings of International Workshop on Structural and Syntactic Pattern Recognition), volume 5 of Series in Machine Perception and Artificial Intelligence. Ed. H. Bunke. Bern: World Scientific, 1992, pp. 363–372.Google Scholar
  15. Blostein, D. and H.S. Baird. “A Critical Survey of Music Image Analysis”. In Structured Document Image Analysis. Ed. H.S. Baird, H. Bunke and K. Yamamoto. Berlin: Springer-Verlag, 1992, pp. 405–434.Google Scholar
  16. Boyle, R. and R. Thomas. Computer Vision: A First Course. Oxford: Blackwell Scientific, 1988.Google Scholar
  17. Bulis, A., R. Almog, M. Gerner and U. Shimony. “Computerized Recognition of Hand-Written Musical Notes”. In Proceedings of the International Computer Music Conference. San Jose, California, 1992, pp. 110–112.Google Scholar
  18. Bunke, H. and P.S.P. Wang, Ed. Handbook of Character Recognition and Document Image Analysis. Singapore: World Scientific, 1997.Google Scholar
  19. Bunke, H. “Attributed Programmed Graph Grammars and Their Application to Schematic Diagram Interpretation”. IEEE Pattern Analysis and Machine Intelligence, 4(6) (Novemver 1982), 574–582.Google Scholar
  20. Carter, N.P. and R.A. Bacon. “Automatic Recognition of Music Notation”. In Proceedings of the International Association for Pattern Recognition Workshop on Syntactic and Structural Pattern Recognition. Murray Hill, New Jersey, June 1990, p. 482.Google Scholar
  21. Carter, N.P. and R.A. Bacon. “Automatic Recognition of Printed Music”. In Structured Document Image Analysis. Ed. H.S. Baird, H. Bunke and K. Yamamoto. Berling: Springer-Verlag, 1992, pp. 456–465.Google Scholar
  22. Carter, N.P. Automatic Recognition of Printed Music in the Context of Electronic Publishing. Ph.D. thesis, Departments of Physics and Music, University of Surrey, Guildford, UK, February 1989.Google Scholar
  23. Carter, N.P. “A New Edition of Walton's Façade Using Automatic Score Recognition”. In Advances in Structural and Syntatic Pattern Recognition (Proceedings of International Workshop on Structural and Syntactic Pattern Recognition), volume 5 of Series in Machine Perception and Artificial Intelligence. Ed. H. Bunke. Bern: World Scientific, 1992, pp. 352–362.Google Scholar
  24. Carter, N.P. “Segmentation and Preliminary Recognition of Madrigals Notated in White Mensural Notation”. Machine Vision and Applications, 5(3) (1992), 223–230.Google Scholar
  25. Carter, N.P. A Generalized Approach to Automatic Recognition of Music Scores. Technical Report STAN-M-87. California, USA: Department of Music, Stanford University, December 1993. 77 pp.Google Scholar
  26. Carter, N.P. “Conversion of the Haydn Symphonies into Electronic form Using Automatic Score Recognition: A Pilot Study”. In Proceedings of the IS&T/SPIE's Symposium on Electronic Imaging: Science and Technology; Conference 2181 – Document Recognition. Ed. L.M. Vincent and T. Pavlidis. San Jose, California, February 1994, pp. 279–290.Google Scholar
  27. Carter, N.P. “Music Score Recognition: Problems and Prospects”. Computing in Musicology: An International Directory of Applications, 9 (1994), 152–158.Google Scholar
  28. Clarke, A.T., B.M. Brown and M.P. Thorne. “Inexpensive Optical Character Recognition of Music Notation: A New Alternative for Publishers”. In Proceedings of the Computers in Music Research Conference. Luncaster, UK, April 1988, pp. 84–87.Google Scholar
  29. Coüasnon, B. and J. Camillerapp. “Using Grammar to Segment and Recognize Music Scores”. In International Association for Pattern Recognition Workshop on Document Analysis Systems. Kaiserslauter, Germany, October 1994, pp. 15–27.Google Scholar
  30. Coüasnon, B., P. Brisset and I. Stephan. “Using Logic Programming Languages for Optical Music Recognition”. In The Third International Conference on the Practical Application of Prolog. Paris, April 1995, pp. 115–134.Google Scholar
  31. Coüasnon, B. Segmentation and Recognition of Documents Guided by A Priori Knowledge: Application to Musical Scores. Ph.D. thesis, IRISA, France, 1997.Google Scholar
  32. Cover, T.M. and P.E. Hart. “Nearest Neighbor Classification”. IEEE Transactions of Information Theory, 13 (1967), 21–27.Google Scholar
  33. Fahmy, H. and D. Blostein. “A Graph Grammar for High-Level Recognition of Music Notation”. In Proceedings of First International Conference on Document Analysis and Recognition, volume 1. Saint Malo, France, 1991, pp. 70–78.Google Scholar
  34. Fahmy, H. and D. Blostein. “Graph Grammar Processing of Uncertain Data”. In Advances in Structural and Syntactic Pattern Recognition (Proceedings of International Workshop on Structural and Syntactic Pattern Recognition), volume 5 of Series in Machine Perception and Artificial Intelligence. Ed. H. Bunke. Bern: World Scientific, 1992, pp. 373–382.Google Scholar
  35. Foley, J.D., A. van Dam, S.K. Feiner and J.F. Hughes. Computer Graphics: Principles and Pratice. Reading, Massachusetts: Addison-Wesley, 1990.Google Scholar
  36. Fujinaga, I., B. Alphonce and B. Pennycook. “Issues in the Design of an Optical Music Recognition Systems”. In Proceedings of the International Computer Music Conference. Ohio State University, November 1989, pp. 113–116.Google Scholar
  37. Fujinaga, I., B. Alphonce, B. Pennycook and N. Boisvert. “Optical Recognition of Music Notation by Computer”. Computers in Music Research, 1 (1989), 161–164.Google Scholar
  38. Fujinaga, I., B. Pennycook and B. Alphonce. “Computer Recognition of Musical Notation”. In Proceedings of the First International Conference on Music Perception and Cognition. 1989, pp. 87–90.Google Scholar
  39. Fujinaga, I., B. Alphonce and K. Hogan. “Optical Music Recognition: Progress Report”. In Proceedings of the International Computer Music Conference. Montreal, Canada, 1991, pp. 66–73.Google Scholar
  40. Fujinaga, I., B. Pennycook and B. Alphonce. “The Optical Music Recognition Project”. Computers in Music Research, 3 (1991), 139–142.Google Scholar
  41. Fujinaga, I., B. Alphonce, B. Pennycook and G. Diener. “Interactive Optical Music Recognition”. In Proceedings of the International Computer Music Conference. San Jose, California, 1992, pp. 117–120.Google Scholar
  42. Fujinaga, I. Optical Music Recognition Using Projections. M.Sc. thesis, McGill University, Montreal, Canada, 1988.Google Scholar
  43. Fujinaga, I. “An OpticalMusic Recognition System that Learns”. In Enabling Technologies for High-Bandwidth Applications, volume SPIE 1785. Ed. J. Maitan. 1992, pp. 210–217.Google Scholar
  44. Fujinaga, I. Adaptive Optical Music Recognition. Ph.D. thesis, Department of Theory, Faculty of Music, McGill University, Montreal, Canada, 1997.Google Scholar
  45. Glass, S. Optical Music Recognition. B.Sc thesis, Department of Computer Science, University of Canterbury, Christchurch, NZ, 1989.Google Scholar
  46. Haken, L. and D. Blostein. “The TiliaMusic Representation: Extensibility, Abstraction, and Notation Contexts for the Lime Music Editor”. Computer Music Journal, 17(3) (1993), 43–58.Google Scholar
  47. Heussenstamm, G. The Norton Manual of Music Notation. New York: W. W. Norton, 1987.Google Scholar
  48. Kassler, M. “Optical Character Recognition of Printed Music: A Review of Two Dissertations”. Perspectives of New Music, 11(2) (1972), 250–254.Google Scholar
  49. Kato, H. and S. Inokuchi. “A Recognition System for Printed Piano Music Using Musical Knowledge and Constraints”. In Proceedings of the International Association for Pattern Recognition Workshop on Syntactic and Structural Pattern Recognition. Murray Hill, New Jersey, June 1990, pp. 231–248.Google Scholar
  50. Matsushima, T., T. Harada, I. Sonomoto, K. Kanamori, A. Uesugi, Y. Nimura, S. Hashimoto and S. Ohteru. “Automated Recognition System for Musical Score – the Vision System of WABOT-2”. In Bulleting of Science and Engineering Research Laboratory, volume 112. Waseda University, Tokyo, Japan, September 1985, pp. 25–52.Google Scholar
  51. McGee, W.F. and P. Merkley. “The Optical Scanning of Medieval Music”. Computers and the Humanities, 25(1) (1991), 47–53.Google Scholar
  52. Modayur, B.R., V. Ramesh, R.M. Maralick and L.G. Shapiro. MUSER: A Protoype Musical Score Recognition System Using Mathematical Morphology, Technical Report. Seattle, USA: Intelligent Systems Laboratory, Electrical Engineering Department, University of Washington, June 1995.Google Scholar
  53. Ng, K.C., R.D. Boyle and D. Cooper. “Automated Optical Musical Score Recognition and Its Enhancement Using High-Level Musical Knowledge”. In Proceedings of the XI Colloquium on Musical Informatics. Bologna, Italy, November 1995, pp. 167–170.Google Scholar
  54. Ng, K.C. and R.D. Boyle. “Recognition and Reconstruction of Primitives in Music Scores”. Image and Vision Computing, 14 (1996), 39–46.Google Scholar
  55. Ng, K.C. Automated Computer Recognition of Music Scores. Ph.D. thesis, University of Leeds, Leeds, UK, 1995.Google Scholar
  56. Pavlidis, T. Algorithms for Graphics and Image Processing. Rockville, Maryland: Computer Science Press, 1982.Google Scholar
  57. Prerau, D.S. Computer Pattern Recognition of Standard Engraved Music Notation. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA, September 1970.Google Scholar
  58. Prerau, D.S. “Computer Pattern Recognition of Printed Music”. In Proceedings of the Fall Joint Computer Conference. Montvale, New Jersey: AFIPS Press, November 1971.Google Scholar
  59. Prerau, D.S. “Do-Re-Mi: A Program that Recognizes Music Notation”. Computers and the Humanities, 9(1) (January 1975), 25–29.Google Scholar
  60. Pruslin, D. Automatic Recognition of Sheet Music. Sc.D. dissertation, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA, June 1966.Google Scholar
  61. Read, G. Music Notation: A Manual of Modern Practice. London: Victor Gollancz, 1974.Google Scholar
  62. Reed, T. Optical Music Recognition. M.Sc. thesis, Department of Computer Science, University of Calgary, Canada, September 1995.Google Scholar
  63. Roach, J.W. and J.E. Tatem. “Using Domain Knowledge in Low-Level Visual Processing to Interpret Handwritten Music: An Experiment”. Pattern Recognition, 21(1) (1988), 33–44.Google Scholar
  64. Roads, C. “The Tsukuba Mustical Robot”. Computer Music Journal, 10(2) (1986), 39–43.Google Scholar
  65. Ross, T. The Art of Music Engraving and Processing: A Complete Manual Reference and Text Book on Preparing Music for Reproduction and Print. Miami: Hansen Press, 1970.Google Scholar
  66. Russ, J.C. The Image Processing Handbook. Boca Raton, Florida: CRC Press, 1992.Google Scholar
  67. Selfridge-Field, E. “Optical Recognition of Music Notation: A Survey of CurrentWork”. Computing in Musicology: An International Directory of Applications, 9 (1994), 109–145.Google Scholar
  68. Witten, I., A. Moffat and T. Bell Managing Gigabytes: Compressing and Indexing Documents and Images. New York: Van Nostrand Reinhold, 1994.Google Scholar
  69. Wolman, J., J. Choi, S. Asgharzadeh and J. Kahana. “Recognition of Handwritten Music Notation”. In Proceedings of the International Computer Music Conference. San Jose, California, 1992, pp. 125–127.Google Scholar
  70. Yadid, O., E. Brutman, L. Dvir, M. Gerner and U. Shimony. “RAMIT: Neural Network for Recognition of Musical Notes”. In Proceedings of the International Computer Music Conference. San Jose, California, 1992, pp. 128–131.Google Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  • David Bainbridge
    • 1
  • Tim Bell
    • 2
  1. 1.Department of Computer ScienceUniversity of WaikatoHamiltonNew Zealand
  2. 2.Department of Computer ScienceUniversity of CanterburyChristchurchNew Zealand

Personalised recommendations