The Journal of Supercomputing

, Volume 73, Issue 1, pp 126–138 | Cite as

Parallel online time warping for real-time audio-to-score alignment in multi-core systems

  • Pedro Alonso
  • Raquel Cortina
  • F. J. Rodríguez-Serrano
  • P. Vera-Candeas
  • M. Alonso-González
  • José Ranilla
Article

Abstract

The audio-to-score framework consists of two separate stages: preprocessing and alignment. The alignment is commonly solved through offline dynamic time warping (DTW), which is a method to find the path over the distortion matrix with the minimum cost to determine the relation between the performance and the musical score times. In this work we propose a parallel online DTW solution based on a client–server architecture. The current version of the application has been implemented for multi-core architectures (\(\times \)86, \(\times \)64 and ARM), thus covering either powerful systems or mobile devices. An extensive experimentation has been conducted to validate the software. The experiments also show that our framework allows to achieve a good score alignment within the real-time window using parallel computing techniques.

Keywords

Audio-to-score alignment Dynamic time warping (DTW) Score following Parallel computing Xeon Phi ARM 

References

  1. 1.
    Joder C, Essid S, Richard G (2011) A conditional random field framework for robust and scalable audio-to-score matching. IEEE Trans Speech Audio Lang Process 19(8):2385–2397CrossRefGoogle Scholar
  2. 2.
    McNab RJ, Smith LA, Witten IH, Henderson CL, Cunningham SJ (1996) Towards the digital music library: tune retrieval from acoustic input. In: DL 96: Proceedings of the first ACM international conference on digital libraries. ACM, New York, pp 11–18Google Scholar
  3. 3.
    Dannenberg RB (2007) An intelligent multi-track audio editor. In: Proceedings of international computer music conference (ICMC), vol 2, pp 89–94Google Scholar
  4. 4.
    Duan Z, Pardo B (2011) Soundprism: an online system for score-informed source separation of music audio. IEEE J Sel Topics Signal Process 5(6):1205–1215CrossRefGoogle Scholar
  5. 5.
    Dixon S (2005) Live tracking of musical performances using on-line time warping. In: Proceedings of the international conference on digital audio effects (DAFx), Madrid, Spain, pp 92–97Google Scholar
  6. 6.
    Orio N, Schwarz D (2001) Alignment of monophonic and polyphonic music to a score. In: Proceedings of the international computer music conference (ICMC), pp 129–132Google Scholar
  7. 7.
    Simon I, Morris D, Basu S (2008) MySong: automatic accompaniment generation for vocal melodies. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 725–734Google Scholar
  8. 8.
    Rodriguez-Serrano FJ, Duan Z, Vera-Candeas P, Pardo B, Carabias-Orti JJ (2015) Online score-informed source separation with adaptive instrument models. J New Music Res Lond 44(2):83–96CrossRefGoogle Scholar
  9. 9.
    Arzt A, Widmer G, Dixon S (2008) Automatic page turning for musicians via real-time machine listening. In: Proceedings of the 18th European conference on artificial intelligence. IOS Press, Amsterdam, pp 241–245Google Scholar
  10. 10.
    Carabias-Orti JJ, Rodriguez-Serrano FJ, Vera-Candeas P, Canadas-Quesada FJ, Ruiz-Reyes N (2015) An audio to score alignment framework using spectral factorization and dynamic time warping. In: 16th International Society for music information retrieval conference, pp 742–748Google Scholar
  11. 11.
    Rodríguez-Serrano FJ, Menéndez-Canal J, Vidal A, Cañadas-Quesada FJ, Cortina R (2015) A DTW based score following method for score-informed sound source separation. In: Proceedings of the 12th sound and music computing conference 2015 (SMC-15), Ireland, pp 491–496Google Scholar
  12. 12.
    Carabias-Ortí JJ, Rodríguez-Serrano FJ, Vera-Candeas P, Cañadas-Quesada FJ, Ruíz-Reyes N (2013) Constrained non-negative sparse coding using learnt instrument templates for realtime music transcription. Eng Appl Artif Intell 26(7):1671–1680CrossRefGoogle Scholar
  13. 13.
    Raphael C (2006) Aligning music audio with symbolic scores using a hybrid graphical model. Mach Learn 65:389–409CrossRefGoogle Scholar
  14. 14.
    Schreck-Ensemble (2001–2004) ComParser 1.42. http://home.hku.nl/~pieter.suurmond/SOFT/CMP/doc/cmp.html. Accessed Sept 2015
  15. 15.
    Itakura F (1975) Minimum prediction residual principle applied to speech recognition. IEEE Trans Acoust Speech Signal Process 23:52–72CrossRefGoogle Scholar
  16. 16.
    Dannenberg R, Hu N (2003) Polyphonic audio matching for score following and intelligent audio editors. In: Proceedings of the international computer music conference. International Computer Music Association, San Francisco, pp 27–34Google Scholar
  17. 17.
    Mueller M, Kurth F, Roeder T (2004) Towards an efficient algorithm for automatic score-to-audio synchronization. In: Proceedings of the 5th international conference on music information retrieval, Barcelona, SpainGoogle Scholar
  18. 18.
    Mueller M, Mattes H, Kurth F (2006) An efficient multiscale approach to audio synchronization. In: Proceedings of the 7th international conference on music information retrieval, Victoria, CanadaGoogle Scholar
  19. 19.
    Kaprykowsky H, Rodet X (2006) Globally optimal short-time dynamic time warping applications to score to audio alignment. In: IEEE ICASSP, Toulouse, France, pp 249–252Google Scholar
  20. 20.
    Fremerey C, Müller M, Clausen M (2010) Handling repeats and jumps in score-performance synchronization. In: Proceedings of ISMIR, pp 243–248Google Scholar
  21. 21.
    Arzt A, Widmer G (2010) Towards effective any-time music tracking. In: Proceedings of starting AI researchers symposium (STAIRS), Lisbon, Portugal, pp 24–36Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Pedro Alonso
    • 1
  • Raquel Cortina
    • 2
  • F. J. Rodríguez-Serrano
    • 3
  • P. Vera-Candeas
    • 3
  • M. Alonso-González
    • 2
  • José Ranilla
    • 2
  1. 1.Depto. de Sistemas Informáticos y ComputaciónUniversitat Politècnica de ValènciaValenciaSpain
  2. 2.Depto. de InformáticaUniversidad de OviedoOviedoSpain
  3. 3.Telecommunication Engineering DepartmentUniversidad de JaénJaénSpain

Personalised recommendations