The Journal of Supercomputing

, Volume 73, Issue 1, pp 343–353 | Cite as

An efficient musical accompaniment parallel system for mobile devices

  • Pedro Alonso
  • P. Vera-Candeas
  • Raquel Cortina
  • José Ranilla
Article
  • 101 Downloads

Abstract

This work presents a software system designed to track the reproduction of a musical piece with the aim to match the score position into its symbolic representation on a digital sheet. Into this system, known as automated musical accompaniment system, the process of score alignment can be carried out real-time. A real-time score alignment, also known as score following, poses an important challenge due to the large amount of computation needed to process each digital frame and the very small time slot to process it. Moreover, the challenge is even greater since we are interested on handheld devices, i.e. devices characterized by both low power consumption and mobility. The results presented here show that it is possible to exploit efficiently several cores of an ARM® processor, or a GPU accelerator (presented in some SoCs from NVIDIA) reducing the processing time per frame under 10 ms in most of the cases.

Keywords

Audio-to-score alignment Score following Musical accompaniment Parallel computing Real-time computing 

References

  1. 1.
    Cont A, Schwarz D, Schnell N, Raphael C (2007) Evaluation of real- time audio-to-score alignment. In: Proc. of the International Conference on Music Information Retrieval (ISMIR) 2007, ViennaGoogle Scholar
  2. 2.
    Arzt A (2008) Score following with dynamic time warping. An automatic page-turner. Master’s Thesis, Vienna University of Technology, ViennaGoogle Scholar
  3. 3.
    Raphael C (2010) Music plus one and machine learning. In: Proc. of the 27 th International Conference on Machine Learning, Haifa, pp 21–28Google Scholar
  4. 4.
    Carabias-Ortí JJ, Rodríguez-Serrano FJ, Vera-Candeas P, Ruiz-Reyes N, Cañadas-Quesada FJ (2015) An audio to score alignment framework using spectral factorization and dynamic time warping. In: Proc. of the International Conference on Music Information Retrieval (ISMIR), Málaga, pp 742–748Google Scholar
  5. 5.
    Cont A (2010) A coupled duration-focused architecture for real-time music-to-score alignment. IEEE Trans. Pattern Anal. Mach. Intell. 32(6):974–987CrossRefGoogle Scholar
  6. 6.
    Montecchio N, Orio N (2009) A discrete filterbank approach to audio to score matching for score following. In: Proc. of the International Conference on Music Information Retrieval (ISMIR), pp 495–500Google Scholar
  7. 7.
    Puckette M (1995) Score following using the sung voice. In: Proc. of the International Computer Music Conference (ICMC), pp 175–178Google Scholar
  8. 8.
    Duan Z, Pardo B (2011) Soundprism: an online system for score-informed source separation of music audio. IEEE J. Sel. Top. Signal Process. 5(6):1205–1215CrossRefGoogle Scholar
  9. 9.
    Cont A (2006) Realtime audio to score alignment for polyphonic music instruments using sparse non-negative constraints and hierarchical hmms. In: Proc. of IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), ToulouseGoogle Scholar
  10. 10.
    Cuvillier P, Cont A (2014) Coherent time modeling of Semi-Markov models with application to realtime audio-to-score alignment. In Proc. of the 2014 IEEE International Workshop on Machine Learning for Signal Processing, p 16Google Scholar
  11. 11.
    Joder C, Essid S, Richard G (2013) Learning optimal features for polyphonic audio-to-score alignment. IEEE Trans. Audio Speech Lang. Process. 21(10):2118–2128CrossRefGoogle Scholar
  12. 12.
    Dixon S (2005) Live tracking of musical performances using on-line time warping. In: Proc. International Conference on Digital Audio Effects (DAFx), Madrid, pp 92–97Google Scholar
  13. 13.
    Hu N, Dannenberg RB, Tzanetakis G (2009) Polyphonic audio matching and alignment for music retrieval. In: Proc. of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), pp 185–188Google Scholar
  14. 14.
    Orio N, Schwarz D (2001) Alignment of monophonic and polyphonic music to a score. In: Proc. International Computer Music Conference (ICMC)Google Scholar
  15. 15.
    Alonso P, Cortina R, Rodríguez-Serrano FJ, Vera-Candeas P, Alonso-Gonzalez M, Ranilla J (2016) Parallel online time warping for real-time audio-to-score alignment in multi-core systems. J. Supercomput. doi:10.1007/s11227-016-1647-5 (published online)
  16. 16.
    Carabias-Ortí JJ, Rodríguez-Serrano FJ, Vera-Candeas P, Cañadas-Quesada FJ, Ruiz-Reyes N (2013) Constrained non-negative sparse coding using learnt instrument templates for realtime music transcription, Eng. Appl. Artif. Intell. 26(7):1671–1680CrossRefGoogle Scholar
  17. 17.
    Carabias-Ortí JJ, Rodríguez-Serrano FJ, Vera-Candeas P, Martínez-Muñoz D (2016) Tempo driven audio-to-score alignment using spectral decomposition and online dynamic time warping. ACM Trans. Intell. Syst. Technol. (accepted)Google Scholar
  18. 18.
    FFTW (2016) http://www.fftw.org. Accessed July 2016
  19. 19.
    NVIDIA CUDA Fast Fourier Transform library (cuFFT) (2016) http://developer.nvidia.com/cufft. Accessed July 2016
  20. 20.
    The OpenMP API specification for parallel programming (2016) http://openmp.org. Accessed July 2016

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Depto. de Sistemas Informáticos y ComputaciónUniversitat Politècnica de ValènciaValènciaSpain
  2. 2.Depto. de InformáticaUniversidad de OviedoGijónSpain
  3. 3.Telecommunication Engineering DepartmentUniversidad de JaénJaénSpain

Personalised recommendations