The Journal of Supercomputing

, Volume 75, Issue 3, pp 1001–1013 | Cite as

HReMAS: hybrid real-time musical alignment system

  • P. Cabañas-Molero
  • Raquel Cortina-ParajónEmail author
  • E. F. Combarro
  • Pedro Alonso
  • F. J. Bris-Peñalver


This paper presents a real-time audio-to-score alignment system for musical applications. The aim of these systems is to synchronize a live musical performance with its symbolic representation in a music sheet. We have used as a base our previous real-time alignment system by enhancing it with a traceback stage, a stage used in offline alignment to improve the accuracy of the aligned note. This stage introduces some delay, what forces to assume a trade-off between output delay and alignment accuracy that must be considered in the design of this type of hybrid techniques. We have also improved our former system to execute faster in order to minimize this delay. Other interesting improvements, like identification of silence frames, have also been incorporated to our proposed system.


Hybrid audio-to-score alignment Audio-to-score alignment Score following Dynamic time warping 



This work has been supported by the “Ministerio de Economía y Competitividad” of Spain and FEDER under Projects TEC2015-67387-C4-{1,2,3}-R.


  1. 1.
    Alonso P, Cortina R, Rodríguez-Serrano FJ, Vera-Candeas P, Alonso-González M, Ranilla J (2017) Parallel online time warping for real-time audio-to-score alignment in multi-core systems. J Supercomput 73(1):126–138CrossRefGoogle Scholar
  2. 2.
    Alonso P, Vera-Candeas P, Cortina R, Ranilla J (2017) An efficient musical accompaniment parallel system for mobile devices. J Supercomput 73(1):343–353CrossRefGoogle Scholar
  3. 3.
    Arzt A (2016) Flexible and robust music tracking. Ph.D. thesis, Johannes Kepler University Linz, Linz, ÖsterreichGoogle Scholar
  4. 4.
    Arzt A, Widmer G, Dixon S (2008) Automatic page turning for musicians via real-time machine listening. In: Proceedings of the 18th European Conference on Artificial Intelligence (ECAI), Amsterdam, pp 241–245Google Scholar
  5. 5.
    Carabias-Orti J, Rodríguez-Serrano F, Vera-Candeas P, Ruiz-Reyes N, Cañadas-Quesada F (2015) An audio to score alignment framework using spectral factorization and dynamic time warping. In: Proceedings of ISMIR, pp 742–748Google Scholar
  6. 6.
    Cont A (2006) Realtime audio to score alignment for polyphonic music instruments, using sparse non-negative constraints and hierarchical HMMs. In: 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, vol 5. pp V–VGoogle Scholar
  7. 7.
    Cont A, Schwarz D, Schnell N, Raphael C (2007) Evaluation of real-time audio-to-score alignment. In: International Symposium on Music Information Retrieval (ISMIR), ViennaGoogle Scholar
  8. 8.
    Dannenberg RB, Raphael C (2006) Music score alignment and computer accompaniment. Commun ACM 49(8):38–43CrossRefGoogle Scholar
  9. 9.
    Devaney J, Ellis D (2009) Handling asynchrony in audio-score alignment. In: Proceedings of the International Computer Music Conference Computer Music Association. pp 29–32Google Scholar
  10. 10.
    Dixon S (2005) An on-line time warping algorithm for tracking musical performances. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). pp 1727–1728Google Scholar
  11. 11.
    Duan Z, Pardo B (2011) Soundprism: an online system for score-informed source separation of music audio. IEEE J Sel Top Signal Process 5(6):1205–1215CrossRefGoogle Scholar
  12. 12.
    Ewert S, Muller M, Grosche P (2009) High resolution audio synchronization using chroma onset features. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2009 (ICASSP 2009). pp 1869–1872Google Scholar
  13. 13.
    Hu N, Dannenberg R, Tzanetakis G (2003) Polyphonic audio matching and alignment for music retrieval. In: 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. pp 185–188Google Scholar
  14. 14.
    Kaprykowsky H, Rodet X (2006) Globally optimal short-time dynamic time warping, application to score to audio alignment. In: Proceedings of the International Conference on Acoustics, Speech and Signal Processing, vol 5. pp. V–VGoogle Scholar
  15. 15.
    Li B, Duan Z (2016) An approach to score following for piano performances with the sustained effect. IEEE/ACM Trans Audio Speech Lang Process 24(12):2425–2438CrossRefGoogle Scholar
  16. 16.
    Miron M, Carabias-Orti JJ, Bosch JJ, Gómez E, Janer J (2016) Score-informed source separation for multichannel orchestral recordings. J Electr Comput Eng 2016(8363507):1–19Google Scholar
  17. 17.
    Muñoz-Montoro A, Cabañas-Molero P, Bris-Peñalver F, Combarro E, Cortina R, Alonso P (2017) Discovering the composition of audio files by audio-to-midi alignment. In: Proceedings of the 17th International Conference on Computational and Mathematical Methods in Science and Engineering. pp 1522–1529Google Scholar
  18. 18.
    Orio N, Schwarz D (2001) Alignment of monophonic and polyphonic music to a score. In: Proceedings of the International Computer Music Conference (ICMC), pp 155–158Google Scholar
  19. 19.
    Pätynen J, Pulkki V, Lokki T (2008) Anechoic recording system for symphony orchestra. Acta Acust United Acust 94(6):856–865CrossRefGoogle Scholar
  20. 20.
    Raphael C (2010) Music plus one and machine learning. In: Proceedings of the 27th International Conference on Machine Learning (ICML), pp 21–28Google Scholar
  21. 21.
    Rodriguez-Serrano FJ, Carabias-Orti JJ, Vera-Candeas P, Martinez-Munoz D (2016) Tempo driven audio-to-score alignment using spectral decomposition and online dynamic time warping. ACM Trans Intell Syst Technol 8(2):22:1–22:20CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • P. Cabañas-Molero
    • 1
  • Raquel Cortina-Parajón
    • 2
    Email author
  • E. F. Combarro
    • 2
  • Pedro Alonso
    • 3
  • F. J. Bris-Peñalver
    • 1
  1. 1.Department of Telecommunication EngineeringUniversity of JaénLinaresSpain
  2. 2.Depto. de InformáticaUniversidad de OviedoGijónSpain
  3. 3.Depto. de Sistemas Informáticos y ComputaciónUniversitat Politècnica de ValènciaValenciaSpain

Personalised recommendations