Advertisement

Ryry: A Real-Time Score-Following Automatic Accompaniment Playback System Capable of Real Performances with Errors, Repeats and Jumps

  • Shinji Sako
  • Ryuichi Yamamoto
  • Tadashi Kitamura
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8610)

Abstract

In this work, we propose an automatic accompaniment playback system called Ryry, which follows human performance and plays a corresponding accompaniment automatically, in an attempt to realize human-computer concerts. Recognizing and anticipating the score position in real-time, known as score following, by a computer is difficult. The proposed system is based on a robust on-line algorithm for real-time audio-to-score alignment. The algorithm is devised using a delayed-decision and anticipation framework by modeling real-time music performance that includes uncertainties such as tempo fluctuation and mistakes. We developed an automatic accompaniment system that is capable of generating polyphonic music signals.

Keywords

Score following Automatic accompaniment Segmental Conditional Random Fields (SCRFs) Linear Dynamic System (LDS) Chord transition 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Dannenberg, R.B., Raphael, C.: Music score alignment and computer accompaniment. Communications of the ACM 49(8), 38–43 (2006)CrossRefGoogle Scholar
  2. 2.
    Cont, A.: Antescofo: Anticipatory synchronization and control of interactive parameters in computer music. In: Proc. of International Computer Music Conference (ICMC) (2008)Google Scholar
  3. 3.
    Cano, P., Loscos, A., Bonada, J.: Score-performance matching using hmms. In: Proc. of International Computer Music Conference (ICMC), pp. 441–444 (1999)Google Scholar
  4. 4.
    Orio, N., Dechelle, F.: Score following using spectral analysis and hidden markov models. In: Proc. of International Computer Music Conference (ICMC), pp. 151–154 (2001)Google Scholar
  5. 5.
    Dixon, S.: Live tracking of musical performances using on–line time warping. In: Proc. of International Conference on Digital Audio Effects (DAFx), pp. 92–97 (2005)Google Scholar
  6. 6.
    Otsuka, T., Nakadai, K., Takahashi, T., Ogata, T., Okuno, H.G.: Real-time audio-to-score alignment using particle filter for coplayer music robots. EURASIP Journal on Advances in Signal Processing, 1–13 (2011)Google Scholar
  7. 7.
    Duan, Z., Pardo, B.: A state space model for online polyphonic audio-score alignment. In: Proc. of International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 197–200 (2011)Google Scholar
  8. 8.
    Montecchio, N., Cont, A.: A unified approach to real time audio-to-score and audio-to-audio alignment using sequential montecarlo inference techniques. In: Proc. of International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 193–196 (2011)Google Scholar
  9. 9.
    Raphael, C.: Aligning music audio with symbolic scores using a hybrid graphical model. Machine Learning Journal 65(2-3), 389–409 (2006)CrossRefGoogle Scholar
  10. 10.
    Cont, A.: A coupled duration-focused architecture for realtime music to score alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(6), 974–987 (2010)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Arzt, A., Widmer, G., Dixon, S.: Automatic page turning for musicians via real-time machine listening. In: Proc. of European Conference on Artificial Intelligence (ECAI), pp. 241–245 (2008)Google Scholar
  12. 12.
    Joder, C., Essid, S., Richard, G.: A conditional random field framework for robust and scalable audio-to-score matching. IEEE Transactions on Audio, Speech and Language Processing 19(8), 2385–2397 (2011)CrossRefGoogle Scholar
  13. 13.
    Joder, C., Essid, S., Richard, G.: A conditional random field viewpoint of symbolic audio-to-score matching. In: Proc. of ACM Multimedia, pp. 871–874 (2010)Google Scholar
  14. 14.
    Joder, C., Essid, S., Richard, G.: A comparative study of tonal acoustic features for a symbolic level music-to-score alignment. In: Proc. of International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 409–412 (2010)Google Scholar
  15. 15.
    Dixon, S.: Onset detection revisited. In: Proc. of International Conference on Digital Audio Effects (DAFx), pp. 133–137 (2006)Google Scholar
  16. 16.
    Hu, N., Dannenberg, R.B., Tzanetakis, G.: Polyphonic audio matching and alignment for music retrieval. In: Proc. of IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 185–188 (2003)Google Scholar
  17. 17.
    Emiya, V., Badeau, R., David, B.: Multipitch estimation of piano sounds using a new probabilistic spectral smoothness principle. IEEE Transactions on Audio, Speech and Language Processing 18(6), 1643–1654 (2010)CrossRefGoogle Scholar
  18. 18.
    Goto, M., Hashiguchi, H., Nishimura, T., Oka, R.: RWC music database: Popular, classical, and jazz music databases. In: Proc. of International Conference on Music Information Retrieval (ISMIR), pp. 287–288 (2002)Google Scholar
  19. 19.
    (IMIRSEL), T.I.M.I.R.S.E.L.: Real-time audio to score alignment (a.k.a score following), http://www.music-ir.org/mirex/wiki/Real-time_Audio_to_Score_Alignment_a.k.a_Score_Following (accessed May 27, 2014)

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Shinji Sako
    • 1
  • Ryuichi Yamamoto
    • 1
  • Tadashi Kitamura
    • 1
  1. 1.Nagoya Institute of TechnologyShowa-kuJapan

Personalised recommendations