Skip to main content

A Real-Time Drum-Wise Volume Visualization System for Learning Volume-Balanced Drum Performance

  • 637 Accesses

Part of the Lecture Notes in Computer Science book series (LNISA,volume 13056)

Abstract

To improve drum performance, it is important to consider the volume balance of the bass drum, snare drum, and hi-hat. However, it is difficult for players to evaluate the balance of each volume of these instruments while playing. In addition, while it is possible to self-diagnose by recording drum performances, it is not always efficient to re-record and re-play drum performances based on the correction points. Therefore, we developed a system that uses semi-supervised non-negative matrix factorization (SSNMF) to separate a player’s drum performance, recorded with a unidirectional microphone installed in front of the drum kit, into the bass drum, snare drum, and hi-hat sound sources in real-time, and estimates each volume at the time of beating. In addition, the system also visualizes their volume balance and enables the player to control the power of beating. We experimented using this system for actual drum performance and clarified its usefulness and points for improvement based on the feedback obtained from the experiment participants.

Keywords

  • Drum performance
  • Sound source separation
  • Volume balance

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-89394-1_12
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-89394-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.

References

  1. Smoliar, S., Waterworth, J., Kellock, P.: PianoFORTE: a system for piano education beyond notation literacy. In: Proceedings of the 3rd ACM International Conference on Multimedia, pp. 457–465 (1995)

    Google Scholar 

  2. Rogers, K., et al.: P.I.A.N.O.: Faster piano learning with interactive projection. Proceedings of the 9th ACM International Conference on Interactive Tabletops and Surfaces, pp. 149–158 (2014)

    Google Scholar 

  3. Beici, L., György, F., Mark, S.: Measurement, recognition, and visualization of piano pedaling gestures and techniques. J. Audio Eng. Soc. 66(6), 448–456 (2018)

    CrossRef  Google Scholar 

  4. Marky, K., Weiß, A., Müller, F., Schmitz, M., Mühlhäuser, M., Kosch, T.: Let’s frets! Mastering guitar playing with capacitive sensing and visual guidance. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, vol. 169, pp. 1–4. New York, NY, USA (2021)

    Google Scholar 

  5. Doi, M., Miyashita, H.: Koto learning support method considering articulations. In: Cheok, A.D., Inami, M., Romão, T. (eds.) ACE 2017. LNCS, vol. 10714, pp. 368–383. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76270-8_26

    CrossRef  Google Scholar 

  6. Nishimoto, K., Ikenoue, A., Unoki, M.: iDAF-drum: supporting practice of drumstick control by exploiting insignificantly delayed auditory feedback. In: Kunifuji, S., Papadopoulos, G.A., Skulimowski, A.M.J., Kacprzyk, J. (eds.) Knowledge, Information and Creativity Support Systems. AISC, vol. 416, pp. 483–497. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-27478-2_36

    CrossRef  Google Scholar 

  7. Holland, S., Bouwer, A. J., Dalgelish, M., Hurtig, T.M.: Feeling the beat where it counts: fostering multi-limb rhythm skills with the haptic drum kit. In: Proceedings of the 4th International Conference on Tangible and Embedded Interaction, pp. 21–28 (2010)

    Google Scholar 

  8. Imada, Y., Ochi, Y.: Hitting arm detection for drum performance learning system using kinect. In: The Seventh International Conference on Informatics and Applications (ICIA2018), pp. 8–10 (2018)

    Google Scholar 

  9. Iwami, N., Miura, M.: A support system for basic practice of playing the drums. In: International Computer Music Conference, pp. 364–367 (2007)

    Google Scholar 

  10. Kavalerov, I., et al.: Universal sound separation. In: IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 175–179 (2019)

    Google Scholar 

  11. Ono, N., et al.: Harmonic and Percussive Sound Separation and its Application to MIR-related Tasks. In: Raś, Z.W., Wieczorkowska, A.A. (eds.) Advances in Music Information Retrieval, pp. 213–236. Springer Berlin Heidelberg, Berlin, Heidelberg (2010). https://doi.org/10.1007/978-3-642-11674-2_10

    CrossRef  Google Scholar 

  12. Grais, E.M., Sen, M.U., Erdogan, H.: Deep neural networks for single channel source separation. In: ICASSP, pp. 3734–3738 (2014)

    Google Scholar 

  13. Hennequin, R., Khlif, A., Voituret, F., Manuel, M.: Spleeter: a fast and efficient music source separation tool with pre-trained models. J. Open Source Softw. 5(50), 2154 (2020)

    CrossRef  Google Scholar 

  14. Stöter, F.R., Uhlich, S., Liutkus, A., Mitsufuji, Y.: Open-Unmix – a reference implementation for music source separation. J. Open Source Softw. 4(41), 1667 (2019)

    CrossRef  Google Scholar 

  15. Lee, D.D., Seung, H.S.: Learning the parts of objects with non-negative matrix factorization. Nature 401, 788–791 (1999)

    CrossRef  Google Scholar 

  16. Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.: Non-negative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley, New York (2009)

    CrossRef  Google Scholar 

  17. Smaragdis, P.: Non-negative matrix factor deconvolution; extraction of multiple sound sources from monophonic inputs. In: Puntonet, C.G., Prieto, A. (eds.) ICA 2004. LNCS, vol. 3195, pp. 494–499. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30110-3_63

    CrossRef  Google Scholar 

  18. Ueda, S., Shibata, K., Wada, Y., Nishikimi, R., Nakamura, E., Yoshii, K.: Bayesian drum transcription based on non-negative matrix factor decomposition with a deep score prior. In: ICASSP, pp. 456–460 (2019)

    Google Scholar 

  19. Paulus, J., Virtanen, T.: Drum transcription with non-negative spectrogram factorisation. In: European Signal Processing Conference (EUSIPCO) (2005)

    Google Scholar 

  20. Dittmar, C., Gärtner, D.: Real-time transcription and separation of drum recordings based on NMF decomposition. In: International Conference on Digital Audio Effects (DAFX), pp. 187–194 (2014)

    Google Scholar 

  21. Wu, C., Lerch, A.: Drum transcription using partially fixed non-negative matrix factorization with template adaptation. In: International Society for Music Information Retrieval Conference (ISMIR), pp. 257–263 (2015)

    Google Scholar 

  22. Vogl, R., Widmer, G., Knees, P.: Towards multi-instrument drum transcription. In: Proceedings of the 21st International Conference on Digital Audio Effects (DAFx-18) (2018)

    Google Scholar 

  23. Böck, S., Widmer, G.: Maximum filter vibrato suppression for onset detection. In: International Conference on Digital Audio Effects (DAFX) (2013)

    Google Scholar 

  24. Lee, H., Yoo, J., Choi, S.: Semi-supervised non-negative matrix factorization. IEEE Signal Process. Lett. 17(1), 4–7 (2010)

    CrossRef  Google Scholar 

  25. C 1509-1:2017 (IEC 61672-1:2013). http://kikakurui.com/c1/C1509-1-2017-01.html (2021). Accessed 26 July 2021

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mitsuki Hosoya .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 IFIP International Federation for Information Processing

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Hosoya, M., Morise, M., Nakamura, S., Yoshii, K. (2021). A Real-Time Drum-Wise Volume Visualization System for Learning Volume-Balanced Drum Performance. In: Baalsrud Hauge, J., C. S. Cardoso, J., Roque, L., Gonzalez-Calero, P.A. (eds) Entertainment Computing – ICEC 2021. ICEC 2021. Lecture Notes in Computer Science(), vol 13056. Springer, Cham. https://doi.org/10.1007/978-3-030-89394-1_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-89394-1_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-89393-4

  • Online ISBN: 978-3-030-89394-1

  • eBook Packages: Computer ScienceComputer Science (R0)