Abstract
We developed a computational evaluation method for piano performance with the goal of building a practice support system for beginners. We recorded students’ performances as audio data and applied several recent methods for audio-to-MIDI transcription based on deep neural networks to extract the pitch, onset time, and offset time of musical notes. To determine the correctness of the performance, we aligned the extracted MIDI data with the musical score using a hidden Markov model (HMM). We compared the audio-to-MIDI transcription methods and optimized the weight on different types of performance errors to conform to teacher’s assessment. Our experiments showed a strong correlation between the rate of performance errors obtained from the alignment and the evaluation by a teacher who listened to the performance. The results that indicate performance errors and tempo stability can be used in a practice support system that provides feedback to learners.
This work was supported by JSPS KAKENHI Grant Numbers 21K02846, 21K12187, 22H03661.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Deja, J.A.: Piano learning and improvisation through adaptive visualisation and digital augmentation. In: Companion Proceedings of the 2022 Conference on Interactive Surfaces and Spaces, pp. 41–45 (2022)
Dorfman, J.: Theory and Practice of Technology-based Music Instruction. Oxford University Press, Oxford (2022)
Fukuda, T., Ikemiya, Y., Itoyama, K., Yoshii, K.: “A score-informed piano tutoring system with mistake detection and score simplification” within the music education contexts. In: Proceedings of the 12th Sound and Music Computing Conference (SMC), vol. 1, pp. 105–110 (2015)
Heyen, F., Ngo, Q.Q., Kurzhals, K., Sedlmair, M.: Data-driven visual reflection on music instrument practice. In: ACM CHI Conference on Human Factors in Computing Systems (2022)
Kim, H., Ramoneda, P., Miron, M., Serra, X.: An overview of automatic piano performance assessment within the music education contexts. Proc. Int. Soc. Music Inf. Retrieval 1, 465–474 (2017)
Kong, Q., Li, B., Song, X., Wan, Y., Wang, Y.: High-resolution piano transcription with pedals by regressing onset and offset times. IEEE/ACM Trans. Audio Speech Lang. Process. 29, 3707–3717 (2021)
Lerch, A., Arthur, C., Pati, A., Gururani, S.: An interdisciplinary review of music performance analysis. Trans. Int. Soc. Music Inf. Retrieval 3(1), 221–245 (2021)
Lima, H.B., Santos, C.G.R.D., Meiguins, B.S.: A survey of music visualization techniques. ACM Comput. Surv. (CSUR) 57(7), 1–29 (2022)
Nakamura, E., Yoshii, K., Katayose, H.: Performance error detection and post-processing for fast and accurate symbolic music alignment. IN: Proceedings of the International Society for Music Information Retrieval, pp. 347–353 (2017)
Shibata, K., Nakamura, E., Yoshi, K.: Non-local musical statistics as guides for audio-to-score piano transcription. Inf. Sci. 566, 262–280 (2021)
Wang, W., Pan, J., Yi, H., Song, Z., Li, M.: Audio-based piano performance evaluation for beginners with convolutional neural network and attention mechanism. IEEE/ACM Trans. Audio Speech Lang. Process. 29, 1119–1133 (2021)
Wu, C.W., Gururani, S., Pati, A., Vidwans, A.: Towards the objective assessment of music performances. In: International Conference on Music Perception and Cognition (ICMPC), pp. 99–103 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kato, N., Nakamura, E., Mine, K., Doeda, O., Yamada, M. (2023). Computational Analysis of Audio Recordings of Piano Performance for Automatic Evaluation. In: Viberg, O., Jivet, I., Muñoz-Merino, P., Perifanou, M., Papathoma, T. (eds) Responsive and Sustainable Educational Futures. EC-TEL 2023. Lecture Notes in Computer Science, vol 14200. Springer, Cham. https://doi.org/10.1007/978-3-031-42682-7_46
Download citation
DOI: https://doi.org/10.1007/978-3-031-42682-7_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42681-0
Online ISBN: 978-3-031-42682-7
eBook Packages: Computer ScienceComputer Science (R0)