Skip to main content
SpringerLink
Log in
Book cover

International Conference on Computational Intelligence in Music, Sound, Art and Design (Part of EvoStar)

EvoMUSART 2022: Artificial Intelligence in Music, Sound, Art and Design pp 131–147Cite as

Towards the Generation of Musical Explanations with GPT-3

Part of the Lecture Notes in Computer Science book series (LNCS,volume 13221)

Abstract

Open AI’s language model, GPT-3, has shown great potential for many NLP tasks, with applications in many different domains. In this work we carry out a first study on GPT-3’s capability to communicate musical decisions through textual explanations when prompted with a textual representation of a piece of music. Enabling a dialogue in human-AI music partnerships is an important step towards more engaging and creative human-AI interactions. Our results show that GPT-3 lacks the necessary intelligence to really understand musical decisions. A major barrier to reach a better performance is the lack of data that includes explanations of the creative process carried out by artists for musical pieces. We believe such a resource would aid the understanding and collaboration with AI music systems.

Keywords

  • Explainability
  • GPT3
  • Music

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://openai.com/blog/gpt-3-apps/?utm_campaign=.

  2. 2.

    https://openai.com/blog/musenet/.

  3. 3.

    OpenEWLD dataset: https://github.com/00sapo/OpenEWLD.

  4. 4.

    Study Data: https://github.com/sjkrol/GPT3MusicalExplanationsData.

  5. 5.

    https://www.youtube.com/watch?v=vqNOTcBfjUM.

References

  1. Abdul, A., Vermeulen, J., Wang, D., Lim, B.Y., Kankanhalli, M.: Trends and trajectories for explainable, accountable and intelligible systems: an HCI research agenda, pp. 1–18. Association for Computing Machinery (2018)

    Google Scholar 

  2. Agrawal, Y., Shanker, R.G.R., Alluri, V.: Transformer-based approach towards music emotion recognition from lyrics. In: Hiemstra, D., Moens, M.-F., Mothe, J., Perego, R., Potthast, M., Sebastiani, F. (eds.) ECIR 2021. LNCS, vol. 12657, pp. 167–175. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72240-1_12

    CrossRef  Google Scholar 

  3. Anjomshoae, S., Najjar, A., Calvaresi, D., Främling, K.: Explainable agents and robots: results from a systematic literature review. In: 18th International Conference on Autonomous Agents and Multiagent Systems, pp. 1078–1088 (2019)

    Google Scholar 

  4. Bacco, L., Cimino, A., Dell’Orletta, F., Merone, M.: Explainable sentiment analysis: a hierarchical transformer-based extractive summarization approach. Electronics 10(18), 2195 (2021)

    CrossRef  Google Scholar 

  5. Bishop, L., Cancino-Chacón, C., Goebl, W.: Moving to communicate, moving to interact: patterns of body motion in musical duo performance. Music. Percept. 37(1), 1–25 (2019)

    CrossRef  Google Scholar 

  6. Bishop, L., Goebl, W.: Beating time: How ensemble musicians’ cueing gestures communicate beat position and tempo. Psychol. Music 46(1), 84–106 (2018)

    CrossRef  Google Scholar 

  7. Bretan, P.M.: Towards an embodied musical mind: generative algorithms for robotic musicians. Ph.D. thesis, Georgia Institute of Technology (2017)

    Google Scholar 

  8. Brown, T., et al..: Language models are few-shot learners. In: Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901. Curran Associates, Inc. (2020)

    Google Scholar 

  9. Chris, W.: abcnotation.com (2021). https://abcnotation.com/

  10. d’Eon, J., Dumpala, S.H., Sastry, C.S., Oore, D., Oore, S.: Musical speech: a transformer-based composition tool. In: Proceedings of Machine Learning Research, NeurIPS 2020, vol. 133, pp. 253–274. PMLR (2020)

    Google Scholar 

  11. Geerlings, C., Meroño-Peñuela, A.: Interacting with GPT-2 to generate controlled and believable musical sequences in ABC notation. In: Proceedings of the 1st Workshop on NLP for Music and Audio, NLP4MUSA (2020)

    Google Scholar 

  12. Gonsalves, R.A.: AI-Tunes: creating new songs with artificial intelligence. Medium. Online article. https://towardsdatascience.com/ai-tunes-creating-new-songs-with-artificial-intelligence-4fb383218146. Accessed Sept 2021

  13. Guzdial, M., Reno, J., Chen, J., Smith, G., Riedl, M.: Explainable PCGML via game design patterns. In: Zhu, J. (ed.) Joint Proceedings of the AIIDE 2018 Workshops co-located with 14th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE 2018, vol. 2282. CEUR-WS.org (2018)

    Google Scholar 

  14. Hoffman, G., Weinberg, G.: Interactive improvisation with a robotic marimba player. Auton. Robot. 31(2), 133–153 (2011)

    CrossRef  Google Scholar 

  15. Hsu, J.L., Chang, S.J.: Generating music transition by using a transformer-based model. Electronics 10(18), 2276 (2021)

    CrossRef  Google Scholar 

  16. Ji, K., Yang, D., Tsai, T.J.: Instrument classification of solo sheet music images. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2021, pp. 546–550. IEEE (2021)

    Google Scholar 

  17. Kovaleva, O., Romanov, A., Rogers, A., Rumshisky, A.: Revealing the dark secrets of BERT. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 4364–4373. ACL (2019)

    Google Scholar 

  18. Llano, M.T., et al.: Explainable computational creativity (2020)

    Google Scholar 

  19. McCormack, J., Gifford, T., Hutchings, P., Llano Rodriguez, M.T., Yee-King, M., d’Inverno, M.: In a silent way: Communication between AI and improvising musicians beyond sound. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–11 (2019)

    Google Scholar 

  20. McCormack, J., Hutchings, P., Gifford, T., Yee-King, M., Llano, M.T., D’inverno, M.: Design considerations for real-time collaboration with creative artificial intelligence. Organised Sound 25(1), 41–52 (2020)

    Google Scholar 

  21. Santoro, J.: Is this the future of music? GPT3-powered musical assistant. Medium. Online article. https://medium.com/swlh/is-this-the-future-of-music-gpt3-powered-musical-assistant-109569e6092c. Accessed Jan 2021

  22. Schuff, H., Yang, H.Y., Adel, H., Vu, N.T.: Does external knowledge help explainable natural language inference? Automatic evaluation vs. human ratings (2021)

    Google Scholar 

  23. Tsai, T., Ji, K.: Composer style classification of piano sheet music images using language model pretraining. In: Proceedings of the 21th International Society for Music Information Retrieval Conference, ISMIR 2020, pp. 176–183 (2020)

    Google Scholar 

  24. Vig, J.: A multiscale visualization of attention in the transformer model. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 37–42. ACL (2019)

    Google Scholar 

  25. Wim, V.: xml2abc (2012). https://wim.vree.org/svgParse/xml2abc.html

  26. Zhu, J., Liapis, A., Risi, S., Bidarra, R., Youngblood, G.M.: Explainable AI for designers: a human-centered perspective on mixed-initiative co-creation. In: 2018 IEEE Conference on Computational Intelligence and Games (CIG), pp. 1–8 (2018)

    Google Scholar 

Download references

Acknowledgements

The work presented here was funded by an Early Career Researcher Seed grant awarded by the Faculty of IT at Monash University.

Author information

Authors and Affiliations

  1. Monash University, Melbourne, Victoria, Australia

    Stephen James Krol, Maria Teresa Llano & Jon McCormack

Authors
  1. Stephen James Krol
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maria Teresa Llano
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jon McCormack
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephen James Krol .

Editor information

Editors and Affiliations

  1. University of Coimbra, Coimbra, Portugal

    Tiago Martins

  2. University of A Coruña, A Coruña, Spain

    Nereida Rodríguez-Fernández

  3. University of Coimbra, Coimbra, Portugal

    Sérgio M. Rebelo

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Krol, S.J., Llano, M.T., McCormack, J. (2022). Towards the Generation of Musical Explanations with GPT-3. In: Martins, T., Rodríguez-Fernández, N., Rebelo, S.M. (eds) Artificial Intelligence in Music, Sound, Art and Design. EvoMUSART 2022. Lecture Notes in Computer Science, vol 13221. Springer, Cham. https://doi.org/10.1007/978-3-031-03789-4_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-03789-4_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-03788-7

  • Online ISBN: 978-3-031-03789-4

  • eBook Packages: Computer ScienceComputer Science (R0)

search

Navigation