Skip to main content
Log in

A comprehensive overview of AI-enabled music classification and its influence in games

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

With the development and advancement of information technology, artificial intelligence (AI) and machine learning are applied in every sector of life. Among these applications, music is one that has gained attention in the last couple of years. AI-based innovative and intelligent techniques are revolutionising the music industry. It is very convenient for composers to compose music of high quality using these technologies. Artificial intelligence and music (AIM) is one of the emerging fields used to generate and manage sounds for different media like the Internet, games, etc. Sound effects in games are very effective and can be made more attractive by implementing AI approaches. The quality of the sounds in the game directly impacts the productivity and experience of the player. With computer-assisted technologies, game designers can create sounds for different scenarios or situations like horror and suspense and provide gamers with information. The practical and productive audio of a game can guide visually impaired people during other events in the game. For the better creation and composition of music, a good quality of knowledge about musicology is essential. Due to AIM, there are a lot of intelligent and interactive tools available for the efficient and effective learning of music. Learners can be provided with a very reliable and interactive environment based on artificial intelligence. The current study has considered presenting a detailed overview of the literature available in the area of research. The study has demonstrated literature analysis from various perspectives, which will provide evidence for researchers to devise novel solutions in the field.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Ahmed HOK (2016) Flipped learning as a new educational paradigm: an analytical critical study. Eur Sci J 12(10)

  • Arditi D (2019) Music everywhere: setting a digital music trap. Crit Sociol 45(4–5):617–630

    Article  Google Scholar 

  • Arutyunov V, Averkin A (2017) Genetic algorithms for music variation on genom platform. Procedia Comput Sci 120:317–324

    Article  Google Scholar 

  • Aska A (2017) Introduction to the study of video game music. Lulu. com

  • Avdeeff M (2019) Artificial intelligence & popular music: SKYGGE, flow machines, and the audio uncanny valley. Arts 8(4):130

    Article  Google Scholar 

  • Birtchnell T (2018) Listening without ears: artificial intelligence in audio mastering. Big Data Soc 5(2):2053951718808553

    Article  Google Scholar 

  • Böttcher N (2013) Current problems and future possibilities of procedural audio in computer games. J Gaming Virtual Worlds 5(3):215–234

    Article  Google Scholar 

  • Brown A, Kerr T (2009) Adaptive music techniques. In: Proceedings of improvise: the Australasian computer music conference 2009. Australasian Computer Music Association, pp 26–31

  • Camurri A (1990) On the role of artificial intelligence in music research. J New Music Res 19(2–3):219–248

    Google Scholar 

  • Collins K (2009) An introduction to procedural music in video games. Contemp Music Rev 28(1):5–15

    Article  Google Scholar 

  • Dannenberg RB (1993) Music representation issues, techniques, and systems. Comput Music J 17(3):20–30

    Article  Google Scholar 

  • Dannenberg RB (2000) Artificial intelligence, machine learning, and music understanding. In: Proceedings of the 2000 Brazilian symposium on computer music: arquivos do simpsio brasileiro de computao musical (SBCM)

  • De Mantaras RL, Arcos JL (2002) AI and music: From composition to expressive performance. AI Mag 23(3):43–43

    Google Scholar 

  • Dixon S (2000) On the computer recognition of solo piano music. In: Proceedings of Australasian computer music conference, pp 31–37

  • Elowsson A, Friberg A (2012) Algorithmic composition of popular music. In: The 12th international conference on music perception and cognition and the 8th triennial conference of the European society for the cognitive sciences of music, pp 276–285

  • Ekman I (2005) Meaningful noise: understanding sound effects in computer games. Proc Digital Arts Cul 17

  • Forsberg A, Dieterich M, Zeleznik R (1998) The music notepad. In: Proceedings of the 11th annual ACM symposium on User interface software and technology, pp 203–210

  • Fox R, Khan A (2013) Artificial intelligence approaches to music composition. In: Proceedings on the international conference on artificial intelligence (ICAI). The Steering Committee of The World Congress in Computer Science, Computer, p 1

  • Frid E, Gomes C, Jin Z (2020) Music creation by example. In: Proceedings of the 2020 CHI conference on human factors in computing systems, pp 1–13

  • Gärdenfors D (2003) Designing sound-based computer games. Digital Creat 14(2):111–114

    Article  Google Scholar 

  • Ghildiyal A, Sharma S (2021) Music genre classification using data filtering algorithm: an artificial intelligence approach. In: 2021 third international conference on inventive research in computing applications (ICIRCA). IEEE, pp 922–926

  • Gorbunova IB, Govorova AA (2018) Music computer technologies as a means of teaching the musical art for visually-impaired people. In: Int'l conference proceedings, pp 19–22

  • Grimshaw M, Tan S-L, Lipscomb SD (2013) Playing with sound: the role of music and sound effects in gaming. Am Psychol Assoc

  • Guéret C, Monmarché N, Slimane M (2004) Ants can play music. In: International workshop on ant colony optimization and swarm intelligence. Springer, pp 310–317

  • Gunawan AA, Suhartono D (2019) Music recommender system based on genre using convolutional recurrent neural networks. Procedia Comput Sci 157:99–109

    Article  Google Scholar 

  • Gungormusler A, Paterson-Paulberg N, Haahr M (2015) barelymusician: an adaptive music engine for video games. In: Audio engineering society conference: 56th international conference: audio for games. Audio Engineering Society

  • Hébert S, Béland R, Dionne-Fournelle O, Crête M, Lupien SJ (2005) Physiological stress response to video-game playing: the contribution of built-in music. Life Sci 76(20):2371–2380

    Article  Google Scholar 

  • Hou Y (2020) Discussion on practice and promotion strategy of college music education informatization based on artificial intelligence. In: 2020 international conference on computers, information processing and advanced education (CIPAE). IEEE, pp 114–116

  • Jørgensen K (2008) Left in the dark: playing computer games with the sound turned off. Ashgate

    Google Scholar 

  • Karageorghis CI, Bigliassi M, Tayara K, Priest D-L, Bird JM (2018) A grounded theory of music use in the psychological preparation of academy soccer players. Sport Exercise Perf Psychol 7(2):109

    Google Scholar 

  • Kazi N, Bhatia S (2013) Various artificial intelligence techniques for automated melody generation. Int J Eng Res 2:1646–1652

    Google Scholar 

  • Kumar L, Goyal P, Kumar R (2020) Creativity in machines: music composition using artificial intelligence. Asian J Converg Technol 6(2):36–40

    Article  Google Scholar 

  • Lee J-H (2012) A study of the changes of game music. J Korea Game Soc 12(1):103–111

    Article  Google Scholar 

  • Lee M, Choi H, Cho D, Lee H (2016) Cannibalizing or complementing?. The impact of online streaming services on music record sales. Procedia Comput Sci 91:662–671

    Article  Google Scholar 

  • Li T, Ogihara M (2003) Detecting emotion in music. In: International conference on music information retrieval

  • Linek SB, Marte B, Albert D (2011) Background music in educational games: Motivational appeal and cognitive impact. Int J Game-Based Learn 1(3):53–64

    Article  Google Scholar 

  • Lopez-Rincon O, Starostenko O, Ayala-San Martín G (2018) Algoritmic music composition based on artificial intelligence: a survey. In: 2018 international conference on electronics, communications and computers (CONIELECOMP). IEEE, pp 187–193

  • Louie R, Coenen A, Huang CZ, Terry M, Cai CJ (2020) Novice-AI music co-creation via AI-steering tools for deep generative models. In: Proceedings of the 2020 CHI conference on human factors in computing systems, pp 1–13

  • Lupker JA, Turkel WJ (2021) Music theory, the missing link between music-related big data and artificial intelligence. DHQ Digital Hum Q 15(1)

  • Malgaonkar S, Nag YB, Devadiga R, Hirave T (2013) An AI based intelligent music composing algorithm: concord. In: 2013 international conference on advances in technology and engineering (ICATE). IEEE, pp 1–6

  • Monteith K, Martinez TR, Ventura D (2010) Automatic generation of music for inducing emotive response. In: ICCC. Citeseer, pp 140–149

  • Müller M, Konz V, Bogler W, Arifi-Müller V (2011) Saarland music data (SMD). In: Proceedings of the international society for music information retrieval conference (ISMIR): late breaking session

  • Neeman S (1996) Arts, computers and artificial intelligence. In: Technology-based re-engineering engineering education proceedings of frontiers in education FIE'96 26th annual conference. IEEE, vol 3, pp 1397–1401

  • Newcomb DL (2012) The fundamentals of the video game music genre. James Madison University

    Google Scholar 

  • Ng P, Nesbitt K (2013) Informative sound design in video games. In: Proceedings of the 9th Australasian conference on interactive entertainment: matters of life and death, pp 1–9

  • Nicholls S, Cunningham S, Picking R (2018) Collaborative artificial intelligence in music production. In: Proceedings of the audio mostly 2018 on sound in immersion and emotion, pp 1–4

  • Pandian MD (2019) Sleep pattern analysis and improvement using artificial intelligence and music therapy. J Artif Intell 1(02):54–62

    Google Scholar 

  • Parker JR, Heerema J (2008) Audio interaction in computer mediated games. Int J Comput Games Technol 2008

  • Pazos A, Del Riego AS, Dorado J, Caldalda JR (1999) Genetic music compositor. In: Proceedings of the 1999 congress on evolutionary computation-CEC99 (Cat. No. 99TH8406). IEEE, vol 2, pp 885–890

  • Peerdeman P (2010) Sound and music in games. VrijeUniversiteit, Amsterdam, pp 2–3

    Google Scholar 

  • Plans D, Morelli D (2012) Experience-driven procedural music generation for games. IEEE Trans Comput Intell AI Games 4(3):192–198

    Article  Google Scholar 

  • Plut C, Pasquier P (2019) Music matters: an empirical study on the effects of adaptive music on experienced and perceived player affect. In: 2019 IEEE conference on games (CoG). IEEE, pp 1–8

  • Plut C, Pasquier P (2020) Generative music in video games: state of the art, challenges, and prospects. Entertain Comput 33:100337

    Article  Google Scholar 

  • Qiu Z et al. (2019) Mind band: a crossmedia AI music composing platform. In: Proceedings of the 27th ACM international conference on multimedia, pp 2231–2233

  • Rietveld HC et al. (2017) Hear the music, play the game. Music and game design: interplays and perspectives. Game 1(6)

  • Roads C (1985) Research in music and artificial intelligence. ACM Comput Surv 17(2):163–190

    Article  Google Scholar 

  • Rogers K (2017) Exploring the role of audio in games. In: Extended abstracts publication of the annual symposium on computer-human interaction in play, pp 727–731

  • Roux-Girard G (2011) Listening to fear: a study of sound in horror computer games. In: Game sound technology and player interaction: concepts and developments. IGI Global, pp 192–212

  • Scott N (2014) Music to middleware: the growing challenges of the game music composer. In: Proceedings of the 2014 conference on interactive entertainment, pp 1–3

  • Shapiro RB, Kelly A, Ahrens M, Johnson B, Politi H, Fiebrink R (2017) Tangible distributed computer music for youth. Comput Music J 41(2):52–68

    Article  Google Scholar 

  • Stere C-C, Trăuşan-Matu Ş (2017) Generation of musical accompaniment for a poem, using artificial intelligence techniques. Int J User-Syst Interact 10(3):250–270

    Google Scholar 

  • Sitonio C, Nucciarelli A (2018) The impact of blockchain on the music industry. In: Presented at the 29th European regional ITS conference, Trento 2018

  • Sturm BL, Iglesias M, Ben-Tal O, Miron M, Gómez E (2019) Artificial intelligence and music: open questions of copyright law and engineering praxis. Arts 8(3):115

    Article  Google Scholar 

  • Su J-H, Chang W-Y, Tseng VS (2013) Personalized music recommendation by mining social media tags. Procedia Comput Sci 22:303–312

    Article  Google Scholar 

  • Tan S-L, Baxa J, Spackman MP (2010) Effects of built-in audio versus unrelated background music on performance in an adventure role-playing game. Int J Gaming Comput-Med Simul 2(3):1–23

    Google Scholar 

  • Tan X, Li X (2021) A tutorial on AI music composition. In: Proceedings of the 29th ACM international conference on multimedia, pp 5678–5680

  • Võ MD, Lê TT, Nguyễn HQ, Trương VTV, Đỗ TV (2015) Music arena game. FUG HCM

  • Waldner F (2011) Cross-industry innovation: the transfer of a service-based business model from the video game industry to the music industry. In: 2011 international conference on emerging intelligent data and web technologies. IEEE, pp 143–147

  • Yang F (2020) Artificial intelligence in music education. In: 2020 international conference on robots & intelligent system (ICRIS). IEEE, pp 483–484

  • Yuan S (2020) Application and study of musical artificial intelligence in music education field. J Phys Conf Ser 1533(3):032033

    Article  Google Scholar 

  • Zhang J, Wan J (2020) A summary of the application of artificial intelligence in music education. In: International conference on education, economics and information management (ICEEIM 2019). Atlantis Press, pp 42–44

  • Zhang X, Yang D (2021) Research on music assisted teaching system based on artificial intelligence technology. J Phys Conf Ser 1852(2):022032

    Article  MathSciNet  Google Scholar 

  • Zulić H (2019) How AI can change/improve/influence music composition, performance and education: three case studies. INSAM J Contemp Music ArtTechnology 1(2):100–114

    Google Scholar 

Download references

Funding

The paper did not receive any financial support.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shah Nazir.

Ethics declarations

Conflict of interest

The authors have no conflict of interest.

Ethical approval

My paper does not deal with any ethical problems.

Informed consent

We declare that all authors have informed Consent.

Additional information

Communicated by Irfan Uddin.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, T., Nazir, S. A comprehensive overview of AI-enabled music classification and its influence in games. Soft Comput 26, 7679–7693 (2022). https://doi.org/10.1007/s00500-022-06734-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-022-06734-4

Keywords

Navigation