Reinforcement Learning and the Creative, Automated Music Improviser

  • Benjamin D. Smith
  • Guy E. Garnett
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7247)


Automated creativity, giving a machine the ability to originate meaningful new concepts and ideas, is a significant challenge. Machine learning models make advances in this direction but are typically limited to reproducing already known material. Self-motivated reinforcement learning models present new possibilities in computational creativity, conceptually mimicking human learning to enable automated discovery of interesting or surprising patterns. This work describes a musical intrinsically motivated reinforcement learning model, built on adaptive resonance theory algorithms, towards the goal of producing humanly valuable creative music. The capabilities of the prototype system are examined through a series of short, promising compositions, revealing an extreme sensitivity to feature selection and parameter settings, and the need for further development of hierarchical models.


Computational creativity machine learning music composition reinforcement learning adaptive resonance theory 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Boden, M.: The Creative Mind: Myths and Mechanisms, 2nd edn. Routledge, London (2004)Google Scholar
  2. 2.
    Carpenter, G.A., Grossberg, S., Rosen, D.B.: Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by an Adaptive Resonance System. Neural Networks 4, 759–771 (1991)CrossRefGoogle Scholar
  3. 3.
    Davis, C.J., Bowers, J.S.: Contrasting five different theories of letter position coding: Evidence from orthographic similarity effects. Journal of Experimental Psychology: Human Perception and Performance 32(3), 535–557 (2006)CrossRefGoogle Scholar
  4. 4.
    Gjerdingen, R.O.: Categorization of musical patterns by self-organizing neuronlike networks. Musical Perception (1990)Google Scholar
  5. 5.
    McCormack, J.: Open problems in evolutionary music and art. Applications of Evolutionary Computing, 428–436 (2005)Google Scholar
  6. 6.
    Murphy, K.P.: Dynamic Bayesian Networks: Representation, Inference and Learning. Ph.D. in Computer Science. University of California, Berkeley (2002)Google Scholar
  7. 7.
    Newell, A., Shaw, J.G., Simon, H.A.: The process of creative thinking. In: Gruber, H.E., Terrell, G., Wertheimer, M. (eds.) Contemporary Approaches to Creative Thinking, pp. 63–119. Atherton, New York (1963)Google Scholar
  8. 8.
    Pearce, M. T. The Construction and Evaluation of Statistical Models of Melodic Structure in Music Perception and Composition. PhD. City University, London (2005)Google Scholar
  9. 9.
    Schmidhuber, J.: Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes. Anticipatory Behavior in Adaptive Learning Systems (2009)Google Scholar
  10. 10.
    Smith, B.D., Garnett, G.E.: The Self-Supervising Machine. In: Proceedings of NIME 2011, Oslo, Norway (2011)Google Scholar
  11. 11.
    Swaminathan, D.: A Dynamic Bayesian Approach to Computational Laban Shape Quality Analysis. Advances in Human-Computer Interaction (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Benjamin D. Smith
    • 1
  • Guy E. Garnett
    • 1
  1. 1.University of Illinois at Urbana-ChampaignUnited States

Personalised recommendations