Deep Artificial Composer: A Creative Neural Network Model for Automated Melody Generation

  • Florian ColomboEmail author
  • Alexander Seeholzer
  • Wulfram Gerstner
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10198)


The inherent complexity and structure on long timescales make the automated composition of music a challenging problem. Here we present the Deep Artificial Composer (DAC), a recurrent neural network model of note transitions for the automated composition of melodies. Our model can be trained to produce melodies with compositional structures extracted from large datasets of diverse styles of music, which we exemplify here on a corpus of Irish folk and Klezmer melodies. We assess the creativity of DAC-generated melodies by a new measure, the novelty of musical sequences, showing that melodies imagined by the DAC are as novel as melodies produced by human composers. We further use the novelty measure to show that the DAC creates melodies musically consistent with either of the musical styles it was trained on. This makes the DAC a promising candidate for the automated composition of convincing musical pieces of any provided style.


Automated music composition Deep neural networks Sequence learning Evaluation of generative models 



The authors thank Samuel P. Muscinelli and Johanni Brea for their guidance and helpful comments. This research was partially supported by the Swiss National Science Foundation (200020_147200) and the European Research Council grant no. 268689 (MultiRules).


  1. 1.
    Seeger, C.: On dissonant counterpoint. Mod. Music 7(4), 25–31 (1930)Google Scholar
  2. 2.
    Xenakis, I.: Formalized Music: Thought and Mathematics in Composition, vol. 6. Pendragon Press, New York (1992)Google Scholar
  3. 3.
    Lovelace, A.: Notes on l. menabrea’s “sketch of the analytical engine invented by charles babbage, esq.”. Taylor’s Scientific Memoirs 3 (1843)Google Scholar
  4. 4.
    Fernández, J.D., Vico, F.: Ai methods in algorithmic composition: a comprehensive survey. J. Artif. Intell. Res. 48, 513–582 (2013)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Hochreiter, S.: The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncertain. Fuzziness Knowl. Based Syst. 6(2), 107–116 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Gers, F.A., Schmidhuber, J., Cummins, F.: Learning to forget: Continual prediction with LSTM. Neural Comput. 12(10), 2451–2471 (2000)CrossRefGoogle Scholar
  7. 7.
    Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint (2014). arXiv:1412.3555
  8. 8.
    Graves, A.: Generating sequences with recurrent neural networks. arXiv preprint (2013). arxiv:1308.0850
  9. 9.
    Karpathy, A.: The unreasonable effectiveness of recurrent neural networks. (2015). Accessed 1 Apr 2016
  10. 10.
    Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint (2014). arXiv:1406.1078
  11. 11.
    Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)Google Scholar
  12. 12.
    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  13. 13.
    Gregor, K., Danihelka, I., Graves, A., Rezende, D.J., Wierstra, D.: Draw: a recurrent neural network for image generation. arXiv preprint (2015). arXiv:1502.04623
  14. 14.
    Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. arXiv preprint (2015). arXiv:1508.06576
  15. 15.
    Todd, P.M.: A connectionist approach to algorithmic composition. Comput. Music J. 13(4), 27–43 (1989)CrossRefGoogle Scholar
  16. 16.
    Boulanger-Lewandowski, N., Bengio, Y., Vincent, P.: Modeling temporal dependencies in high-dimensional sequences: application to polyphonic music generation and transcription. arXiv preprint (2012). arXiv:1206.6392
  17. 17.
    Franklin, J.A.: Computational models for learning pitch and duration using lstm recurrent neural networks. In: Proceedings of the Eighth International Conference on Music Perception and Cognition (ICMPC8), Adelaide, Australia, Causal Productions (2004)Google Scholar
  18. 18.
    Eck, D., Schmidhuber, J.: Finding temporal structure in music: blues improvisation with LSTM recurrent networks. In: Proceedings of the 2002 12th IEEE Workshop on Neural Networks for Signal Processing, pp. 747–756. IEEE (2002)Google Scholar
  19. 19.
    Colombo, F., Muscinelli, S.P., Seeholzer, A., Brea, J., Gerstner, W.: Algorithmic composition of melodies with deep recurrent neural networks. In: Proceedings of the First Conference on Computer Simulation of Musical Creativity (CSMC 2016), Huddersfield, UK (2016)Google Scholar
  20. 20.
    Walshaw, C.: abc notation. Accessed 11 March 2016
  21. 21.
    Cuthbert, M., Ariza, C., Hogue, B., Oberholtzer, J.W.: music21, a toolkit for computer-aided musicology. Accessed 11 Nov 2016
  22. 22.
    Ames, C.: The markov process as a compositional model: a survey and tutorial. Leonardo 22, 175–187 (1989)CrossRefGoogle Scholar
  23. 23.
    Hébert, S., Peretz, I.: Recognition of music in long-term memory: are melodic and temporal patterns equal partners? Mem. Cogn. 25(4), 518–533 (1997)CrossRefGoogle Scholar
  24. 24.
    Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., Bengio, Y.: Theano: a CPU and GPU math expression compiler. In: Proceedings of the Python for Scientific Computing Conference (SciPy), vol. 4, p. 3. TX, Austin (2010)Google Scholar
  25. 25.
    Jozefowicz, R., Zaremba, W., Sutskever, I.: An empirical exploration of recurrent network architectures. In: Proceedings of the 32nd International Conference on Machine Learning (ICML 2015), pp. 2342–2350 (2015)Google Scholar
  26. 26.
    Kingma, D., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint (2014). arXiv:1412.6980
  27. 27.
    Norbeck, H.: Henrik Norbecks’s corpus of irish tunes. Accessed 03 Nov 2016
  28. 28.
    Chamber, J.: John chambers’s corpus of klezmer tunes. Accessed 03 Nov 2016
  29. 29.
    Gorney, E.: Dictionary of creativity: terms, concepts, theories and findings in creativity research (2007)Google Scholar
  30. 30.
    Fix, E., Hodges, J.L.: Discriminatory analysis-nonparametric discrimination: consistency properties. Technical report, DTIC Document (1951)Google Scholar
  31. 31.
    Nevill-Manning, C.G., Witten, I.H.: Identifying hierarchical strcture in sequences: a linear-time algorithm. J. Artif. Intell. Res. (JAIR) 7, 67–82 (1997)zbMATHGoogle Scholar
  32. 32.
    Jordanous, A.: A standardised procedure for evaluating creative systems: computational creativity evaluation based on what it is to be creative. Cogn. Comput. 4(3), 246–279 (2012)CrossRefGoogle Scholar
  33. 33.
    Loughran, R., ONeill, M.: Generative music evaluation: why do we limit to human? In: Proceedings of the first Conference on Computer Simulation of Musical Creativity (CSMC 2016), Huddersfield, UK (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Florian Colombo
    • 1
    Email author
  • Alexander Seeholzer
    • 1
  • Wulfram Gerstner
    • 1
  1. 1.School of Computer and Communication Sciences and School of Life Sciences, Brain-Mind InstituteEcole Polytechnique Fédérale de LausanneLausanneSwitzerland

Personalised recommendations