Cognitive Computation

, Volume 4, Issue 3, pp 246–279 | Cite as

A Standardised Procedure for Evaluating Creative Systems: Computational Creativity Evaluation Based on What it is to be Creative

Article

Abstract

Computational creativity is a flourishing research area, with a variety of creative systems being produced and developed. Creativity evaluation has not kept pace with system development with an evident lack of systematic evaluation of the creativity of these systems in the literature. This is partially due to difficulties in defining what it means for a computer to be creative; indeed, there is no consensus on this for human creativity, let alone its computational equivalent. This paper proposes a Standardised Procedure for Evaluating Creative Systems (SPECS). SPECS is a three-step process: stating what it means for a particular computational system to be creative, deriving and performing tests based on these statements. To assist this process, the paper offers a collection of key components of creativity, identified empirically from discussions of human and computational creativity. Using this approach, the SPECS methodology is demonstrated through a comparative case study evaluating computational creativity systems that improvise music.

Keywords

Computational creativity Creativity evaluation Cognitively inspired evaluation Methodology Creativity Musical creativity 

References

  1. 1.
    Cardoso A, Veale T, Wiggins GA. Converging on the divergent: the history (and future) of the international joint workshops in computational creativity. AI Mag. 2009;30(3):15–22.Google Scholar
  2. 2.
    Ritchie G. Some empirical criteria for attributing creativity to a computer program. Minds Mach. 2007;17:67–99.CrossRefGoogle Scholar
  3. 3.
    Colton S. Creativity versus the perception of creativity in computational systems. In: Proceedings of AAAI symposium on creative systems; 2008. p. 14–20.Google Scholar
  4. 4.
    Pease A, Colton S. On impact and evaluation in computational creativity: a discussion of the turing test and an alternative proposal. In: Proceedings of the AISB’11 convention. York, UK: AISB; 2011. p. 1–8.Google Scholar
  5. 5.
    Bundy A. What kind of field is AI? In: Partridge D, Wilks Y, editors. The foundations of artificial intelligence. Cambridge: Cambridge University Press; 1990. p. 215–222.CrossRefGoogle Scholar
  6. 6.
    Pearce MT, Meredith D, Wiggins GA. Motivations and methodologies for automation of the compositional process. Musicae Scientae. 2002;6(2):119–147.Google Scholar
  7. 7.
    Jordanous A. A fitness function for creativity in Jazz improvisation and beyond. In: Proceedings of the international conference on computational creativity. Lisbon, Portugal; 2010. p. 223–227.Google Scholar
  8. 8.
    Biles JA. Improvising with genetic algorithms: GenJam. In: Miranda ER, Biles JA, editors. Evolutionary computer music. London: Springer; 2007. p. 137–169.CrossRefGoogle Scholar
  9. 9.
    Gillick J, Tang K, Keller RM. Machine learning of jazz grammars. Compu Music J. 2010;34(3):56–66.CrossRefGoogle Scholar
  10. 10.
    Lewis GE. Too many notes: computers, complexity and culture in Voyager. Leonardo Music J. 2000;10:33–39.CrossRefGoogle Scholar
  11. 11.
    Ritchie G. Assessing creativity. In: Proceedings of the AISB symposium on AI and creativity in arts and science. York; 2001. p. 3–11.Google Scholar
  12. 12.
    Colton S, Pease A, Ritchie G. The effect of input knowledge on creativity. In: Proceedings of workshop program of ICCBR-Creative systems: approaches to creativity in AI and cognitive science; 2001. p. 1–6.Google Scholar
  13. 13.
    Computational Creativity. The computational creativity website; Last accessed 25th May 2012. URL:http://www.computationalcreativity.net.
  14. 14.
    Wiggins GA. A preliminary framework for description, analysis and comparison of creative systems. Knowl Based Syst. 2006;19(7):449–458.CrossRefGoogle Scholar
  15. 15.
    Widmer G, Flossmann S, Grachten M. YQX plays chopin. AI Mag. 2009;30(3):35–48.Google Scholar
  16. 16.
    León C, Gervás P. The role of evaluation-driven rejection in the successful exploration of a conceptual space of stories. Minds Mach. 2010;20(4):615–634.CrossRefGoogle Scholar
  17. 17.
    Pérez y Pérez R. MEXICA: a computer model of creativity in writing, Ph.D. thesis. Brighton: University of Sussex; 1999.Google Scholar
  18. 18.
    Colton S, de Mataras RL, Stock O. Computational creativity: coming of age. AI Mag. 2009;30(3):11–14.Google Scholar
  19. 19.
    Wiggins GA. Closing the loop: computational creativity from a model of music cognition. COGS research seminar, School of Informatics, University of Sussex; 2008 (October 2008).Google Scholar
  20. 20.
    Pease A, Winterstein D, Colton S. Evaluating machine creativity. In: Proceedings of ICCBR workshop on approaches to creativity; 2001. p. 129–137.Google Scholar
  21. 21.
    Peinado F, Gervas P. Evaluation of automatic generation of basic stories. New Generation Comput. 2006;24(3):289–302.CrossRefGoogle Scholar
  22. 22.
    Pereira FC, Cardoso A. Experiments with free concept generation in Divago. Knowl Based Syst. 2006;19(7):459 – 470.CrossRefGoogle Scholar
  23. 23.
    Alvarado Lopez J, Pérez y Pérez R. A computer model for the generation of monophonic musical melodies. In: Proceedings of the 5th international joint workshop on computational creativity. Madrid, Spain; 2008. p. 117–126.Google Scholar
  24. 24.
    Brown D. Computational artistic creativity and its evaluation. In: Computational creativity: an interdisciplinary approach. No. 09291 in Dagstuhl Seminar proceedings. Dagstuhl, Germany; 2009. p. 1–8.Google Scholar
  25. 25.
    Chordia P, Rae A. Tabla Gyan: an artificial Tabla improviser. In: Proceedings of the international conference on computational creativity. Lisbon, Portugal; 2010. p. 155–164.Google Scholar
  26. 26.
    Pease A. Personal communications; 2012. In conversation.Google Scholar
  27. 27.
    ICCC’11. Proceedings of the international conference for computational creativity, Mexico City, Mexico; 2011. URL:http://iccc11.cua.uam.mx/proceedings (last accessed 25th May 2012).
  28. 28.
    ICCC’10. Proceedings of the international conference for computational creativity, Lisbon, Portugal; 2010. URL:http://eden.dei.uc.pt/~amilcar/ftp/e-Proceedings_ICCC-X.pdf (last accessed 25th May 2012).
  29. 29.
    IJWCC’07. Proceedings of the international joint workshop for computational creativity, London, UK; 2007. URL:http://doc.gold.ac.uk/isms/CC07/CC07Proceedings.pdf (last accessed 25th May 2012).
  30. 30.
    Ventura D. A Reductio Ad Absurdum experiment in sufficiency for evaluating (computational) creative systems. In: Proceedings of the 5th international joint workshop on computational creativity. Madrid, Spain; 2008. p. 11–19.Google Scholar
  31. 31.
    Tearse B, Mawhorter P, Mateas M, Wardrip-Fonin N. Experimental results from a rational reconstruction of MINSTREL. In: Proceedings of the 2nd international conference on computational creativity. Mexico City, Mexico; 2011. p. 54–59.Google Scholar
  32. 32.
    Gervás P. Exploring quantitative evaluations of the creativity of automatic poets. In: Proceedings of the 2nd. Workshop on creative systems, approaches to creativity in artificial intelligence and cognitive science (ECAI 2002); 2002. p. 1–8.Google Scholar
  33. 33.
    Pereira FC, Mendes M, Gervás P, Cardoso A. Experiments with assessment of creative systems: an application of Ritchie’s criteria. In: Proceedings of the workshop on computational creativity (IJCAI 05); 2005. p. 1–8.Google Scholar
  34. 34.
    Colton S. Automated theory formation in pure mathematics. Distinguished dissertations. Springer, London, UK, 2002.Google Scholar
  35. 35.
    Colton S, Charnley J, Pease A. Computational creativity theory: the FACE and IDEA descriptive models. In: Proceedings of the 2nd international conference on computational creativity. Mexico City, Mexico; 2011. p. 90–95.Google Scholar
  36. 36.
    Pease A, Colton S. Computational creativity theory: inspirations behind the FACE and the IDEA models. In: Proceedings of the 2nd international conference on computational creativity. Mexico City, Mexico; 2011. p. 72–77.Google Scholar
  37. 37.
    Pearce M, Wiggins G. Towards a framework for the evaluation of machine compositions. In: Proceedings of the AISB symposium on AI and creativity in arts and science. York, UK; 2001. p. 1–12.Google Scholar
  38. 38.
    Brown P. Autonomy, Signature and creativity. In: Computational creativity: an interdisciplinary approach. No. 09291 in Dagstuhl Seminar proceedings. Dagstuhl, Germany; 2009. p. 1–7.Google Scholar
  39. 39.
    Gervas P. Computational approaches to storytelling and creativity. AI Mag. 2009;30(3):49–62.Google Scholar
  40. 40.
    Meehan J. Tale-Spin. In: Schank RC, Riesbeck CK, editors. Inside computer understanding: five programs plus minatures. Hillside, NJ: Lawrence Erlbaum Associates; 1981. p. 197–227.Google Scholar
  41. 41.
    Turner SR. The creative process: a computer model of storytelling and creativity. Hillside, NJ: Erlbaum; 1994.Google Scholar
  42. 42.
    Bringsjord S. Artificial intelligence and literary creativity: inside the mind of BRUTUS. London, UK: Lawrence Erlbaum Associates; 2000.Google Scholar
  43. 43.
    Pérez y Pérez R, Sharples M. Three computer-based models of storytelling: BRUTUS, MINSTREL and MEXICA. Knowl Based Syst. 2004;17(1):15–29.CrossRefGoogle Scholar
  44. 44.
    Peinado F, Francisco V, Hervás R, Gervás P. Assessing the novelty of computer-generated narratives using empirical metrics. Minds Mach. 2010;20(4):565–588.CrossRefGoogle Scholar
  45. 45.
    Pérez y Pérez R, Aguilar A, Negrete S. The ERI-designer: a computer model for the arrangement of furniture. Minds Mach. 2010;20(4):533–564.CrossRefGoogle Scholar
  46. 46.
    Aguilar A, Hernandez D, Pérez y Pérez R, Rojas M, Zambrano MdL. A computer model for novel arrangements of furniture. In: Proceedings of the 5th international joint workshop on computational creativity. Madrid, Spain; 2008. p. 157–162.Google Scholar
  47. 47.
    Whorley RP, Wiggins GA, Pearce MT. Systematic evaluation and improvement of statistical models of harmony. In: Proceedings of the 4th international joint workshop on computational creativity. London, UK; 2007. p. 81–88.Google Scholar
  48. 48.
    Whorley R, Wiggins G, Rhodes C, Pearce M. Development of techniques for the computational modelling of harmony. In: Proceedings of the international conference on computational creativity. Lisbon, Portugal; 2010. p. 11–15.Google Scholar
  49. 49.
    Gervás P, Perez y Perez R. On the fly collaborative story-telling: revising contributions to match a shared partial story line. In: Proceedings of the 4th international joint workshop on computational creativity. London, UK; 2007. p. 13–20.Google Scholar
  50. 50.
    Montfort N, Pérez y Pérez R. Integrating a plot generator and an automatic narrator to create and tell stories. In: Proceedings of the 5th international joint workshop on computational creativity. Madrid, Spain; 2008. p. 61–70.Google Scholar
  51. 51.
    Pérez y Pérez R, Negrete S, Penãlosa E, Ávila R, Castellanos V, Lemaitre C. MEXICA-Impro: a computational model for narrative improvisation. In: Proceedings of the international conference on computational creativity. Lisbon, Portugal; 2010. p. 90–99.Google Scholar
  52. 52.
    Plucker JA. Beware of simple conclusions: the case for content generality of creativity. Creativ Res J. 1998;11(2):179–182.CrossRefGoogle Scholar
  53. 53.
    Baer J. The case for domain specificity of creativity. Creativ Res J. 1998;11(2):173–177.CrossRefGoogle Scholar
  54. 54.
    Plucker JA, Beghetto RA. Why creativity is domain general, why it looks domain specific, and why the distinction doesn’t matter. In: Sternberg RJ, Grigorenko EL, Singer JL, editors. Creativity: from potential to realization. Washington, DC: American Psychological Association; 2004. p. 153–167.CrossRefGoogle Scholar
  55. 55.
    Baer J. Is Creativity domain-specific? In: Kaufman JC, Sternberg RJ, editors. The Cambridge handbook of creativity. New York, NY: Cambridge University Press; 2010. p. 321–341.Google Scholar
  56. 56.
    Rhodes M. An analysis of creativity. Phi Delta Kappan. 1961;42(7):305–310.Google Scholar
  57. 57.
    Plucker JA, Beghetto RA, Dow GT. Why isn’t creativity more important to educational psychologists? Potentials, pitfalls, and future directions in creativity research. Educ Psychol. 2004;39(2):83–96.CrossRefGoogle Scholar
  58. 58.
    Sternberg RJ, Lubart TI. The concept of creativity: prospects and paradigms. In: Sternberg RJ, editor. Handbook of creativity. Cambridge, UK: Cambridge University Press; 1999. p. 3–15.Google Scholar
  59. 59.
    Veale T, Gervás P, Pease A. Understanding creativity: a computational perspective. New Generation Comput. 2006;24(3):203–207.CrossRefGoogle Scholar
  60. 60.
    Newell A, Shaw JG, Simon HA. The process of creative thinking. In: Gruber HE, Terrell G, Wertheimer E, editors. Contemporary approaches to creative thinking. New York: Atherton; 1963. p. 63–119.Google Scholar
  61. 61.
    McCarthy J. Ascribing mental qualities to machines. In: Ringle M, editor. Philosophical perspectives in artificial intelligence. Atlantic Highlands, NJ: Humanities Press; 1979. p. 161–195.Google Scholar
  62. 62.
    Wiggins GA. Searching for computational creativity. New Generation Comput. 2006;24(3):209–222.CrossRefGoogle Scholar
  63. 63.
    Jennings KE. Search strategies and the creative process. In: Proceedings of the international conference on computational creativity. Lisbon, Portugal; 2010. p. 130–139.Google Scholar
  64. 64.
    Ventura D. No free lunch in the search for creativity. In: Proceedings of the 2nd international conference on computational creativity. Mexico City, Mexico; 2011. p. 108–110.Google Scholar
  65. 65.
    Boden MA. The creative mind: Myths and mechanisms. 2nd ed. London, UK: Routledge; 2004.Google Scholar
  66. 66.
    Ritchie G. Uninformed resource creation for humour simulation. In: Proceedings of the 5th international joint workshop on computational creativity. Madrid, Spain; 2008. p. 147–150.Google Scholar
  67. 67.
    Cohen LM. A Review of: “Expanding visions of creative intelligence: an interdisciplinary exploration by Don Ambrose”. Creativi Res J. 2009;21(2–3):307–308.CrossRefGoogle Scholar
  68. 68.
    Williams F. The mystique of unconscious creation. In: Kagan J, editor. Creativity and learning. Boston: Beacon Press; 1967. p. 142–152.Google Scholar
  69. 69.
    Albert RS, Runco MA. A history of research on creativity. In: Sternberg RJ, editor. Handbook of creativity. Cambridge, MA: Cambridge University Press; 1999. p. 16–31.Google Scholar
  70. 70.
    Kaufman JC. Creativity 101. The Psych 101 series. New York: Springer; 2009.Google Scholar
  71. 71.
    Poincaré H. Mathematical creation. In: The foundations of science: science and hypothesis, the value of science, science and method, vol. Science and method (Original French version published 1908, Authorized translation by George Bruce Halsted). New York: The Science Press; 1929. p. 383–394.Google Scholar
  72. 72.
    Wallas G. The art of thought. abridged ed. London, UK: C. A. Watts & Co; 1945.Google Scholar
  73. 73.
    Csikszentmihalyi M. Society, culture, and person: a systems view of creativity. In: Sternberg RJ, editor. The nature of creativity. Cambridge, UK: Cambridge University Press; 1988. p. 325–339.Google Scholar
  74. 74.
    Resnick M. Sowing the seeds for a more creative society. Learn Leadi Technol. 2007;35(4).Google Scholar
  75. 75.
    Guilford JP. Creativity. Am Psychol. 1950;5:444–454.PubMedCrossRefGoogle Scholar
  76. 76.
    Mednick SA. The remote associates test. Boston: Houghton Mifflin Company; 1967.Google Scholar
  77. 77.
    Torrance EP. The nature of creativity as manifest in its testing. In: Sternberg RJ, editor. The nature of creativity. Cambridge, UK: Cambridge University Press; 1988. p. 43–75.Google Scholar
  78. 78.
    Mayer RE. Fifty years of creativity research. In: Sternberg RJ, editor. Handbook of creativity. Cambridge, UK: Cambridge University Press; 1999. p. 449–460.Google Scholar
  79. 79.
    Ivcevic Z. Creativity map: toward the next generation of theories of creativity. Psychol Aesthet Creativ Arts. 2009;3(1):17–21.CrossRefGoogle Scholar
  80. 80.
    Stein MI. A transactional approach to creativity. In: Taylor CW, Barron F, editors. Scientific creativity: its recognition and development. New York: Wiley; 1963. p. 217–227.Google Scholar
  81. 81.
    MacKinnon DW. Creativity: a multi-faceted phenomenon. In: Roslansky JD, editor. Creativity: a discussion at the nobel conference. Amsterdam, The Netherlands: North-Holland Publishing Company; 1970. p. 17–32.Google Scholar
  82. 82.
    Odena O, Welch G. A generative model of teachers’ thinking on musical creativity. Psychol Music. 2009;37(4):416–442.CrossRefGoogle Scholar
  83. 83.
    Clifford RD. Random numbers, Chaos theory, and cogitation: a search for the minimal creativity standard in copyright law. Denver Univ Law Rev. 2004;82(2):259–299.Google Scholar
  84. 84.
    Mandel GN. To promote the creative process: intellectual property law and the psychology of creativity. Notre Dame Law Rev. 2011;86(1):1999–2024 (online).Google Scholar
  85. 85.
    Feist. Feist Publications, Inc. v. Rural Telephone Service Co. 499 US 340. 1991;111 S. Ct 1282, 113 L. Ed. 2d 358(Supreme Court).Google Scholar
  86. 86.
    Karjala DS. Copyright and creativity. UCLA Entertain Law Rev. 2008;15:169–201.Google Scholar
  87. 87.
    Copyright, Designs and Patents Act. UK Government Legislation; 1988. Ch. 48/1988.Google Scholar
  88. 88.
    Noll AM. The beginnings of computer art in the United States: a memoir. Leonardo. 1994;27(1):39–44.CrossRefGoogle Scholar
  89. 89.
    Holmes N. The automation of originality: when originality is automated, what becomes of personality? Computer. 2009;98–100 (March 2009).Google Scholar
  90. 90.
    Warner J. The absence of creativity in Feist and the computational process. J Am Soc Inform Sci Technol. 2010;61(11):2324–2336.CrossRefGoogle Scholar
  91. 91.
    Hennessey BA, Amabile TM. Creativity. Ann Rev Psychol. 2010;61:569–598.PubMedCrossRefGoogle Scholar
  92. 92.
    Sternberg RJ. A three-facet model of creativity. In: Sternberg RJ, editor. The nature of creativity. Cambridge, UK: Cambridge University Press; 1988. p. 125–147.Google Scholar
  93. 93.
    Weisberg RW. Problem solving and creativity. In: Sternberg RJ, editor. The nature of creativity. Cambridge, UK: Cambridge University Press; 1988. p. 148–176.Google Scholar
  94. 94.
    Bryan-Kinns N. Everyday creativity. In: Proceedings of the 7th ACM conference on creativity and cognition. Berkeley, California; 2009. p. 1.Google Scholar
  95. 95.
    Torrance EP. Torrance tests of creative thinking. Bensenville, IL: Scholastic Testing Service; 1974.Google Scholar
  96. 96.
    Kraft U. Unleashing creativity. Scientific American Mind. 2005;April.Google Scholar
  97. 97.
    Runco MA, Dow G, Smith WR. Information, experience, and divergent thinking: an empirical test. Creativ Res J. 2006;18(3):267–277.Google Scholar
  98. 98.
    Kaufman JC, Kaufman SB, Lichtenberger EO. Finding creative potential on intelligence tests via divergent production finding creative potential on intelligence tests via divergent production finding creative potential on intelligence tests via divergent production. Can J School Psychol. 2011;26(2):83–106.Google Scholar
  99. 99.
    Boden MA, editor. Dimensions of creativity. Cambridge, MA: MIT Press; 1994.Google Scholar
  100. 100.
    Seth AK. Explanatory correlates of consciousness: theoretical and computational challenges. Cogn Comput. 2009;1:50–63.CrossRefGoogle Scholar
  101. 101.
    McCrae RR, Costa Jr PT. A five-factor theory of personality. In: Pervin LA, John OP, editors. Handbook of personality: theory and research. 2nd ed. New York: The Guilford Press; 1999. p. 139–153.Google Scholar
  102. 102.
    Romero P, Calvillo-Gamez E. Towards an embodied view of flow. In: Proceedings of the 2nd international workshop on user models for motivational systems: the affective and the rational routes to persuasion (UMMS 2011). Girona, Spain; 2011. p. 100–105.Google Scholar
  103. 103.
    Huron D. Tone and voice: a derivation of the rules of voice-leading from perceptual principles. Music Percep. 2001;19(1):1–64.CrossRefGoogle Scholar
  104. 104.
    Evans V, Green M. Cognitive linguistics: an introduction. Edinburgh, UK: Edinburgh University Press; 2006.Google Scholar
  105. 105.
    Oakes MP. Statistics for corpus linguistics. Edinburgh, UK: Edinburgh University Press; 1998.Google Scholar
  106. 106.
    Kilgarriff A. Comparing corpora. Int J Corpus Linguist. 2001;6(1):97–133.CrossRefGoogle Scholar
  107. 107.
    Kilgarriff A. Where to go if you would like to find out more about a word than the dictionary tells you. Macmillan English Dictionary Mag. 2006;Issue 35 (Jan–Feb).Google Scholar
  108. 108.
    Lin D. An information-theoretic definition of similarity. In: Proceedings of the 15th international conference on machine learning. Madison, WI; 1998. p. 296–304.Google Scholar
  109. 109.
    Biemann C. Chinese Whispers: an efficient graph clustering algorithm and its application to natural language processing problems. In: Proceedings of textGraphs: the first workshop on graph based methods for natural language processing. Morristown, NJ: Association for Computational Linguistics; 2006. p. 73–80.Google Scholar
  110. 110.
    Jordanous A. Evaluating evaluation: assessing progress in computational creativity research. In: Proceedings of the second international conference on computational creativity (ICCC-11). Mexico City, Mexico; 2011. p. 102–107.Google Scholar
  111. 111.
    Boden MA. What is creativity? In: Boden MA, editor. Dimensions of creativity. Cambridge, MA: MIT Press; 1994. p. 75–117.Google Scholar
  112. 112.
    Moffat DC, Kelly M. An investigation into people’s bias against computational creativity in music composition. In: Proceedings of the 3rd international joint workshop on computational creativity (ECAI06 Workshop). Riva del Garda, Italy; 2006. p. 1–8.Google Scholar
  113. 113.
    Haenen J, Rauchas S. Investigating artificial creativity by generating melodies, using connectionist knowledge representation. In: Proceedings of the 3rd international joint workshop on computational creativity (ECAI06 Workshop). Riva del Garda, Italy; 2006. p. 33–38.Google Scholar
  114. 114.
    Pearce MT, Wiggins GA. Evaluating cognitive models of musical composition. In: Proceedings of the 4th international joint workshop on computational creativity. London, UK; 2007. p. 73–80.Google Scholar
  115. 115.
    Lewis GE. Improvising with creative machines: reflections on human-machine interaction (keynote talk). In: Proceedings of the 2nd international conference on computational creativity. Mexico City, Mexico; 2011. p. xii–xiii.Google Scholar
  116. 116.
    Parker E. Drifting on a reed; 2011. Keynote presentation at The Improvised Space. London, UK: Techniques, Traditions and Technologies.Google Scholar
  117. 117.
    Csikszentmihalyi M. The creative person and the creative system (keynote address). In: Proceeding of the seventh ACM conference on creativity and cognition. Berkeley, California; 2009. p. 5–6.Google Scholar
  118. 118.
    Friis-Olivarius M, Wallentin M, Vuust P. Improvisation—the neural foundation for creativity (poster). In: Proceedings of the 7th ACM creativity and cognition conference. Berkeley, California; 2009. p. 411–412.Google Scholar
  119. 119.
    Berkowitz AL, Ansari D. Expertise-related deactivation of the right temporoparietal junction during musical improvisation. NeuroImage. 2010;49(1):712–719.PubMedCrossRefGoogle Scholar
  120. 120.
    Berliner PF. Thinking in jazz: the infinite art of improvisation. Chicago studies in ethnomusicology. Chicago, IL: The University of Chicago Press; 1994.Google Scholar
  121. 121.
    Gibbs L. Evaluating Creative (jazz) Improvisation: Distinguishing Invention and Creativity. In: Proceedings of leeds international Jazz conference 2010: Improvisation—jazz in the creative moment. Leeds, UK; 2010. p. 1–4.Google Scholar
  122. 122.
    Bailey D. Improvisation: its nature and practice in music. New York: Da Capo Press; 1993.Google Scholar
  123. 123.
    Biles JA. GenJam: a genetic algorithm for generating Jazz Solos. In: Proceedings of the international computer music conference. Denmark; 1994. p. 131–137.Google Scholar
  124. 124.
    Jordanous A. Defining creativity: finding keywords for creativity using corpus linguistics techniques. In: Proceedings of the international conference on computational creativity. Lisbon, Portugal; 2010. p. 278–287.Google Scholar
  125. 125.
    BNC Consortium. The British National Corpus, version 3 (BNC XML Edition); 2007. Distributed by Oxford University Computing Services on behalf of the BNC Consortium. URL: http://www.natcorp.ox.ac.uk (last accessed 25th May 2012).
  126. 126.
    Jordanous A. Evaluating computational creativity: a Standardised Procedure for Evaluating Creative Systems and its application, Ph.D. thesis. University of Sussex. Brighton, UK; forthcoming.Google Scholar
  127. 127.
    Zhu X, Xu Z, Khot T. How creative is your writing? A linguistic creativity measure from computer science and cognitive psychology perspectives. In: Proceedings of NAACL HLT workshop on computational approaches to linguistic creativity (ACL). Boulder, Colorado; 2009. p. 87–93.Google Scholar
  128. 128.
    Sauro J, Kindlund E. A Method to standardize usability metrics into a single score. In: Proceedings of the CHI’05 conference on human factors in computing systems. Portland, OR; 2005. p. 401–409.Google Scholar
  129. 129.
    Temperley N, Wollny P. Bach Revival; Last accessed 25th May 2012. Available at Grove Music Online, part of Oxford Music Online (subscription required). URL: http://www.oxfordmusiconline.com/subscriber/article/grove/music/01708.

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Centre for e-Research, Department of Digital HumanitiesKing’s College LondonLondonUK

Personalised recommendations