Cognitive Computation

, Volume 4, Issue 3, pp 246–279

A Standardised Procedure for Evaluating Creative Systems: Computational Creativity Evaluation Based on What it is to be Creative


DOI: 10.1007/s12559-012-9156-1

Cite this article as:
Jordanous, A. Cogn Comput (2012) 4: 246. doi:10.1007/s12559-012-9156-1


Computational creativity is a flourishing research area, with a variety of creative systems being produced and developed. Creativity evaluation has not kept pace with system development with an evident lack of systematic evaluation of the creativity of these systems in the literature. This is partially due to difficulties in defining what it means for a computer to be creative; indeed, there is no consensus on this for human creativity, let alone its computational equivalent. This paper proposes a Standardised Procedure for Evaluating Creative Systems (SPECS). SPECS is a three-step process: stating what it means for a particular computational system to be creative, deriving and performing tests based on these statements. To assist this process, the paper offers a collection of key components of creativity, identified empirically from discussions of human and computational creativity. Using this approach, the SPECS methodology is demonstrated through a comparative case study evaluating computational creativity systems that improvise music.


Computational creativity Creativity evaluation Cognitively inspired evaluation Methodology Creativity Musical creativity 

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Centre for e-Research, Department of Digital HumanitiesKing’s College LondonLondonUK

Personalised recommendations