Skip to main content

Evaluating Evaluation: Assessing Progress and Practices in Computational Creativity Research

Part of the Computational Synthesis and Creative Systems book series (CSACS)


Computational creativity research has produced many computational systems that are described as ‘creative’. Historically, these ‘creative systems’ have not received much in terms of evaluation of the actual creativity of the systems, although this has recently attracted more attention as a research perspective. As a scientific research community, computational creativity researchers can benefit from more systematic/standardised approaches to evaluation of the creativity of our systems, to help us progress in understanding creativity and modelling it computationally. A methodology for creativity evaluation should accommodate different manifestations of creativity but also requires a clear, definitive statement of the tests used for evaluation. Here a historical perspective is given on how computational creativity researchers have evaluated (or not evaluated) the creativity of their systems, considering contextual reasons behind this. Different evaluation approaches and frameworks are currently available, though it is not yet clear which (if any) of several recently proposed methods are emerging as the preferred options to use. The Standardised Procedure for Evaluating Creative Systems (SPECS) forms an overarching set of guidelines for how to tackle evaluation of creative systems and can incorporate recent proposals for creativity evaluation. To help decide which evaluation method is best to use, this chapter concludes by exploring five meta-evaluation criteria devised from cross-disciplinary research into good evaluative practice. Together, these considerations help us explore best practice in computational creativity evaluation, helping us develop the tools we have available to us as computational creativity researchers.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   139.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Anna Jordanous .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Jordanous, A. (2019). Evaluating Evaluation: Assessing Progress and Practices in Computational Creativity Research. In: Veale, T., Cardoso, F. (eds) Computational Creativity. Computational Synthesis and Creative Systems. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-43608-1

  • Online ISBN: 978-3-319-43610-4

  • eBook Packages: Computer ScienceComputer Science (R0)