Minds and Machines

, Volume 20, Issue 4, pp 565–588 | Cite as

Assessing the Novelty of Computer-Generated Narratives Using Empirical Metrics

  • Federico Peinado
  • Virginia Francisco
  • Raquel Hervás
  • Pablo Gervás


Novelty is a key concept to understand creativity. Evaluating a piece of artwork or other creation in terms of novelty requires comparisons to other works and considerations about the elements that have been reused in the creative process. Human beings perform this analysis intuitively, but in order to simulate it using computers, the objects to be compared and the similarity metrics to be used should be formalized and explicitly implemented. In this paper we present a study on relevant elements for the assessment of novelty in computer-generated narratives. We focus on the domain of folk-tales, working with simple plots and basic narrative elements: events, characters, props and scenarios. Based on the empirical results of this study we propose a set of computational metrics for the automatic assessment of novelty. Although oriented to the implementation of our own story generation system, the measurement methodology we propose can be easily generalized to other creative systems.


Similarity metrics Novelty assessment Story generation Creative systems Computational creativity 


  1. Boden, M. A. (1990). The creative mind: Myths and mechanisms. London: Weidenfeld & Nicholson.Google Scholar
  2. Gomes, P., Bento, C., & Gago, P. (1998). A function for novelty evaluation in design. Paper presented at the 4th international round-table conference on computational and cognitive models of creative design, Sydney.Google Scholar
  3. Loke, K. S. (2005). Notes on defining novelty for computational creativity. Paper presented at the 19th international joint conference on artificial intelligence, Edinburgh.Google Scholar
  4. Macedo, L., & Cardoso, A. (2002). Assessing creativity: The importance of unexpected novelty. Paper presented at the 2nd workshop on creative systems: Approaches to creativity in artificial intelligence and cognitive science, from the 15th European conference on artificial intelligence, Lyon.Google Scholar
  5. McGraw, G., & Hofstadter, D. R. (1993). Letter spirit: An architecture for creativity in a microdomain. Paper presented at the 3rd congress of the Italian association for artificial intelligence, Torino.Google Scholar
  6. Pease, A., Winterstein, D., & Colton, S. (2001). Evaluating machine creativity. Paper presented at the 1st workshop on creative systems: Approaches to creativity in artificial intelligence and cognitive science, from the 4th international conference on case-based reasoning, Washington.Google Scholar
  7. Peinado, F., & Gervás, P. (2006). Evaluation of automatic generation of basic stories. New Generation Computing, computational paradigms and computational intelligence, special issue: Computational creativity, 24(3), 289–302.Google Scholar
  8. Peinado, F. (2008). Un Armazón para el Desarrollo de Aplicaciones de Narración Automática basado en Componentes Ontológicos Reutilizables. Ph.D. Dissertation (in Spanish), Universidad Complutense de Madrid.Google Scholar
  9. Pérez y Pérez, R., & Sharples, M. (2001). Mexica: A computer model of a cognitive account of creative writing. Journal of Experimental and Theoretical Artificial Intelligence, 13(2), 119–139.CrossRefGoogle Scholar
  10. Pérez y Pérez, R., & Sharples, M. (2004). Three computer-based models of storytelling: Brutus, Minstrel and Mexica. International Journal on Knowledge Based Systems, 17(1), 15–29.CrossRefGoogle Scholar
  11. Propp, V. (1968). Morphology of the Folktale. Austin: University of Texas Press.Google Scholar
  12. Ritchie, G. (2001). Assessing creativity. Paper presented at the 3rd AISB symposium on artificial intelligence and creativity in arts and science, New York.Google Scholar
  13. Ritchie, G. (2007). Some empirical criteria for attributing creativity to a computer program. Minds & Machines, 17, 67–99.CrossRefGoogle Scholar
  14. Rosenman, M. A., & Gero, J. S. (1993). Creativity in design using a design prototype approach. In J. S. Gero, & M. L. Maher (Ed.), Modeling creativity and knowledge-based creative design (pp. 111–138). Hillsdale: Lawrence Erlbaum.Google Scholar
  15. Sharples, M. (1999). How we write: Writing as creative design. London: Routledge.Google Scholar
  16. Turner, S. R. (1992). Minstrel: A computer model of creativity and storytelling. Ph.D. Dissertation, University of California.Google Scholar
  17. Wiggins, G. A. (2006). A preliminary framework for description, analysis and comparison of creative systems. International Journal of Knowledge Based Systems, 19(7), 449–458.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  • Federico Peinado
    • 1
  • Virginia Francisco
    • 1
  • Raquel Hervás
    • 1
  • Pablo Gervás
    • 2
  1. 1.Departamento de Ingeniería del Software e Inteligencia Artificial, Facultad de InformáticaUniversidad Complutense de MadridMadridSpain
  2. 2.Instituto de Tecnología del Conocimiento, Universidad Complutense de MadridMadridSpain

Personalised recommendations