Interaction Design for Metacreative Systems

  • Oliver BownEmail author
  • Andrew R. Brown
Part of the Human–Computer Interaction Series book series (HCIS)


In this paper, we examine digital creativity as a collective activity performed through socio-technological networks of agency. We introduce metacreation—the automation of creative tasks with machines—as a domain that is usefully examined from a 3rd wave HCI approach. We discuss four general human-computer interaction activities that commonly appear in metacreation: (1) metagenerating form; (2) searching/finding; (3) helping machines learn; and (4) evaluation/iteration. These are not necessarily specific to metacreation, but nevertheless point to particular design considerations in a metacreative context. Four creative interaction design themes are considered in their relation to metacreation: direct manipulation and real-time control; supporting playful interaction and divergent goals; the programmatic design of behaviours, and; managing distributed creativity. We then identify three paradigms of interaction design for metacreation: operation-based interaction, involving the direct manipulation of generative algorithms; request-based interaction, involving the submission of requests to a system that returns results; and ambient interaction, that involves the operation of autonomous metacreative processes in the background. Our discussion of these suggests possible trends for design: an increasingly complex and modular future for networked human-machine digital creativity; an increasing role for request-based metacreative systems where users specify, rather than construct, outcomes; the increasing role of metacreation in ‘prosumer’ content creation; and, consequently, the reduction of labour involved in creating media. The chapter makes clear, we hope, that metacreative practices present unique challenges and opportunities for interaction design.


  1. Amabile T (1996) Creativity in context. Westview Press, BoulderGoogle Scholar
  2. Besold TR, Schorlemmer M, Smaill A (eds) (2015) Computational creativity research: towards creative machines. Atlantis Press, BerlinGoogle Scholar
  3. Blackwell AF (2002) First steps in programming: a rationale for attention investment models. In: Proceedings of the IEEE 2002 Symposia on human centric computing languages and environments, 2002. IEEE, Piscataway, pp 2–10Google Scholar
  4. Bloom P (2010) How pleasure works: the new science of why we like what we like. Random House, New YorkGoogle Scholar
  5. Bødker S (2006) When second wave HCI meets third wave challenges. In: Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles. ACM, New York, pp 1–8Google Scholar
  6. Bødker S (2015) Third-wave HCI, 10 years later—participation and sharing. Interactions 22(5):24–31CrossRefGoogle Scholar
  7. Bown O (2011) Experiments in modular design for the creative composition of live algorithms. Comput Music J 35(3):73–85CrossRefGoogle Scholar
  8. Bown O (2012) Generative and adaptive creativity: a unified approach to creativity in nature, humans and machines. In: Computers and creativity. Springer, Berlin/Heidelberg, pp 361–381CrossRefGoogle Scholar
  9. Bown O (2014) Empirically grounding the evaluation of creative systems: incorporating interaction design. In: Proceedings of the fifth international conference on computational creativity, pp 112–119Google Scholar
  10. Bray L, Bown O (2017) How can we deal with the design principle of visibility in highly encapsulated computationally creative systems? In: Proceedings of the international conference on computational creativity, June 19–23, AtlantaGoogle Scholar
  11. Brown AR (2016) Understanding musical practices as agency networks. In: Proceedings of the international conference on computational creativity. [Online]. 2016 Paris, Association of Computational Creativity. Available from:
  12. Brown AR, Sorensen A (2009) Interacting with generative music through live coding. Contemp Music Rev 28(1):17–29CrossRefGoogle Scholar
  13. Brown A, Gifford T, Voltz B (2013) Controlling interactive music performance (CIM). In: Proceedings of the fourth international conference on computational creativity, vol 221Google Scholar
  14. Bruckner S, Gröller ME (2007) Style transfer functions for illustrative volume rendering. In: Computer graphics forum (Vol. 26, No. 3, pp. 715–724). Blackwell Publishing Ltd, CambridgeCrossRefGoogle Scholar
  15. Clark A, Chalmers DJ (1998) The extended mind. Analysis 58:10–23CrossRefGoogle Scholar
  16. Cohen H (1995) The further exploits of AARON, painter. Stanf Humanit Rev 4(2):141–158Google Scholar
  17. Cope D, Mayer MJ (1996) Experiments in musical intelligence, vol 12. AR Editions, MadisonGoogle Scholar
  18. Davis L (1991) Handbook of genetic algorithms. Van Nostrand Reinhold, New YorkGoogle Scholar
  19. Di Scipio A (2003) Sound is the interface: from interactive to ecosystemic signal processing. Organised Sound 8(03):269–277CrossRefGoogle Scholar
  20. Draves S (1999) Electric sheep [Online]. Spotworks LLC. Available from:
  21. Eno B, Chilvers P (2008–2016) Bloom, trope, air, quarta & reflection. iPhone App Store, Opal LimitedGoogle Scholar
  22. Ferguson J, Brown AR (2016) Fostering a post-digital avant-garde: research-led teaching of music technology. Organised Sound 21(2):127–137CrossRefGoogle Scholar
  23. Galanter P (2003) What is generative art? Complexity theory as a context for art theory. In: GA2003–6th generative art conference. 10–12 December, 2003, Milan, ItalyGoogle Scholar
  24. Gaver WW, Bowers J, Boucher A, Gellerson H et al (2004) The drift table: designing for ludic engagement. In: CHI’04 extended abstracts on Human factors in computing systems. ACM, New York, pp 885–900Google Scholar
  25. Ghedini F, Pachet F, Roy P (2016) Creating music and texts with flow machines. In: Multidisciplinary contributions to the science of creative thinking. Springer, Singapore, pp 325–343CrossRefGoogle Scholar
  26. Harrison S, Tatar D, Sengers P (2007) The three paradigms of HCI. In Alt. Chi. Session at the SIGCHI conference on human factors in computing systems. San Jose, California, USA, pp 1–18Google Scholar
  27. Hudson R (2008) Frameworks for practical parametric design in architecture. In: Proceedings of the 26th eCAADe conference. Antwerp, Belgium, pp. 847–854Google Scholar
  28. Kantosalo A, Toivanen JM, Toivonen HT (2015) Interaction evaluation for human-computer co-creativity. In: Proceedings of the sixth international conference on computational creativity. June 29–July 2, Park City, Utah, USAGoogle Scholar
  29. Konak A, Coit DW, Smith AE (2006) Multi-objective optimization using genetic algorithms: a tutorial. Reliab Eng Syst Saf 91(9):992–1007CrossRefGoogle Scholar
  30. Langston P (1989) Six techniques for algorithmic music composition. In: 15th international computer music conference. ACMA, Columbus, pp 2–5Google Scholar
  31. Latour B (1995) Mixing humans and nonhuman together: the sociology of a door-closer. In: Ecologies of Knowledge. New York, State University of New York Press, pp 257–277Google Scholar
  32. Latour B (2011) What’s the story? Organizing as a mode of existence. In: Passoth JH, Peuker B, Schillmeier M (eds) Agency without actors. Routledge, LondonGoogle Scholar
  33. Lehman J, Stanley KO (2008) Exploiting open-endedness to solve problems through the search for novelty. In: Proceedings of the Eleventh International Conference on Artificial Life (ALIFE XI). MIT Press, Cambridge, MA, pp 329–336Google Scholar
  34. Lessig L (2008) Remix: making art and commerce thrive in the hybrid economy. Penguin, New YorkCrossRefGoogle Scholar
  35. Lubart T (2005) How can computers be partners in the creative process: classification and commentary on the special issue. Int J of Hum Comput Stud 63(4):365–369CrossRefGoogle Scholar
  36. MacCallum B, Leroi A, Mauch M, Welburn S et al (2009) DarwinTunes [Online]. London, Imperial College and Queen Mary University. Available from:
  37. MacCallum RM, Mauch M, Burt A, Leroi AM (2012) Evolution of music by public choice. Proc Natl Acad Sci 109(30):12081–12086CrossRefGoogle Scholar
  38. Malafouris L (2008) At the potter’s wheel: an argument for material agency. In: Knappett C, Malafouris L (eds) Material agency: towards a non-anthropocentric approach. Springer, New York, pp 19–36CrossRefGoogle Scholar
  39. Martin A, Jin CT, Bown O (2011) A toolkit for designing interactive musical agents. In: Proceedings of the 23rd Australian computer-human interaction conference. November 28–December 02, 2011, Canberra, Australia, pp 194–197Google Scholar
  40. McCormack J (2004) Aesthetic evolution of L-systems revisited. In: Workshops on applications of evolutionary computation. Springer, Berlin/Heidelberg, pp 477–488Google Scholar
  41. McCormack J, D’Inverno M (eds) (2012) Computers and creativity. Springer, BerlinGoogle Scholar
  42. Nielsen, Jakob. 1993. Usability Engineering. New York: Morgan Kaufmann, pp 115–164CrossRefGoogle Scholar
  43. Norman D (1988) The psychology of everyday things. Basic books, New YorkGoogle Scholar
  44. Pask G (1976) Conversation theory. Applications in education and epistemology. Elsevier, Amsterdam/New YorkGoogle Scholar
  45. Perkins DN (1994) Creativity: beyond the darwinian paradigm. In Boden M (ed), Dimensions of creativity, Bradford Books, Cambridge, MA. pp 119–142Google Scholar
  46. Plans Casal D, Morelli D (2007) Remembering the future: an overview of co-evolution in musical improvisation. In: Proceedings of the international computer music conference. Copenhagen, International Computer Music Association, pp 200–206Google Scholar
  47. Rammert W (2008) Where the action is: distributed agency between humans, machines, and programsGoogle Scholar
  48. Salganik MJ, Watts DJ (2008) Leading the herd astray: an experimental study of self-fulfilling prophecies in an artificial cultural market. Soc Psychol Q 71(4):338–355CrossRefGoogle Scholar
  49. Secretan J, Beato N, D Ambrosio DB, Rodriguez A, Campbell A, Stanley KO (2008) Picbreeder: evolving pictures collaboratively online. In: Proceedings of the SIGCHI conference on human factors in computing systems. April 27–May 02, 2013, Paris, France, pp 1759–1768Google Scholar
  50. Shneiderman B (1993) 1.1 Direct manipulation: a step beyond programming languages. Sparks of innovation in human-computer interaction, 17Google Scholar
  51. Simonton DK (1999) Origins of genius: Darwinian perspectives on creativity. Oxford University Press, New YorkGoogle Scholar
  52. Suchman LA (1987) Plans and situated actions: the problem of human-machine communication. Cambridge university press, New YorkGoogle Scholar
  53. Togelius J, Yannakakis GN, Stanley KO, Browne C (2011) Search-based procedural content generation: a taxonomy and survey. IEEE Trans Comput Intell AI Games 3(3):172–186CrossRefGoogle Scholar
  54. Ventura D (2016) Mere generation: essential barometer or dated concept. In: Proceedings of the seventh international conference on computational creativity, ICCC. June 27–July 1, Paris, FranceGoogle Scholar
  55. Wallas G (1926) The art of thought. Harcourt Brace, New YorkGoogle Scholar
  56. Whitelaw M (2004) Metacreation: art and artificial life. The MIT Press, Cambridge, MAGoogle Scholar
  57. Winograd T, Flores F (1986) Understanding computers and cognition: a new foundation for design. Addison Wesley, ReadingzbMATHGoogle Scholar
  58. Woolley BG, Stanley KO (2014) A novel human-computer collaboration: combining novelty search with interactive evolution. In: Proceedings of the 2014 annual conference on genetic and evolutionary computation, 12–16 July, 2014, Vancouver, BC, Canada, pp 233–240Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of New South WalesSydneyAustralia
  2. 2.Griffith UniversityNathanAustralia

Personalised recommendations