Skip to main content
SpringerLink
Log in

Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts

  • Conference paper
  • First Online:
Artificial Intelligence in Music, Sound, Art and Design (EvoMUSART 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13221))

Abstract

Evolutionary algorithms (ES) have been used in the digital art scene since the 1970s. A popular application of genetic algorithms is to optimize the procedural placement of vector graphic primitives to resemble a given painting. In recent years, deep learning-based approaches have also been proposed to generate procedural drawings, which can be optimized using gradient descent. In this work, we revisit the use of evolutionary algorithms for computational creativity. We find that modern ES algorithms, when tasked with the placement of shapes, offer large improvements in both quality and efficiency compared to traditional genetic algorithms, and even comparable to gradient-based methods. We demonstrate that ES is also well suited at optimizing the placement of shapes to fit the CLIP model, and can produce diverse, distinct geometric abstractions that are aligned with human interpretation of language.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alteredqualia. Evolution of mona lisa in javascript and canvas (2008). https://alteredqualia.com/visualization/evolve/. Accessed 1 Dec 2021

  2. Bertoni, F.: Minimalist architecture. Birkhäuser (2002)

    Google Scholar 

  3. Beyer, H.G.: The Theory of Evolution Strategies. Springer, Heidelberg (2001). https://doi.org/10.1007/978-3-662-04378-3

  4. Beyer, H.G., Schwefel, H.P.: Evolution strategies-a comprehensive introduction. Nat. Comput. 1(1), 3–52 (2002)

    Article  MathSciNet  Google Scholar 

  5. Cason, K.: Genetic Draw (2016). https://github.com/kennycason/genetic_draw. Accessed 1 Dec 2021

  6. Commons, W.: File:040 okertalsperre.jpg – wikimedia commons the free media repository (2020). https://commons.wikimedia.org/w/index.php?title=File:040_Okertalsperre.jpg&oldid=496749636. Accessed 1 Dec 2021

  7. Crowson, K.: Introduction to vqgan+clip (2021). https://docs.google.com/document/d/1Lu7XPRKlNhBQjcKr8k8qRzUzbBW7kzxb5Vu72GMRn2E/edit. Accessed 1 Dec 2021

  8. Dabrowski, M.: Geometric abstraction (2004). https://www.metmuseum.org/toah/hd/geab/hd_geab.htm. Accessed 1 Dec 2021

  9. Fernando, C.: Royal academy summer exhibition 2021 submission (2021). https://www.chrisantha.co.uk/post/royal-academy-summer-exhibition-2021-submission. Accessed 1 Dec 2021

  10. Fernando, C., Eslami, S., Alayrac, J.B., Mirowski, P., Banarse, D., Osindero, S.: Generative art using neural visual grammars and dual encoders. arXiv preprint arXiv:2105.00162 (2021)

  11. Fogleman, M.: Primitive pictures (2016). https://github.com/fogleman/primitive. Accessed 1 Dec 2021

  12. Frans, K., Soros, L., Witkowski, O.: Clipdraw: exploring text-to-drawing synthesis through language-image encoders. arXiv preprint arXiv:2106.14843 (2021)

  13. Galatolo, F.A., Cimino, M.G., Vaglini, G.: Generating images from caption and vice versa via clip-guided generative latent space search. arXiv preprint arXiv:2102.01645 (2021)

  14. Gonsalves, R.A.: Ganscapes: using AI to create new impressionist paintings (2021). https://towardsdatascience.com/ganscapes-using-ai-to-create-new-impressionist-paintings-d6af1cf94c56. Accessed 1 Dec 2021

  15. Ha, D.: Evolving stable strategies. blog.otoro.net (2017). http://blog.otoro.net/2017/11/12/evolving-stable-strategies/. Accessed 1 Dec 2021

  16. Ha, D., Eck, D.: A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 (2017)

  17. Harvey, I.: The microbial genetic algorithm. In: Kampis, G., Karsai, I., Szathmáry, E. (eds.) ECAL 2009. LNCS (LNAI), vol. 5778, pp. 126–133. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21314-4_16

  18. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  19. Huang, F., Schoop, E., Ha, D., Canny, J.: Scones: towards conversational authoring of sketches. In: Proceedings of the 25th International Conference on Intelligent User Interfaces, pp. 313–323 (2020)

    Google Scholar 

  20. Huang, Z., Heng, W., Zhou, S.: Learning to paint with model-based deep reinforcement learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8709–8718 (2019)

    Google Scholar 

  21. Jia, C., et al.: Scaling up visual and vision-language representation learning with noisy text supervision. In: International Conference on Machine Learning, pp. 4904–4916. PMLR (2021)

    Google Scholar 

  22. Johansson, R.: Genetic programming: evolution of mona lisa (2008). https://rogerjohansson.blog/2008/12/07/genetic-programming-evolution-of-mona-lisa/. Accessed 1 Dec 2021

  23. Kato, H., et al.: Differentiable rendering: a survey. arXiv preprint arXiv:2006.12057 (2020)

  24. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  25. Kolmogorov, A.: Three approaches to the quantitative definition of information. Probl. Inf. Trans. 1(1), 1–7 (1965)

    Google Scholar 

  26. Kuiper, K.: Modernism (2021). https://www.britannica.com/art/Modernism-art. Accessed 1 Dec 2021

  27. Laine, S., Hellsten, J., Karras, T., Seol, Y., Lehtinen, J., Aila, T.: Modular primitives for high-performance differentiable rendering. ACM Trans. Graph. 39(6), 1–14 (2020)

    Article  Google Scholar 

  28. Li, T.M., Lukáč, M., Gharbi, M., Ragan-Kelley, J.: Differentiable vector graphics rasterization for editing and learning. ACM Trans. Graph. 39(6), 1–15 (2020)

    Google Scholar 

  29. Lindenmayer, A.: Mathematical models for cellular interactions in development i. filaments with one-sided inputs. J. Theoretic. Biol. 18(3), 280–299 (1968)

    Article  Google Scholar 

  30. Liu, S., et al.: Paint transformer: feed forward neural painting with stroke prediction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6598–6607 (2021)

    Google Scholar 

  31. Lopes, R.G., Ha, D., Eck, D., Shlens, J.: A learned representation for scalable vector graphics. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7930–7939 (2019)

    Google Scholar 

  32. Malkevitchn, J.: Mathematics and art (2003). https://www.ams.org/publicoutreach/feature-column/fcarc-art1. Accessed 1 Dec 2021

  33. Mellor, J.F., et al.: Unsupervised doodling and painting with improved spiral. arXiv preprint arXiv:1910.01007 (2019)

  34. Modern, T.: Minimalism (2018). https://www.tate.org.uk/art/art-terms/m/minimalism. Accessed 1 Dec 2021

  35. Paauw, M., van den Berg, D.: Paintings, polygons and plant propagation. In: Ekárt, A., Liapis, A., Castro Pena, M. (eds.) EvoMUSART 2019. LNCS, vol. 11453, pp. 84–97. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-16667-0_6

    Chapter  Google Scholar 

  36. Paul, S.: Abstract expressionism (2004). https://www.metmuseum.org/toah/hd/abex/hd_abex.htm. Accessed 1 Dec 2021

  37. Radford, A., et al.: Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020 (2021)

  38. Rewald, S.: Cubism. The Metropolitan Museum of Art (2014). https://www.metmuseum.org/toah/hd/cube/hd_cube.htm. Accessed 1 Dec 2021

  39. Rose, B.: Abc art. Art Am. 53(5), 57–69 (1965)

    Google Scholar 

  40. Salimans, T., Ho, J., Chen, X., Sidor, S., Sutskever, I.: Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864 (2017)

  41. Schmidhuber, J.: Low-complexity art. Leonardo 30(2), 97–103 (1997)

    Article  Google Scholar 

  42. Sehnke, F., Osendorfer, C., Rückstieß, T., Graves, A., Peters, J., Schmidhuber, J.: Parameter-exploring policy gradients. Neural Netw. 23(4), 551–559 (2010)

    Article  Google Scholar 

  43. Shahrabi, S.: Procedural paintings with genetic evolution algorithm (2020). https://github.com/IRCSS/Procedural-painting. Accessed 1 Dec 2021

  44. Sizigi Studios. Waifu labs (2019). https://waifulabs.com/. Accessed 1 Dec 2021

  45. Tate. Process art (2021). https://www.tate.org.uk/art/art-terms/p/process-art. Accessed 1 Dec 2021

  46. Toklu, N.E.: Pgpelib (2020). https://github.com/nnaisense/pgpelib. Accessed 1 Dec 2021

  47. Toklu, N.E., Liskowski, P., Srivastava, R.K.: Clipup: a simple and powerful optimizer for distribution-based policy evolution. In: International Conference on Parallel Problem Solving from Nature, pp. 515–527. Springer (2020)

    Google Scholar 

  48. Verostko, R.: Algorithmic art (1994). http://www.verostko.com/algorithm.html. Accessed 1 Dec 2021

  49. Wang, P.: Big sleep: a simple command line tool for text to image generation, using openai’s clip and a biggan (2021). https://github.com/lucidrains/big-sleep. Accessed 1 Dec 2021

  50. White, T.: Shared visual abstractions. arXiv preprint arXiv:1912.04217 (2019)

Download references

Acknowledgements

We thank Toru Lin, Jerry Li, Yujin Tang, Yanghua Jin, Jesse Engel, Yifu Zhao for their comments, suggestions and kind helps.

Author information

Authors and Affiliations

  1. Google Research, Brain Team, Tokyo, Japan

    Yingtao Tian & David Ha

Authors
  1. Yingtao Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Ha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingtao Tian .

Editor information

Editors and Affiliations

  1. University of Coimbra, Coimbra, Portugal

    Tiago Martins

  2. University of A Coruña, A Coruña, Spain

    Nereida Rodríguez-Fernández

  3. University of Coimbra, Coimbra, Portugal

    Sérgio M. Rebelo

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tian, Y., Ha, D. (2022). Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts. In: Martins, T., Rodríguez-Fernández, N., Rebelo, S.M. (eds) Artificial Intelligence in Music, Sound, Art and Design. EvoMUSART 2022. Lecture Notes in Computer Science, vol 13221. Springer, Cham. https://doi.org/10.1007/978-3-031-03789-4_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-03789-4_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-03788-7

  • Online ISBN: 978-3-031-03789-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation