Skip to main content
Book cover

International Conference on ArtsIT, Interactivity and Game Creation

ArtsIT 2021: ArtsIT, Interactivity and Game Creation pp 295–308Cite as

AI in Art: Simulating the Human Painting Process

  • 315 Accesses

Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST,volume 422)

Abstract

While AI is being used more and more to generate images, the generation usually does not resemble a human painting process. However, for applications in the field of art, it is useful to simulate the human painting process—e.g. in relation to location, order, shape, color and contours of the areas being painted in each step. Such applications are for example when a robot paints a picture or a program teaches humans to paint. Consequently, in this paper we evaluate and compare different approaches to simulate the human painting process. Additionally, we present our solution for this task which is based on a combination of filters and semantic segmentation. In our survey, this approach was rated as better and more realistic than the most realistic approach for this task so far which is a reinforcement learning approach: In all surveyed categories—location, order, shape, color and contours of the areas being painted in each step—always a significant majority of the participants prefers our approach to simulate the human painting process. When we displayed two time-lapse videos with the painting process of Edvard Munch’s The Scream in parallel, even 79% found our generated process more realistic than the reinforcement learning-based process.

Keywords

  • AI art
  • AI in Art
  • Painting
  • Semantic Segmentation
  • Semantic Labeling
  • Machine Learning

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-95531-1_20
  • Chapter length: 14 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-95531-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.
Fig. 13.
Fig. 14.

Notes

  1. 1.

    https://ainorn.art

  2. 2.

    https://www.cloudpainter.com

  3. 3.

    https://www.nvidia.com/en-us/deep-learning-ai/ai-art-gallery

  4. 4.

    https://vialps.com

References

  1. Aubert, M., et al.: Pleistocene Cave Art from Sulawesi. Indonesia Nat. 514, 223–227 (2014)

    Google Scholar 

  2. Driscoll, S.: Painting. Salem Press Encyclopedia (2019)

    Google Scholar 

  3. earthryse: An AI-based Robot that Creates Fine Art Paintings (2021). https://earthryse.prowly.com/130623-an-ai-based-robot-that-creates-fine-art-paintings. Accessed 24 May 2021

  4. 3KICKS fine art studio: Sean Cheetham’s Demo in Advanced Portraiture Class (4/25/11) (2011). http://3kicks.blogspot.com/2011/05/sean-cheethams-demo-in-advanced.html. Accessed 24 May 2021

  5. Durani, B.: Acrylic Painting Techniques: A Series of Nature Themed Acrylic Paintings. Ph.D. thesis, Yeshiva College, Yeshiva University (2020)

    Google Scholar 

  6. Nakano, R.: Neural Painters: A Learned Differentiable Constraint for Generating Brushstroke Paintings. ArXiv abs/1904.08410 (2019)

    Google Scholar 

  7. Reyner, N.: How to paint with layers - in acrylic and oil (2017). https://nancyreyner.com/2017/12/25/what-is-layering-for-painting/. Accessed 26 May 2021

  8. Singh, J., Zheng, L.: Combining Semantic Guidance and Deep Reinforcement Learning for Generating Human Level Paintings. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16387–16396, June 2021

    Google Scholar 

  9. Kotovenko, D., Wright, M., Heimbrecht, A., Ommer, B.: Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12196–12205, June 2021

    Google Scholar 

  10. Zou, Z., Shi, T., Qiu, S., Yuan, Y., Shi, Z.: Stylized Neural Painting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15689–15698, June 2021

    Google Scholar 

  11. Liu, S., et al.: Paint Transformer: Feed Forward Neural Painting with Stroke Prediction. CoRR abs/2108.03798 (2021). https://arxiv.org/abs/2108.03798

  12. Johansson, R.: Genetic Programming: Evolution of Mona Lisa (2008). https://rogerjohansson.blog/2008/12/07/genetic-programming-evolution-of-mona-lisa. Accessed 24 May 2021

  13. Google Creative Lab: The Quick, Draw! Dataset. https://github.com/googlecreativelab/quickdraw-dataset (2017), accessed: 2021–05-24

  14. Diaz-Aviles, E.: Dreaming of Electric Sheep (2018). https://medium.com/libreai/dreaming-of-electric-sheep-d1aca32545dc. Accessed 24 May 2021

  15. Zhao, A., Balakrishnan, G., Lewis, K.M., Durand, F., Guttag, J., Dalca, A.V.: Painting Many Pasts: Synthesizing Time Lapse Videos of Paintings. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8432–8442 (2020)

    Google Scholar 

  16. Ganin, Y., Kulkarni, T., Babuschkin, I., Eslami, S.M.A., Vinyals, O.: Synthesizing Programs for Images using Reinforced Adversarial Learning. In: Dy, J.G., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10–15, 2018, Proceedings of Machine Learning Research, vol. 80, pp. 1652–1661. PMLR (2018)

    Google Scholar 

  17. Huang, Z., Zhou, S., Heng, W.: Learning to Paint with Model-based Deep Reinforcement Learning. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8708–8717 (2019)

    Google Scholar 

  18. François-Lavet, V., Henderson, P., Islam, R., Bellemare, M.G., Pineau, J.: An Introduction to Deep Reinforcement Learning. Found. Trends Mach. Learn. 11(3–4), 219–354 (2018)

    CrossRef  Google Scholar 

  19. Culjak, I., Abram, D., Pribanic, T., Dzapo, H., Cifrek, M.: A Brief Introduction to OpenCV. In: 2012 Proceedings of the 35th International Convention MIPRO, pp. 1725–1730 (2012)

    Google Scholar 

  20. Kim, W., Kanezaki, A., Tanaka, M.: Unsupervised Learning of Image Segmentation based on Differentiable Feature Clustering. IEEE Trans. Image Process. 29, 8055–8068 (2020). https://doi.org/10.1109/TIP.2020.3011269

    CrossRef  Google Scholar 

  21. Pessoa, T., Medeiros, R., Nepomuceno, T., Bian, G.B., Albuquerque, V., Filho, P.P.: Performance Analysis of Google Colaboratory as a Tool for Accelerating Deep Learning Applications, p. 1. IEEE Access (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IU International University of Applied Sciences, Bad Honnef, Germany

    Alexander Leiser & Tim Schlippe

Authors
  1. Alexander Leiser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tim Schlippe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tim Schlippe .

Editor information

Editors and Affiliations

  1. Karlsruhe University of Applied Sciences, Karlsruhe, Germany

    Prof. Dr. Matthias Wölfel

  2. Baden State Museum, Karlsruhe, Germany

    Dr. Johannes Bernhardt

  3. Baden State Museum, Karlsruhe, Germany

    Sonja Thiel

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Leiser, A., Schlippe, T. (2022). AI in Art: Simulating the Human Painting Process. In: Wölfel, M., Bernhardt, J., Thiel, S. (eds) ArtsIT, Interactivity and Game Creation. ArtsIT 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 422. Springer, Cham. https://doi.org/10.1007/978-3-030-95531-1_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-95531-1_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-95530-4

  • Online ISBN: 978-3-030-95531-1

  • eBook Packages: Computer ScienceComputer Science (R0)