Abstract
While AI is being used more and more to generate images, the generation usually does not resemble a human painting process. However, for applications in the field of art, it is useful to simulate the human painting process—e.g. in relation to location, order, shape, color and contours of the areas being painted in each step. Such applications are for example when a robot paints a picture or a program teaches humans to paint. Consequently, in this paper we evaluate and compare different approaches to simulate the human painting process. Additionally, we present our solution for this task which is based on a combination of filters and semantic segmentation. In our survey, this approach was rated as better and more realistic than the most realistic approach for this task so far which is a reinforcement learning approach: In all surveyed categories—location, order, shape, color and contours of the areas being painted in each step—always a significant majority of the participants prefers our approach to simulate the human painting process. When we displayed two time-lapse videos with the painting process of Edvard Munch’s The Scream in parallel, even 79% found our generated process more realistic than the reinforcement learning-based process.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aubert, M., et al.: Pleistocene Cave Art from Sulawesi. Indonesia Nat. 514, 223–227 (2014)
Driscoll, S.: Painting. Salem Press Encyclopedia (2019)
earthryse: An AI-based Robot that Creates Fine Art Paintings (2021). https://earthryse.prowly.com/130623-an-ai-based-robot-that-creates-fine-art-paintings. Accessed 24 May 2021
3KICKS fine art studio: Sean Cheetham’s Demo in Advanced Portraiture Class (4/25/11) (2011). http://3kicks.blogspot.com/2011/05/sean-cheethams-demo-in-advanced.html. Accessed 24 May 2021
Durani, B.: Acrylic Painting Techniques: A Series of Nature Themed Acrylic Paintings. Ph.D. thesis, Yeshiva College, Yeshiva University (2020)
Nakano, R.: Neural Painters: A Learned Differentiable Constraint for Generating Brushstroke Paintings. ArXiv abs/1904.08410 (2019)
Reyner, N.: How to paint with layers - in acrylic and oil (2017). https://nancyreyner.com/2017/12/25/what-is-layering-for-painting/. Accessed 26 May 2021
Singh, J., Zheng, L.: Combining Semantic Guidance and Deep Reinforcement Learning for Generating Human Level Paintings. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16387–16396, June 2021
Kotovenko, D., Wright, M., Heimbrecht, A., Ommer, B.: Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12196–12205, June 2021
Zou, Z., Shi, T., Qiu, S., Yuan, Y., Shi, Z.: Stylized Neural Painting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15689–15698, June 2021
Liu, S., et al.: Paint Transformer: Feed Forward Neural Painting with Stroke Prediction. CoRR abs/2108.03798 (2021). https://arxiv.org/abs/2108.03798
Johansson, R.: Genetic Programming: Evolution of Mona Lisa (2008). https://rogerjohansson.blog/2008/12/07/genetic-programming-evolution-of-mona-lisa. Accessed 24 May 2021
Google Creative Lab: The Quick, Draw! Dataset. https://github.com/googlecreativelab/quickdraw-dataset (2017), accessed: 2021–05-24
Diaz-Aviles, E.: Dreaming of Electric Sheep (2018). https://medium.com/libreai/dreaming-of-electric-sheep-d1aca32545dc. Accessed 24 May 2021
Zhao, A., Balakrishnan, G., Lewis, K.M., Durand, F., Guttag, J., Dalca, A.V.: Painting Many Pasts: Synthesizing Time Lapse Videos of Paintings. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8432–8442 (2020)
Ganin, Y., Kulkarni, T., Babuschkin, I., Eslami, S.M.A., Vinyals, O.: Synthesizing Programs for Images using Reinforced Adversarial Learning. In: Dy, J.G., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10–15, 2018, Proceedings of Machine Learning Research, vol. 80, pp. 1652–1661. PMLR (2018)
Huang, Z., Zhou, S., Heng, W.: Learning to Paint with Model-based Deep Reinforcement Learning. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8708–8717 (2019)
François-Lavet, V., Henderson, P., Islam, R., Bellemare, M.G., Pineau, J.: An Introduction to Deep Reinforcement Learning. Found. Trends Mach. Learn. 11(3–4), 219–354 (2018)
Culjak, I., Abram, D., Pribanic, T., Dzapo, H., Cifrek, M.: A Brief Introduction to OpenCV. In: 2012 Proceedings of the 35th International Convention MIPRO, pp. 1725–1730 (2012)
Kim, W., Kanezaki, A., Tanaka, M.: Unsupervised Learning of Image Segmentation based on Differentiable Feature Clustering. IEEE Trans. Image Process. 29, 8055–8068 (2020). https://doi.org/10.1109/TIP.2020.3011269
Pessoa, T., Medeiros, R., Nepomuceno, T., Bian, G.B., Albuquerque, V., Filho, P.P.: Performance Analysis of Google Colaboratory as a Tool for Accelerating Deep Learning Applications, p. 1. IEEE Access (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Leiser, A., Schlippe, T. (2022). AI in Art: Simulating the Human Painting Process. In: Wölfel, M., Bernhardt, J., Thiel, S. (eds) ArtsIT, Interactivity and Game Creation. ArtsIT 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 422. Springer, Cham. https://doi.org/10.1007/978-3-030-95531-1_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-95531-1_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95530-4
Online ISBN: 978-3-030-95531-1
eBook Packages: Computer ScienceComputer Science (R0)