Skip to main content
SpringerLink
Log in

Artificial Neural Networks and Deep Learning in the Visual Arts: a review

  • S. I : Neural Networks in Art, sound and Design
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this article, we perform an exhaustive analysis of the use of Artificial Neural Networks and Deep Learning in the Visual Arts. We begin by introducing changes in Artificial Intelligence over the years and examine in depth the latest work carried out in prediction, classification, evaluation, generation, and identification through Artificial Neural Networks for the different Visual Arts. While we highlight the contributions of photography and pictorial art, there are also other uses for 3D modeling, including video games, architecture, and comics. The results of the investigations discussed show that the use of Artificial Neural Networks in the Visual Arts continues to evolve and have recently experienced significant growth. To complement the text, we include a glossary and table with information about the most commonly employed image datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Availability of data and material

No data were used to support this study.

References

  1. McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5(4):115–133

    MathSciNet  MATH  Google Scholar 

  2. Wani IM, Arora S (2020) Deep neural networks for diagnosis of osteoporosis: a review. In: Proceedings of ICRIC 2019. Springer, pp 65–78

  3. Negassi M, Suarez-Ibarrola R, Hein S, Miernik A, Reiterer A (2020) Application of artificial neural networks for automated analysis of cystoscopic images: a review of the current status and future prospects. World J Urol 38:2349–2358. https://doi.org/10.1007/s00345-019-03059-0

    Article  Google Scholar 

  4. Moon S, Ahmadnezhad P, Song H-J, Thompson J, Kipp K, Akinwuntan AE, Devos H (2020) Artificial neural networks in neurorehabilitation: a scoping review. NeuroRehabilitation 1–11

  5. Yucel M, Nigdeli SM, Bekdaş G (2020) Artificial neural networks (anns) and solution of civil engineering problems: anns and prediction applications. In: Artificial intelligence and machine learning applications in civil, mechanical, and industrial engineering. IGI Global, pp 13–37

  6. Pradhan B, Sameen MI (2020) Review of traffic accident predictions with neural networks. In: Laser scanning systems in highway and safety assessment. Springer, Cham, pp 97–109

    Google Scholar 

  7. Kalogirou SA (2001) Artificial neural networks in renewable energy systems applications: a review. Renew Sustain Energy Rev 5(4):373–401

    Google Scholar 

  8. Sundararaj A, Ravi R, Thirumalai P, Radhakrishnan G (1999) Artificial neural network applications in electrochemistry—a review. Bull Electrochem 15(12):552–555

    Google Scholar 

  9. Risi S, Togelius J (2015) Neuroevolution in games: state of the art and open challenges. IEEE Trans Comput Intell AI Games 9(1):25–41

    Google Scholar 

  10. Lipton ZC, Berkowitz J, Elkan C A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:1506.00019

  11. Mammone RJ (1994) Artificial neural networks for speech and vision, vol 4. Chapman & Hall, London

    Google Scholar 

  12. Lippmann RP (1989) Review of neural networks for speech recognition. Neural Comput 1(1):1–38

    Google Scholar 

  13. Kamble BC (2016) Speech recognition using artificial neural network—a review. Int J Comput Commun Instrum Eng 3(1):61–64

    Google Scholar 

  14. Dias FM, Antunes A, Mota AM (2004) Artificial neural networks: a review of commercial hardware. Eng Appl Artif Intell 17(8):945–952

    Google Scholar 

  15. Giebel H (1971) Feature extraction and recognition of handwritten characters by homogeneous layers. In: Zeichenerkennung durch biologische und technische Systeme/Pattern Recognition in Biological and Technical Systems. Springer, pp 162–169

  16. Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern 36(4):193–202

    MATH  Google Scholar 

  17. LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551

    Google Scholar 

  18. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  19. Berg A, Deng J, Fei-Fei L (2010) Large scale visual recognition challenge (ilsvrc). http://www.image-net.org/challenges/LSVRC, p 3

  20. Le QV et al (2015) A tutorial on deep learning part 2: autoencoders, convolutional neural networks and recurrent neural networks. Google Brain 1–20

  21. Wu J (2017) Introduction to convolutional neural networks, National Key Lab for Novel Software Technology. Nanjing Univ China 5:23

    Google Scholar 

  22. Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath AA (2018) Generative adversarial networks: an overview. IEEE Signal Process Mag 35(1):53–65

    Google Scholar 

  23. Romero JJ (2008) The art of artificial evolution: a handbook on evolutionary art and music. Springer, Berlin

    Google Scholar 

  24. Romero J (2020) Artificial intelligence in music, sound, art and design: 9th international conference, EvoMUSART 2020, Held as Part of EvoStar 2020, Seville, Spain, April 15–17, 2020, Proceedings. Springer

  25. Association for computational creativity. https://computationalcreativity.net/home/

  26. The bridges conference. https://www.bridgesmathart.org/

  27. Evomusart. https://evomusart-index.dei.uc.pt/

  28. Dorin A (2015) Artificial life art, creativity, and techno-hybridization (editor’s introduction). Artif Life 21(3):261–270

    Google Scholar 

  29. Greenfield G, Machado P (2012) Guest editor’s introduction, special issue on mathematical models used in aesthetic evaluation. J Math Arts 6(2–3):59–64

    MathSciNet  Google Scholar 

  30. Romero J, Johnson C, McCormack J (2019) Complex systems in aesthetics and arts. Complexity 2019:2. https://doi.org/10.1155/2019/9836102

    Article  Google Scholar 

  31. Galanter P (2012) Computational aesthetic evaluation: steps towards machine creativity. In: ACM SIGGRAPH 2012 courses. pp 1–162

  32. Spratt EL, Elgammal A (2014) Computational beauty: aesthetic judgment at the intersection of art and science. In: European conference on computer vision. Springer, pp 35–53

  33. Toivonen H, Gross O (2015) Data mining and machine learning in computational creativity. Data Min Knowl Disc 5(6):265–275

    Google Scholar 

  34. Upadhyaya N, Dixit M, Pradesh DM (2016) A review: relating low level features to high level semantics in cbir. Int J Signal Process Image Process Pattern Recognit 9(3):433–444

    Google Scholar 

  35. Johnson CG, McCormack J, Santos I, Romero J (2019) Understanding aesthetics and fitness measures in evolutionary art systems. Complexity 2019:14. https://doi.org/10.1155/2019/3495962

    Article  Google Scholar 

  36. Todd PM, Werner GM (1999) Frankensteinian methods for evolutionary music. Musical networks: parallel distributed perception and performance, pp 313–340

  37. Lewis M (2008) Evolutionary visual art and design. In: The art of artificial evolution. Springer, Berlin, pp 3–37

  38. Briot J-P, Hadjeres G, Pachet F-D, Deep learning techniques for music generation—a survey. arXiv preprint arXiv:1709.01620

  39. Briot J-P, Hadjeres G, Pachet F (2019) Deep learning techniques for music generation, vol 10. Springer, Berlin

    Google Scholar 

  40. Papers index of this soa. https://cutt.ly/4fCCWGs

  41. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: a deep convolutional activation feature for generic visual recognition. In: International conference on machine learning, pp 647–655

  42. Elhoseiny M, Cohen S, Chang W, Price B, Elgammal A, Automatic annotation of structured facts in images. arXiv:1604.00466

  43. Wilber MJ, Fang C, Jin H, Hertzmann A, Collomosse J, Belongie S (2017) Bam! the behance artistic media dataset for recognition beyond photography. In: Proceedings of the IEEE international conference on computer vision, pp 1202–1211

  44. Masui K, Ochiai A, Yoshizawa S, Nakayama H (2017) Recurrent visual relationship recognition with triplet unit. In: 2017 IEEE international symposium on multimedia (ISM). IEEE, pp 69–76

  45. Nguyen K. Relational networks for visual relationship detection in images

  46. Zhang J, Shih K, Tao A, Catanzaro B, Elgammal A. Introduction to the 1st place winning model of openimages relationship detection challenge. arXiv:1811.00662

  47. Peyre J, Laptev I, Schmid C, Sivic J, Detecting rare visual relations using analogies. arXiv:1812.05736

  48. Detección del logotipo del vehículo utilizando una red neuronal convolucional y una pirámide de histograma de gradientes orientados, in: \(11^{{\rm a}}\) Conferencia Internacional Conjunta de 2014 sobre Ciencias de la Computación e Ingeniería de Software (JCSSE)

  49. Dulecha TG, Giachetti A, Pintus R, Ciortan I, Jaspe A, Gobbetti E (2019) Crack detection in single- and multi-light images of painted surfaces using convolutional neural networks. In: Eurographics Workshop on Graphics and Cultural Heritage. https://doi.org/10.2312/gch.20191347

  50. Hall P, Cai H, Wu Q, Corradi T (2015) Cross-depiction problem: recognition and synthesis of photographs and artwork. Comput Visual Media 1(2):91–103

    Google Scholar 

  51. Seguin B, Striolo C, Kaplan F et al (2016) Visual link retrieval in a database of paintings. In: European conference on computer vision. Springer, Berlin, pp 753–767

  52. Inoue N, Furuta R, Yamasaki T, Aizawa K (2018) Cross-domain weakly-supervised object detection through progressive domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5001–5009

  53. Gonthier N, Gousseau Y, Ladjal S, Bonfait O (2018) Weakly supervised object detection in artworks. In: Proceedings of the European conference on computer vision (ECCV)

  54. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems. pp 91–99

  55. Westlake N, Cai H, Hall P (2016) Detecting people in artwork with cnns. In: European conference on computer vision. Springer, pp 825–841

  56. Seguin B, Costiner L, di Lenardo I, Kaplan F (2018) New techniques for the digitization of art historical photographic archives-the case of the cini foundation in venice. In: Archiving conference, vol 2018. Society for Imaging Science and Technology, pp 1–5

  57. Gonthier N, Ladjal S, Gousseau Y. Multiple instance learning on deep features for weakly supervised object detection with extreme domain shifts. arXiv:2008.01178

  58. Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: European conference on computer vision. Springer, Berlin, pp 740–755

  59. Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65(6):386

    Google Scholar 

  60. Thomas C, Kovashka A (2018) Artistic object recognition by unsupervised style adaptation. In: Asian conference on computer vision. Springer, pp 460–476

  61. Crowley EJ, Zisserman A (2014) In search of art. In: European conference on computer vision. Springer, pp 54–70

  62. Nguyen N-V, Rigaud C, Burie J-C (2018) Digital comics image indexing based on deep learning. J Imaging 4(7):89

    Google Scholar 

  63. Ogawa T, Otsubo A, Narita R, Matsui Y, Yamasaki T, Aizawa K. Object detection for comics using manga109 annotations. arXiv:1803.08670

  64. Matsui Y, Ito K, Aramaki Y, Fujimoto A, Ogawa T, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimed Tools Appl 76(20):21811–21838

    Google Scholar 

  65. Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 7263–7271

  66. Niitani Y, Ogawa T, Saito S, Saito M (2017) Chainercv: a library for deep learning in computer vision. In: Proceedings of the 25th ACM international conference on multimedia. pp 1217–1220

  67. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: single shot multibox detector. In: European conference on computer vision. Springer, pp 21–37

  68. Simonyan K, Zisserman A, Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  69. Dubray D, Laubrock J, Deep cnn-based speech balloon detection and segmentation for comic books. arXiv:1902.08137

  70. Dunst A, Hartel R, Laubrock J (2017) The graphic narrative corpus (gnc): design, annotation, and analysis for the digital humanities. In: 2017 14th IAPR international conference on document analysis and recognition (ICDAR), vol 3. IEEE, pp 15–20

  71. Murray N, Marchesotti L, Perronnin F (2012) Ava: a large-scale database for aesthetic visual analysis. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 2408–2415

  72. Lu X, Lin Z, Jin H, Yang J, Wang JZ (2015) Rating image aesthetics using deep learning. IEEE Trans Multimed 17(11):2021–2034

    Google Scholar 

  73. Karayev S, Trentacoste M, Han H, Agarwala A, Darrell T, Hertzmann A, Winnemoeller H, Recognizing image style. arXiv:1311.3715

  74. Website: Flickr. https://www.flickr.com/

  75. All data, trained predictors, and code of sergey karayev. http://sergeykarayev.com/recognizing-image-style

  76. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on Multimedia. pp 675–678

  77. Bar Y, Levy N, Wolf L (2014) Classification of artistic styles using binarized features derived from a deep neural network. In: European conference on computer vision. Springer, pp 71–84

  78. Wikiart. visual art encyclopedia. https://www.wikiart.org/en/artists-by-painting-school

  79. Bergamo A, Torresani L, Fitzgibbon AW (2011) Picodes: learning a compact code for novel-category recognition. In: Advances in neural information processing systems. pp 2088–2096

  80. Khan FS, Beigpour S, Van de Weijer J, Felsberg M (2014) Painting-91: a large scale database for computational painting categorization. Mach Vis Appl 25(6):1385–1397

    Google Scholar 

  81. Mensink T, Van Gemert J (2014) The rijksmuseum challenge: museum-centered visual recognition. In: Proceedings of international conference on multimedia retrieval. ACM, p 451

  82. Van Noord N, Hendriks E, Postma E (2015) Toward discovery of the artist’s style: learning to recognize artists by their artworks. IEEE Signal Process Mag 32(4):46–54

    Google Scholar 

  83. Jboor NH, Belhi A, Al-Ali AK, Bouras A, Jaoua A (2019) Towards an inpainting framework for visual cultural heritage. In: 2019 IEEE Jordan international joint conference on electrical engineering and information technology (JEEIT). IEEE, pp 602–607

  84. Castro L, Perez R, Santos A, Carballal A (2014) Authorship and aesthetics experiments: comparison of results between human and computational systems. In: International conference on evolutionary and biologically inspired music and art. Springer, pp 74–84

  85. Götz KO, Götz K (1974) The maitland graves design judgment test judged by 22 experts. Percept Mot Skills 39(1):261–262

    Google Scholar 

  86. Eysenck H, Castle M (1971) Comparative study of artists and nonartists on the maitland graves design judgment test. J Appl Psychol 55(4):389

    Google Scholar 

  87. Machado P, Romero J, Manaris B (2008) Experiments in computational aesthetics. In: The art of artificial evolution. Springer, pp 381–415

  88. Saleh B, Elgammal A (2015) A unified framework for painting classification. In: 2015 IEEE international conference on data mining workshop (ICDMW). IEEE, pp 1254–1261

  89. Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 42(3):145–175

    MATH  Google Scholar 

  90. Torresani L, Szummer M, Fitzgibbon A (2010) Efficient object category recognition using classemes. In: European conference on computer vision. Springer, pp 776–789

  91. Tan WR, Chan CS, Aguirre HE, Tanaka K (2016) Ceci n’est pas une pipe: a deep convolutional network for fine-art paintings classification. In: 2016 IEEE international conference on image processing (ICIP). IEEE, pp 3703–3707

  92. Tan WR, Chan CS, Aguirre HE, Tanaka K, Ceci n’est pas une pipe: a deep convolutional network for fine-art paintings classification

  93. Saleh B, Elgammal A, Large-scale classification of fine-art paintings: learning the right metric on the right feature. arXiv:1505.00855

  94. Banerji S, Sinha A (2016) Painting classification using a pre-trained convolutional neural network. In: International conference on computer vision, graphics, and image processing. Springer, pp 168–179

  95. Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y, Overfeat: integrated recognition, localization and detection using convolutional networks. arXiv:1312.6229

  96. Baumer M, Chen D, Understanding visual art with cnns

  97. Bianco S, Mazzini D, Schettini R (2017) Deep multibranch neural network for painting categorization. In: International conference on image analysis and processing. Springer, pp 414–423

  98. Lecoutre A, Negrevergne B, Yger F, Recognizing art style automatically with deep learning

  99. Cazenave T (2017) Residual networks for computer go. IEEE Trans Games 10(1):107–110

    Google Scholar 

  100. ergart. https://www.ergsart.com

  101. Mao H, Cheung M, She J (2017) Deepart: learning joint representations of visual arts. In: Proceedings of the 25th ACM international conference on Multimedia. ACM, pp 1183–1191

  102. Deepart. http://deepart2.ece.ust.hk/

  103. Art500k. http://deepart2.ece.ust.hk/ART500K/art500k.html

  104. Google arts and culture. https://artsandculture.google.com/

  105. Web gallery of art. https://www.wga.hu/

  106. Strezoski G, Worring M, Omniart: multi-task deep learning for artistic data analysis. arXiv:1708.00684

  107. Met. https://www.metmuseum.org/art/collection

  108. Omniart dataset. http://www.vistory-omniart.com/

  109. Couprie LD (1983) Iconclass: an iconographic classification system. Art Librar J 8(2):32–49

    Google Scholar 

  110. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1–9

  111. Hicsonmez S, Samet N, Sener F, Duygulu P (2017) Draw: deep networks for recognizing styles of artists who illustrate children’s books. In: Proceedings of the 2017 ACM on international conference on multimedia retrieval. pp 338–346

  112. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Google Scholar 

  113. Lowe DG (1999) Object recognition from local scale-invariant features. In: Proceedings of the seventh IEEE international conference on computer vision, vol 2. IEEE, pp 1150–1157

  114. Sener F, Samet N, Sahin PD (2012) Identification of illustrators. In: European conference on computer vision. Springer, pp 589–597

  115. Rodriguez CS, Lech M, Pirogova E (2018) Classification of style in fine-art paintings using transfer learning and weighted image patches. In: 2018 12th international conference on signal processing and communication systems (ICSPCS). IEEE, pp 1–7

  116. Florea C, Condorovici R, Vertan C (2018) Pandora. http://imag.pub.ro/pandora/pandora_download.html

  117. Florea C, Toca C, Gieseke F (2017) Artistic movement recognition by boosted fusion of color structure and topographic description. In: 2017 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 569–577

  118. Mooers CN (1977) Preventing software piracy. Computer 10(3):29–30

    Google Scholar 

  119. Hua K-L, Ho T-T, Jangtjik K-A, Chen Y-J, Yeh M-C (2020) Artist-based painting classification using Markov random fields with convolution neural network. Multimed Tools Appl 79:12635–12658. https://doi.org/10.1007/s11042-019-08547-4

    Article  Google Scholar 

  120. Jangtjik KA, Yeh M-C, Hua K-L (2016) Artist-based classification via deep learning with multi-scale weighted pooling. In: Proceedings of the 24th ACM international conference on Multimedia. pp 635–639

  121. Elgammal A, Kang Y, Den Leeuw M (2018) Picasso, matisse, or a fake? Automated analysis of drawings at the stroke level for attribution and authentication. In: Thirty-second AAAI conference on artificial intelligence

  122. Chen J, Deng A (2018) Comparison of machine learning techniques for artist identification

  123. Sandoval C, Pirogova E, Lech M (2019) Two-stage deep learning approach to the classification of fine-art paintings. IEEE Access 7:41770–41781

    Google Scholar 

  124. Kim Y-M (2018) What makes the difference in visual styles of comics: from classification to style transfer. In: 2018 3rd international conference on computational intelligence and applications (ICCIA). IEEE, pp 181–185

  125. Young-Min K (2019) Feature visualization in comic artist classification using deep neural networks. J Big Data 6(1):56

    Google Scholar 

  126. Furusawa C, Hiroshiba K, Ogaki K, Odagiri Y (2017) Comicolorization: semi-automatic manga colorization. In: SIGGRAPH Asia 2017 Technical Briefs. pp 1–4

  127. Yoshimura Y, Cai B, Wang Z, Ratti C (2019) Deep learning architect: classification for architectural design through the eye of artificial intelligence. In: International conference on computers in urban planning and urban management. Springer, pp 249–265

  128. Liao W, Lan C, Zeng W, Yang MY, Rosenhahn B, Exploring the semantics for visual relationship detection. arXiv:1904.02104

  129. Zhang J, Shih KJ, Elgammal A, Tao A, Catanzaro B (2019) Graphical contrastive losses for scene graph parsing. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 11535–11543

  130. Gu J, Zhao H, Lin Z, Li S, Cai J, Ling M (2019) Scene graph generation with external knowledge and image reconstruction. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1969–1978

  131. Zhang J, Kalantidis Y, Rohrbach M, Paluri M, Elgammal A, Elhoseiny M (2019) Large-scale visual relationship understanding. In: Proceedings of the AAAI conference on artificial intelligence, vol 33. pp 9185–9194

  132. Tian X, Dong Z, Yang K, Mei T (2015) Query-dependent aesthetic model with deep learning for photo quality assessment. IEEE Trans Multimed 17(11):2035–2048

    Google Scholar 

  133. Luo W, Wang X, Tang X (2011) Content-based photo quality assessment. In: 2011 International conference on computer vision. IEEE, pp 2206–2213

  134. Wagner M, Lin H, Li S, Saupe D, Algorithm selection for image quality assessment. arXiv:1908.06911

  135. Machado P, Romero J, Nadal M, Santos A, Correia J, Carballal A (2015) Computerized measures of visual complexity. Acta Psychol 160:43–57

    Google Scholar 

  136. Haykin S (1994) Neural networks: a comprehensive foundation. Prentice Hall PTR, London

    MATH  Google Scholar 

  137. Denzler J, Rodner E, Simon M (2016) Convolutional neural networks as a computational model for the underlying processes of aesthetics perception. In: European conference on computer vision. Springer, pp 871–887

  138. Amirshahi SA, Denzler J, Redies C, Jenaesthetics—a public dataset of paintings for aesthetic research. Computer Vision Group [Google Scholar], Jena

  139. Redies C, Amirshahi SA, Koch M, Denzler J (2012) Phog-derived aesthetic measures applied to color photographs of artworks, natural scenes and objects. In: European conference on computer vision. Springer, pp 522–531

  140. Amirshahi SA, Redies C, Denzler J (2013) How self-similar are artworks at different levels of spatial resolution?. In: Proceedings of the symposium on computational aesthetics, pp 93–100

  141. Carballal A, Santos A, Romero J, Machado P, Correia J, Castro L (2018) Distinguishing paintings from photographs by complexity estimates. Neural Comput Appl 30(6):1957–1969

    Google Scholar 

  142. Prasad M, Jwala Lakshmamma B, Chandana AH, Komali K, Manoja M, Rajesh Kumar P, Sasi Kiran P (2018) An efficient classification of flower images with convolutional neural networks. Int J Eng Technol 7(11):384–391

    Google Scholar 

  143. Collomosse J, Bui T, Wilber MJ, Fang C, Jin H (2017) Sketching with style: visual search with sketches and aesthetic context. In: Proceedings of the IEEE international conference on computer vision, pp 2660–2668

  144. Eitz M, Hays J, Alexa M (2012) How do humans sketch objects? ACM Trans Graphics TOG 31(4):1–10

    Google Scholar 

  145. Lu NGM, Deformsketchnet: Deformable convolutional networks for sketch classification

  146. Shen X, Efros AA, Mathieu A, Discovering visual patterns in art collections with spatially-consistent feature learning. arXiv:1903.02678

  147. Brueghel family: Jan brueghel the elder. University of california, Berkeley. https://www.janbrueghel.net/

  148. Dutta A, Zisserman A, The vgg image annotator (via). arXiv:1904.10699

  149. En S, Nicolas S, Petitjean C, Jurie F, Heutte L (2016) New public dataset for spotting patterns in medieval document images. J Electron Imaging 26(1):011010

    Google Scholar 

  150. Fernando B, Tommasi T, Tuytelaars T (2015) Location recognition over large time lags. Comput Vis Image Underst 139:21–28

    Google Scholar 

  151. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2007) Object retrieval with large vocabularies and fast spatial matching. In IEEE conference on computer vision and pattern recognition. IEEE, pp 1–8

  152. Castellano G, Vessio G (2020) Towards a tool for visual link retrieval and knowledge discovery in painting datasets. In: Italian research conference on digital libraries. Springer, pp 105–110

  153. Kaggle. https://www.kaggle.com/ikarus777/best-artworks-of-all-time

  154. Datta R, Joshi D, Li J, Wang JZ (2006) Studying aesthetics in photographic images using a computational approach. In: European conference on computer vision. Springer, pp 288–301

  155. Ke Y, Tang X, Jing F (2006) The design of high-level features for photo quality assessment. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), vol 1. IEEE, pp 419–426

  156. Wong L-K, Low K-L (2009) Saliency-enhanced image aesthetics class prediction. In: 2009 16th IEEE international conference on image processing (ICIP). pp 997–1000

  157. Marchesotti L, Perronnin F, Larlus D, Csurka G (2011) Assessing the aesthetic quality of photographs using generic image descriptors. In: 2011 international conference on computer vision. IEEE, pp 1784–1791

  158. Wang W, Cai D, Wang L, Huang Q, Xu X, Li X (2016) Synthesized computational aesthetic evaluation of photos. Neurocomputing 172:244–252

    Google Scholar 

  159. Xia Y, Liu Z, Yan Y, Chen Y, Zhang L, Zimmermann R (2017) Media quality assessment by perceptual gaze-shift patterns discovery. IEEE Trans Multimedia 19(8):1811–1820

    Google Scholar 

  160. Tong H, Li M, Zhang H-J, He J, Zhang C (2004) Classification of digital photos taken by photographers or home users. In: Pacific-Rim conference on multimedia. Springer, pp 198–205

  161. Friedman J, Hastie T, Tibshirani R et al (2000) Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors). Ann Statist 28(2):337–407

    MathSciNet  MATH  Google Scholar 

  162. Luo Y, Tang X (2008) Photo and video quality evaluation: focusing on the subject. In: European conference on computer vision. Springer, pp 386–399

  163. Wu O, Hu W, Gao J (2011) Learning to predict the perceived visual quality of photos. In: 2011 international conference on computer vision. IEEE, pp 225–232

  164. Tan Y, Zhou Y, Li G, Huang A (2016) Computational aesthetics of photos quality assessment based on improved artificial neural network combined with an autoencoder technique. Neurocomputing 188:50–62

    Google Scholar 

  165. Gao F, Wang Y, Li P, Tan M, Yu J, Zhu Y (2017) Deepsim: deep similarity for image quality assessment. Neurocomputing 257:104–114

    Google Scholar 

  166. Meng X, Gao F, Shi S, Zhu S, Zhu J (2018) Mlans: image aesthetic assessment via multi-layer aggregation networks. In: 2018 Eighth international conference on image processing theory, tools and applications (IPTA). IEEE, pp 1–6

  167. Talebi H, Milanfar P (2018) Nima: neural image assessment. IEEE Trans Image Process 27(8):3998–4011

    MathSciNet  MATH  Google Scholar 

  168. Zhang W, Ma K, Yan J, Deng D, Wang Z (2018) Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Trans Circuits Syst Video Technol 30(1):36–47. https://doi.org/10.1109/TCSVT.2018.2886771

    Article  Google Scholar 

  169. Verkoelen SD, Lamers MH, van der Putten P (2017) Exploring the exactitudes portrait series with restricted boltzmann machines. In: International conference on evolutionary and biologically inspired music and art. Springer, pp 321–337

  170. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    MathSciNet  MATH  Google Scholar 

  171. Exactitude website. https://exactitudes.com/collectie/?v=s

  172. Larson EC, Chandler DM (2010) Most apparent distortion: full-reference image quality assessment and the role of strategy. J Electron Imaging 19(1):011006

    Google Scholar 

  173. Ghadiyaram D, Bovik AC (2015) Massive online crowd sourced study of subjective and objective picture quality. IEEE Trans Image Process 25(1):372–387

    MATH  Google Scholar 

  174. Jayaraman D, Mittal A, Moorthy AK, Bovik AC (2012) Objective quality assessment of multiply distorted images. In: 2012 Conference record of the forty sixth asilomar conference on signals, systems and computers (ASILOMAR). IEEE, pp 1693–1697

  175. Ponomarenko N, Ieremeiev O, Lukin V, Egiazarian K, Jin L, Astola J, Vozel B, Chehdi K, Carli M, Battisti F et al (2013) Color image database tid2013: peculiarities and preliminary results. In: European workshop on visual information processing (EUVIP). IEEE, pp 106–111

  176. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H, Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861

  177. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2818–2826

  178. Sheikh HR, Sabir MF, Bovik AC (2006) A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans Image Process 15(11):3440–3451

    Google Scholar 

  179. Ponomarenko N, Jin L, Ieremeiev O, Lukin V, Egiazarian K, Astola J, Vozel B, Chehdi K, Carli M, Battisti F et al (2015) Image database tid2013: peculiarities, results and perspectives. Sig Process Image Commun 30:57–77

    Google Scholar 

  180. Ma K, Duanmu Z, Wu Q, Wang Z, Yong H, Li H, Zhang L (2016) Waterloo exploration database: new challenges for image quality assessment models. IEEE Trans Image Process 26(2):1004–1016

    MathSciNet  MATH  Google Scholar 

  181. Carballal A, Perez R, Santos A, Castro L (2014) A complexity approach for identifying aesthetic composite landscapes. In: International conference on evolutionary and biologically inspired music and art. Springer, pp 50–61

  182. Lu X, Lin Z, Jin H, Yang J, Wang JZ (2014) Rapid: rating pictorial aesthetics using deep learning. In: Proceedings of the 22nd ACM international conference on Multimedia. pp 457–466

  183. Zhou Y, Li G, Tan Y (2015) Computational aesthetics of photos quality assessment and classification based on artificial neural network with deep learning methods. Int J Signal Process Image Process Pattern Recognit 8(7):273–282

    Google Scholar 

  184. Dong Z, Tian X (2015) Multi-level photo quality assessment with multi-view features. Neurocomputing 168:308–319

    Google Scholar 

  185. Campbell A, Ciesielksi V, Qin AK (2015) Feature discovery by deep learning for aesthetic analysis of evolved abstract images. In: International conference on evolutionary and biologically inspired music and art. Springer, pp 27–38

  186. Xu Q, D’Souza D, Ciesielski V (2007) Evolving images for entertainment. In: Proceedings of the 4th Australasian conference on Interactive entertainment. RMIT University, p 26

  187. Wang W, Zhao M, Wang L, Huang J, Cai C, Xu X (2016) A multi-scene deep learning model for image aesthetic evaluation. Signal Process Image Commun 47:511–518

    Google Scholar 

  188. Jin X, Chi J, Peng S, Tian Y, Ye C, Li X (2016) Deep image aesthetics classification using inception modules and fine-tuning connected layer. In: 2016 8th international conference on wireless communications and signal processing (WCSP). IEEE, pp 1–6

  189. Kao Y, Huang K, Maybank S (2016) Hierarchical aesthetic quality assessment using deep convolutional neural networks. Signal Process Image Commun 47:500–510

    Google Scholar 

  190. Kao Y, He R, Huang K, Visual aesthetic quality assessment with multi-task deep learning. arXiv:1604.04970 5

  191. Malu G, Bapi RS, Indurkhya B, Learning photography aesthetics with deep cnns. arXiv:1707.03981

  192. Kong S, Shen X, Lin Z, Mech R, Fowlkes C (2016) Photo aesthetics ranking network with attributes and content adaptation. In: European conference on computer vision. Springer, pp 662–679

  193. Tan Y, Tang P, Zhou Y, Luo W, Kang Y, Li G (2017) Photograph aesthetical evaluation and classification with deep convolutional neural networks. Neurocomputing 228:165–175

    Google Scholar 

  194. Li Y-X, Pu Y-Y, Xu D, Qian W-H, Wang L-P (2017) Image aesthetic quality evaluation using convolution neural network embedded learning. Optoelectron Lett 13(6):471–475

    Google Scholar 

  195. Lemarchand F et al (2017) From computational aesthetic prediction for images to films and online videos. AVANT. Pismo Awangardy Filozoficzno-Naukowej (S):69–78

  196. Tzelepis C, Mavridaki E, Mezaris V, Patras I (2016) Video aesthetic quality assessment using kernel support vector machine with isotropic gaussian sample uncertainty (ksvm-igsu). In: 2016 IEEE international conference on image processing (ICIP). IEEE, pp 2410–2414

  197. Murray N, Gordo A, A deep architecture for unified aesthetic prediction. arXiv:1708.04890

  198. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 770–778

  199. Bianco S, Celona L, Napoletano P, Schettini R (2016) Predicting image aesthetics with deep learning. In: International conference on advanced concepts for intelligent vision systems. Springer, pp 117–125

  200. Zhou B, Lapedriza A, Xiao J, Torralba A, Oliva A (2014) Learning deep features for scene recognition using places database. In: Advances in neural information processing systems. pp 487–495

  201. Lemarchand F (2018) Fundamental visual features for aesthetic classification of photographs across datasets. Pattern Recogn Lett 112:9–17

    Google Scholar 

  202. Zhang C, Zhu C, Xu X, Liu Y, Xiao J, Tillo T (2018) Visual aesthetic understanding: sample-specific aesthetic classification and deep activation map visualization. Signal Process Image Commun 67:12–21

    Google Scholar 

  203. Zhang C, Zhu C, Xu X, Liu Y, Xiao J, Tillo T, Modelos de cnn de zhan c et al. https://github.com/galoiszhang/AWCU

  204. Jin X, Wu L, Zhao G, Zhou X, Zhang X, Li X (2020) IDEA: a new dataset for image aesthetic scoring. Multimed Tools Appl 79(21):14341–14355

    Google Scholar 

  205. Jin X, Wu L, Zhao G, Zhou X, Zhang X, Li X (2018) Photo aesthetic scoring through spatial aggregation perception dcnn on a new idea dataset. In: International symposium on artificial intelligence and robotics. Springer, pp 41–50

  206. Apostolidis K, Mezaris V (2019) Image aesthetics assessment using fully convolutional neural networks. In: International conference on multimedia modeling. Springer, pp 361–373

  207. Keras neural network api. https://keras.io/

  208. Implementation en keras neural network api. https://github.com/bmezaris/fullyconvolutionalnetworks

  209. Sheng K, Dong W, Chai M, Wang G, Zhou P, Huang F, Hu B-G, Ji R, Ma C, Revisiting image aesthetic assessment via self-supervised feature learning. arXiv:1911.11419

  210. Carballal A, Fernandez-Lozano C, Heras J, Romero J (2019) Transfer learning features for predicting aesthetics through a novel hybrid machine learning method. Neural Comput Appl 32:5889–5900. https://doi.org/10.1007/s00521-019-04065-4

    Article  Google Scholar 

  211. Cetinic E, Lipic T, Grgic S (2019) A deep learning perspective on beauty, sentiment, and remembrance of art. IEEE Access 7:73694–73710

    Google Scholar 

  212. Dai Y, Cnn-based repetitive self-revised learning for photos’ aesthetics imbalanced classification. arXiv:2003.03081

  213. Dai Y, Sample-specific repetitive learning for photo aesthetic assessment and highlight region extraction. arXiv:1909.08213

  214. 500px. https://web.500px.com/

  215. Semmo A, Isenberg T, Döllner J (2017) Neural style transfer: a paradigm shift for image-based artistic rendering?. In: Proceedings of the symposium on non-photorealistic animation and rendering. pp 1–13

  216. Gatys L, Ecker AS, Bethge M (2015) Texture synthesis using convolutional neural networks. In: Advances in neural information processing systems. pp 262–270

  217. Portilla J, Simoncelli EP (2000) A parametric texture model based on joint statistics of complex wavelet coefficients. Int J Comput Vis 40(1):49–70

    MATH  Google Scholar 

  218. Gatys LA, Ecker AS, Bethge M, A neural algorithm of artistic style. arXiv:1508.06576

  219. Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2414–2423

  220. Gatys LA, Bethge M, Hertzmann A, Shechtman E, Preserving color in neural artistic style transfer. arXiv:1606.05897

  221. Chen Y-L, Hsu C-T (2016) Towards deep style transfer: a content-aware perspective. In: BMVC

  222. Champandard AJ, Semantic style transfer and turning two-bit doodles into fine artworks. arXiv:1603.01768

  223. Li C, Wand M (2016) Combining markov random fields and convolutional neural networks for image synthesis. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2479–2486

  224. Joshi B, Stewart K, Shapiro D (2017) Bringing impressionism to life with neural style transfer in come swim. In: Proceedings of the ACM SIGGRAPH digital production symposium. pp 1–5

  225. Chen Y, Lai Y-K, Liu Y-J (2017) Transforming photos to comics using convolutional neural networks. In: 2017 IEEE international conference on image processing (ICIP). IEEE, pp 2010–2014

  226. Krishnan U, Sharma A, Chattopadhyay P, Feature fusion from multiple paintings for generalized artistic style transfer. Available at SSRN 3387817

  227. Jing Y, Yang Y, Feng Z, Ye J, Yu Y, Song M (2020) Neural style transfer: a review. IEEE Trans Visual Comput Graph 26(11):3365–3385. https://doi.org/10.1109/TVCG.2019.2921336

    Article  Google Scholar 

  228. Portfolio. greg surma. https://gsurma.github.io/

  229. Correia J, Martins T, Martins P, Machado P (2016) X-faces: the exploit is out there. In: Proceedings of the seventh international conference on computational creativity

  230. Machado P, Correia J, Romero J (2012) Improving face detection. In: European conference on genetic programming. Springer, pp 73–84

  231. Machado P, Correia J, Romero J (2012) Expression-based evolution of faces. In: International conference on evolutionary and biologically inspired music and art. Springer, pp 187–198

  232. Correia J, Martins T, Machado P (2019) Evolutionary data augmentation in deep face detection. In: Proceedings of the genetic and evolutionary computation conference companion. pp 163–164

  233. Machado P, Vinhas A, Correia J, Ekárt A (2015) Evolving ambiguous images. In: Twenty-fourth international joint conference on artificial intelligence

  234. Tian C, Xu Y, Li Z, Zuo W, Fei L, Liu H (2020) Attention-guided CNN for image denoising. Neural Netw 124:117–129. https://doi.org/10.1016/j.neunet.2019.12.024

    Article  Google Scholar 

  235. Adnet, URL: http://github.com/hellloxiaotian/ADNet

  236. Colton S, Halskov J, Ventura D, Gouldstone I, Cook M, Ferrer BP (2015) The painting fool sees! new projects with the automated painter. In: ICCC. pp 189–196

  237. Krzeczkowska A, El-Hage J, Colton S, Clark S (2010) Automated collage generation-with intent. In: ICCC. pp 36–40

  238. Colton S (2008) Automatic invention of fitness functions with application to scene generation. In: Workshops on applications of evolutionary computation. Springer, pp 381–391

  239. Colton S (2008) Experiments in constraint-based automated scene generation. In: Proceedings of the 5th international joint workshop on computational creativity. pp 127–136

  240. Colton S, Ferrer BP (2012) No photos harmed/growing paths from seed: an exhibition. In: Proceedings of the symposium on non-photorealistic animation and rendering. pp 1–10

  241. Colton S (2012) Evolving a library of artistic scene descriptors. In: International conference on evolutionary and biologically inspired music and art. Springer, pp 35–47

  242. The painting fool. about me. http://www.thepaintingfool.com/about/index.html

  243. Colton S, Ventura D (2014) You can’t know my mind: a festival of computational creativity. In: ICCC. pp 351–354

  244. Dataset darci. http://darci.cs.byu.edu

  245. Radford A, Metz L, Chintala S, Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434

  246. Yu F, Seff A, Zhang Y, Song S, Funkhouser T, Xiao J, Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv:1506.03365

  247. Dbpedia. https://wiki.dbpedia.org/

  248. Krizhevsky A, Hinton G et al (2009) Learning multiple layers of features from tiny images

  249. Tan WR, Chan CS, Aguirre HE, Tanaka K (2017) Artgan: artwork synthesis with conditional categorical gans. In: 2017 IEEE International conference on image processing (ICIP). IEEE, pp 3760–3764

  250. Elgammal A, Liu B, Elhoseiny M, Mazzone M, Can: creative adversarial networks, generating “art” by learning about styles and deviating from style norms. arXiv:1706.07068

  251. Neumann A, Pyromallis C, Alexander B (2018) Evolution of images with diversity and constraints using a generative adversarial network. In: International conference on neural information processing. Springer, pp 452–465

  252. Neumann A, Pyromallis C, Alexander B, Evolution of images with diversity and constraints using a generator network. arXiv:1802.05480

  253. Talebi H, Milanfar P (2018) Learned perceptual image enhancement. In: 2018 IEEE international conference on computational photography (ICCP). IEEE, pp 1–13

  254. Bychkovsky V, Paris S, Chan E, Durand F (2011) Learning photographic global tonal adjustment with a database of input/output image pairs. In: CVPR 2011. IEEE, pp 97–104

  255. Bontrager P, Lin W, Togelius J, Risi S (2018) Deep interactive evolution. In: International conference on computational intelligence in music, sound, art and design. Springer, pp 267–282

  256. Liu Z, Luo P, Wang X, Tang X (2015) Deep learning face attributes in the wild. In: Proceedings of the IEEE international conference on computer vision. pp 3730–3738

  257. Yu A, Grauman K (2014) Fine-grained visual comparisons with local learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 192–199

  258. Aubry M, Maturana D, Efros AA, Russell BC, Sivic J (2014) Seeing 3d chairs: exemplar part-based 2d-3d alignment using a large dataset of cad models. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3762–3769

  259. Van Noord N, Postma E, Light-weight pixel context encoders for image inpainting. arXiv:1801.05585

  260. Pathak D, Krahenbuhl P, Donahue J, Darrell T, Efros AA (2016) Context encoders: feature learning by inpainting. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2536–2544

  261. Tanjil F, Ross BJ (2019) Deep learning concepts for evolutionary art. In: International conference on computational intelligence in music, sound, art and design (Part of EvoStar). Springer, pp 1–17

  262. Deng J, Berg A, Satheesh S, Su H, Khosla A, Li F (2012) Large scale visual recognition challenge 2012. In: ILSVRC 2012 workshop

  263. Elgammal A (2019) Ai is blurring the definition of artist: advanced algorithms are using machine learning to create art autonomously. Am Sci 107(1):18–22

    Google Scholar 

  264. Aican.io. https://www.aican.io

  265. Blair A (2019) Adversarial evolution and deep learning—How does an artist play with our visual system?. In: International conference on computational intelligence in music, sound, art and design (part of EvoStar). Springer, pp 18–34

  266. Shen X, Darmon F, Efros AA, Aubry M, Ransac-flow: generic two-stage image alignment. arXiv:2004.01526

  267. Shen X, Darmon F, Efros AA, Aubry M (2020) Ransac-flow: generic two-stage image alignment. In: 16th European conference on computer vision

  268. Barath D, Matas J (2018) Graph-cut ransac. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 6733–6741

  269. Barath D, Matas J, Noskova J (2019) Magsac: marginalizing sample consensus. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 10197–10205

  270. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    MathSciNet  Google Scholar 

  271. Plötz T, Roth S (2018) Neural nearest neighbors networks. In: Advances in neural information processing systems. pp 1087–1098

  272. Qi CR, Su H, Mo K, Guibas LJ (2017) Pointnet: deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 652–660

  273. Raguram R, Chum O, Pollefeys M, Matas J, Frahm J-M (2012) Usac: a universal framework for random sample consensus. IEEE Trans Pattern Anal Mach Intell 35(8):2022–2038

    Google Scholar 

  274. Ranftl R, Koltun V (2018) Deep fundamental matrix estimation. In: Proceedings of the European conference on computer vision (ECCV). pp 284–299

  275. Zhang J, Sun D, Luo Z, Yao A, Zhou L, Shen T, Chen Y, Quan L, Liao H (2019) Learning two-view correspondences and geometry using order-aware network. In: Proceedings of the IEEE international conference on computer vision. pp 5845–5854

  276. Jason JY, Harley AW, Derpanis KG (2016) Back to basics: unsupervised learning of optical flow via brightness constancy and motion smoothness. In: European conference on computer vision. Springer, pp 3–10

  277. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Google Scholar 

  278. Yin Z, Shi J (2018) Geonet: unsupervised learning of dense depth, optical flow and camera pose. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1983–1992

  279. Temizel A et al (2018) Paired 3d model generation with conditional generative adversarial networks. In: Proceedings of the European conference on computer vision (ECCV)

  280. Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3d shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1912–1920

  281. Li H, Zheng Y, Wu X, Cai Q (2019) 3d Model generation and reconstruction using conditional generative adversarial network. Int J Comput Intell Syst 12(2):697–705

    Google Scholar 

  282. Chang AX, Funkhouser T, Guibas L, Hanrahan P, Huang Q, Li Z, Savarese S, Savva M, Song S, Su H et al Shapenet: an information-rich 3d model repository. arXiv:1512.03012

  283. Lim JJ, Pirsiavash H, Torralba A (2013) Parsing ikea objects: fine pose estimation. In: Proceedings of the IEEE international conference on computer vision. pp 2992–2999

  284. Volz V, Schrum J, Liu J, Lucas SM, Smith A, Risi S (2018) Evolving mario levels in the latent space of a deep convolutional generative adversarial network. In: Proceedings of the genetic and evolutionary computation conference. pp 221–228

  285. Summerville AJ, Snodgrass S, Mateas M, Ontanón S, The vglc: the video game level corpus. arXiv:1606.07487

  286. Togelius J, Karakovskiy S, Baumgarten R (2010) The 2009 mario ai competition. In: IEEE congress on evolutionary computation. IEEE, pp 1–8

  287. Hollingsworth B, Schrum J (2019) Infinite art gallery: a game world of interactively evolved artwork. In: 2019 IEEE congress on evolutionary computation (CEC). IEEE, pp 474–481

  288. Romero J, Automatic real estate image evaluation by artificial intelligence. Present. https://cutt.ly/DfVT6VI

  289. Bishop CM (2006) Pattern recognition and machine learning. springer, Berlin

    MATH  Google Scholar 

  290. McLachlan GJ, Do K-A, Ambroise C (2005) Analyzing microarray gene expression data, vol 422. Wiley, New York

    MATH  Google Scholar 

  291. Kramer MA (1991) Nonlinear principal component analysis using autoassociative neural networks. AIChE J 37(2):233–243

    Google Scholar 

  292. Géron A (2019) Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: concepts, tools, and techniques to build intelligent systems. O’Reilly Media, Newton

    Google Scholar 

  293. Lin T-Y, RoyChowdhury A, Maji S (2015) Bilinear cnn models for fine-grained visual recognition. In: Proceedings of the IEEE international conference on computer vision. pp 1449–1457

  294. Kingma DP, Ba J, Adam: a method for stochastic optimization. arXiv:1412.6980

  295. Bengio Y (2009) Learning deep architectures for AI. Now Publishers Inc, New York

    MATH  Google Scholar 

  296. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems. pp 2672–2680

  297. Hochreiter S, Ja1 4 rgen schmidhuber (1997) “long short-term memory”. Neural Comput 9(8)

  298. Barriga NA (2019) A short introduction to procedural content generation algorithms for videogames. Int J Artif Intell Tools 28(02):1930001

    Google Scholar 

  299. Togelius J, Kastbjerg E, Schedl D, Yannakakis GN (2011) What is procedural content generation? Mario on the borderline. In: Proceedings of the 2nd international workshop on procedural content generation in games. pp 1–6

  300. Sokolova M, Lapalme G (2009) A systematic analysis of performance measures for classification tasks. Inf Process Manag 45(4):427–437

    Google Scholar 

  301. Smolensky P (1986) Information processing in dynamical systems: foundations of harmony theory, Technical report. Colorado University at Boulder Department of Computer Science

  302. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  303. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2008) Lost in quantization: improving particular object retrieval in large scale image databases. In: 2008 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–8

  304. Ren J, Shen X, Lin Z, Mech R, Foran DJ (2017) Personalized image aesthetics. In: Proceedings of the IEEE international conference on computer vision. pp 638–647

  305. You Q, Luo J, Jin H, Yang J (2015) Robust image sentiment analysis using progressively trained and domain transferred deep networks. In: Twenty-ninth AAAI conference on artificial intelligence

  306. Katsurai M, Satoh S (2016) Image sentiment analysis using latent correlations among visual, textual, and sentiment views. In: 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 2837–2841

  307. Khosla A, Raju AS, Torralba A, Oliva A (2015) Understanding and predicting image memorability at a large scale. In: Proceedings of the IEEE international conference on computer vision, pp 2390–2398

  308. Mohammad S, Kiritchenko S (2018) Wikiart emotions: an annotated dataset of emotions evoked by art. In: Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018)

  309. Yanulevskaya V, Uijlings J, Bruni E, Sartori A, Zamboni E, Bacci F, Melcher D, Sebe N (2012) In the eye of the beholder: employing statistical analysis and eye tracking for analyzing abstract paintings. In: Proceedings of the 20th ACM international conference on multimedia. pp 349–358

Download references

Acknowledgements

CITIC, as a Research Centre of the Galician University System, is financed by the Regional Ministry of Education, University and Vocational Training of the Xunta de Galicia through the European Regional Development Fund (ERDF) with 80% of funding provided by Operational Programme ERDF Galicia 2014–2020 and the remaining 20% by the General Secretariat of Universities (Ref. ED431G 2019/01).

Funding

This work has also been supported by the General Directorate of Culture, Education and University Management of Xunta de Galicia (Ref. ED431G01, ED431D 201716), and Competitive Reference Groups (Ref. ED431C 201849).

Author information

Authors and Affiliations

  1. Department of Computer Science, Faculty of Computer Science, CITIC, University of A Coruña, 15071, A Coruña, Spain

    Iria Santos, Luz Castro, Nereida Rodriguez-Fernandez, Álvaro Torrente-Patiño & Adrián Carballal

Authors
  1. Iria Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luz Castro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nereida Rodriguez-Fernandez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Álvaro Torrente-Patiño
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adrián Carballal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Iria Santos.

Ethics declarations

Conflict of interest

The authors declares that there are no conflicts of interest regarding the publication of this paper.

Code availability

No code was used to support this study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Glossary

Glossary

This section explains a number of concepts that are mentioned throughout the document that readers may not be familiar with. The article is extensive and has numerous annotations, so those that are considered most difficult to understand or most commonly used have been selected.

  • 1-vs-rest or One-vs-Rest strategy [289]: splits a multi-class classification into one binary classification problem per class.

  • 10-Fold Cross-Validation strategy [290]: the original sample is divided into 10 samples of the same size and one of these subsamples is kept as test data, with the rest used as training data. The same process is carried out with all samples and the results can be subsequently averaged.

  • Accuracy: How close the measured result is to the actual value. It is common to use this value as a percentage.

  • AUROC (area under the receiver operating characteristic): a measure of discrimination, which discriminates between positive and negative examples. For example, a randomly selected x image will have a value set to look like y. If x is close to y the value will be high, and if it is not similar, the value will be low.

  • Autoencoder/automatic encoder or Auto-encoding Neural Network [291]: type of Artificial Neuron Network, used for unsupervised learning of efficient data encoding.

  • CNN or ConvNet (Convolutional Neural Networks): class of Deep Neural Network that uses a mathematical operation called convolution.

  • Convolutional layers [292]: the convolutional layer is the main nucleus of a CNN.

  • DB-CNN (Deep Bilinear-CNN) or Deep Bilinear model [168, 293]: grouping in a single representation of two bilinial models with pre-trained characteristics.

  • Deep Convolutional Network or DCNN (Deep Convolutional Neural Network): consists of many neural network layers and uses convolution.

  • Deep Neural Network [294, 295]: ANN with multiple layers between input and output.

  • Degradation identification loss: probability of loss due to degradation.

  • Dense SIFT: descriptor that divides the image into overlapping cells before using Histogram of Oriented Gradients (HOG) to describe the interest points. Important not to confuse with SIFT that detects interest points using Difference of Gaussian Filtering (DoG) and before using HOG to describe these interest points. Color Dense SIFT [114] is similar to Dense SIFT except that it also contains color information.

  • Entropy estimation: estimation of differential entropy with an observing system. Commonly with histograms.

  • F1-score: measure of a test’s accuracy with the precision and the recall. Precision is the number of correctly identified positive results divided by the number of all positive results. Recall is the number of correctly identified positive results divided by the number of all samples that should have been identified as positive, the relevance.

  • GAN (Generative Adversarial Network) [296]: artificial intelligence algorithms used in unsupervised learning. GAN has two neural networks, a generator and an evaluator. DCGANs (Deep Convolutional Generative Adversarial Networks) [245] are a direct extension of the GAN that use convolutional layers in the discriminator and convolutional-transpose layers in the generator.

  • GP (Genetic Programming): extension of the Genetic Algorithm (GA), in which the structures that are adapted are hierarchical computer programs, which vary in size and structure.

  • GAP (Global Average Precision): average precision based on the top 20 predictions.

  • Histogram of oriented gradient: feature descriptor when the distribution (histograms) of directions of gradients (oriented gradients) are used as features. This technique is used to detect objects.

  • Hu moments: weighted average of pixel intensities within an image.

  • kNN (k-Nearest): supervised instance algorithm. Not to be confused with k-means, that is unsupervised.

  • KRCC (Kendall’s rank correlation coefficient): statistic used to measure the ordinal association between two measured quantities.

  • LSTM (Long Short-Term Memory) [297]: artificial recurrent neural network (RNN) architecture composed of a cell, an input gate, an output gate and a forget gate (commonly).

  • mAP (mean Average Precision): mean of the average precision scores for each query.

  • MMMS (Mean Minimum Matrix Strategy) [261]: reduces dimensions and identifies the most relevant high-level activation maps using reduced activation matrices for a skill.

  • MRSSE (Mean Residual Sum of Squares Error): mean of the residual sum of squares (RSS), a statistical technique used to measure the amount of variance in a dataset that is not possible to explain by a regression model.

  • PCG (Procedural Content Generation) [298, 299]: automation of media production, for example, PCG for games is the use of algorithms to produce game content that would otherwise be created by a designer.

  • Real-AdaBoost [161]: is the use of decision trees for Adaptive Boosting (AdaBoost), a machine learning meta-algorithm. Each node on the sheet is modified to produce half of the transformations.

  • Recall [300]: measure of quantity calculated as the number of true positives divided by the total number of true positives and false negatives.

  • RBM (Restricted Boltzmann Machines) [301]: generative stochastic Artificial Neural Network that can learn a probability distribution over its set of inputs.

  • SIFT keypoints [113]: keypoints uses to detect and describe local features in images with scale-invariant feature transform (SIFT), a feature detection algorithm in computer vision.

  • SVM (Support Vector Machine) [302]: supervised learning models that are formed by hyperplane or set of hyperplanes in high or infinite dimensional space, which can be used for tasks such as classification or regression.

  • Total loss function: expected loss (in average) of a group of items.

Table 9 Table of compilation of the data sets used in the different experiments with ANN addressed throughout the article, which are available online
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Santos, I., Castro, L., Rodriguez-Fernandez, N. et al. Artificial Neural Networks and Deep Learning in the Visual Arts: a review. Neural Comput & Applic 33, 121–157 (2021). https://doi.org/10.1007/s00521-020-05565-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05565-4

Keywords

Search

Navigation