Reflecting on How Artworks Are Processed and Analyzed by Computer Vision

  • Sabine LangEmail author
  • Björn Ommer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11130)


The intersection between computer vision and art history has resulted in new ways of seeing, engaging and analyzing digital images. Innovative methods and tools have assisted with the evaluation of large datasets, performing tasks such as classification, object detection, image description and style transfer or assisting with a form and content analysis. At this point, in order to progress, past works and established practices must be revisited and evaluated on the ground of their usability for art history. This paper provides a reflection from an art historical perspective to point to erroneous assumptions and where improvements are still needed.


Computer vision Art history Critical reflection Distant viewing Close reading Object detection Image description Style transfer 

Supplementary material

478816_1_En_49_MOESM1_ESM.pdf (95 kb)
Supplementary material 1 (pdf 95 KB)


  1. 1.
    Bishop, C.: Against digital art history. Int. J. Digi. Art Hist. (3), 123–133 (2018)Google Scholar
  2. 2.
    Boeykens, S., Maekelberg, S., De Jonge, K.: (Re-) creating the past: 10 years of digital historical reconstructions using bim. Int. J. Digit. Art Hist. (3), 63–87 (2018)Google Scholar
  3. 3.
    Bonfiglioli, R., Nanni, F.: From close to distant and back: how to read with the help of machines. In: Gadducci, F., Tavosanis, M. (eds.) HaPoC 2015. IAICT, vol. 487, pp. 87–100. Springer, Cham (2015). Scholar
  4. 4.
    Crowley, E.J., Zisserman, A.: In search of art. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8925, pp. 54–70. Springer, Cham (2015). Scholar
  5. 5.
    Crowley, E.J., Zisserman, A.: The art of detection. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 721–737. Springer, Cham (2016). Scholar
  6. 6.
    Drucker, J.: Is there a “digital” art history? Visual Resour. 29(1–2), 5–13 (2013)CrossRefGoogle Scholar
  7. 7.
    Drucker, J., Helmreich, A., Lincoln, M., Rose, F.: Digital art history: la scène américaine. une discussion entre johanna drucker, anne helmreich et matthew lincoln, introduite et modérée par francesca rose. Perspective. Actualité en histoire de l’art (2), 27–42 (2015)Google Scholar
  8. 8.
    Elgammal, A., Kang, Y., Leeuw, M.D.: Picasso, matisse, or a fake? Automated analysis of drawings at the stroke level for attribution and authentication. arXiv preprint arXiv:1711.03536 (2017)
  9. 9.
    Fouhey, D.F., Gupta, A., Zisserman, A.: 3D shape attributes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1516–1524 (2016)Google Scholar
  10. 10.
    Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576 (2015)
  11. 11.
    Gatys, L.A., Ecker, A.S., Bethge, M., Hertzmann, A., Shechtman, E.: Controlling perceptual factors in neural style transfer. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3730–3738 (2017)Google Scholar
  12. 12.
    Hatt, M., Klonk, C.: Art History: A Critical Introduction to Its Methods. Manchester University Press, Manchester (2006)Google Scholar
  13. 13.
    Hristova, S.: Images as data: cultural analytics and Aby Warburg’s Mnemosyne. Int. J. Digit. Art Hist. (2), 117–135 (2016)Google Scholar
  14. 14.
    Johnson, J., Karpathy, A., Fei-Fei, L.: DenseCap: fully convolutional localization networks for dense captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4565–4574 (2016)Google Scholar
  15. 15.
    Karayev, S., et al.: Recognizing image style. arXiv preprint arXiv:1311.3715 (2013)
  16. 16.
    Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 3128–3137 (2015)Google Scholar
  17. 17.
    Kienle, M.: Digital art history “beyond the digitized slide library”: an interview with Johanna Drucker and Miriam Posner. Artl@s Bull. 6(3), 9 (2017)Google Scholar
  18. 18.
    Kinghorn, P., Zhang, L., Shao, L.: A region-based image caption generator with refined descriptions. Neurocomputing 272, 416–424 (2018)CrossRefGoogle Scholar
  19. 19.
    Krause, J., Johnson, J., Krishna, R., Fei-Fei, L.: A hierarchical approach for generating descriptive image paragraphs. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3337–3345. IEEE (2017)Google Scholar
  20. 20.
    Li, J., Yao, L., Hendriks, E., Wang, J.Z.: Rhythmic brushstrokes distinguish van Gogh from his contemporaries: findings via automated brushstroke extraction. IEEE Tran. Pattern Anal. Mach. Intell. 34(6), 1159–1176 (2012)CrossRefGoogle Scholar
  21. 21.
    Li, L.J., Socher, R., Fei-Fei, L.: Towards total scene understanding: classification, annotation and segmentation in an automatic framework. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 2036–2043. IEEE (2009)Google Scholar
  22. 22.
    van der Maaten, L., Erdmann, R.G.: Automatic thread-level canvas analysis: a machine-learning approach to analyzing the canvas of paintings. IEEE Signal Process. Mag. 32(4), 38–45 (2015)CrossRefGoogle Scholar
  23. 23.
    Manovich, L.: How to compare one million images? In: Berry, D.M. (ed.) Understanding Digital Humanities, pp. 249–278. Palgrave Macmillan, London (2012). Scholar
  24. 24.
    Parikh, D., Grauman, K.: Relative attributes. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 503–510. IEEE (2011)Google Scholar
  25. 25.
    Riegl, A., Castriota, D., Zerner, H.: Problems of Style: Foundations for a History of Ornament. Princeton University Press, Princeton (1992)Google Scholar
  26. 26.
    Sanakoyeu, A., Kotovenko, D., Lang, S., Ommer, B.: A style-aware content loss for real-time HD style transfer. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11212, pp. 715–731. Springer, Cham (2018). Scholar
  27. 27.
    Tan, W.R., Chan, C.S., Aguirre, H.E., Tanaka, K.: Ceci n’est pas une pipe: a deep convolutional network for fine-art paintings classification. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3703–3707. IEEE (2016)Google Scholar
  28. 28.
    Underhill, J.: In conversation with CyArk: digital heritage in the 21st century. Int. J. Digit. Art Hist. (3), 111–123 (2018)Google Scholar
  29. 29.
    Warburg, A.: Der Bilderatlas Mnemosyne, vol. 2. Akademie Verlag, Berlin (2008)Google Scholar
  30. 30.
    Wilber, M.J., Fang, C., Jin, H., Hertzmann, A., Collomosse, J., Belongie, S.: BAM! the behance artistic media dataset for recognition beyond photography. In: Proceedings of the ICCV, vol. 1, pp. 1211–1220 (2017)Google Scholar
  31. 31.
    Wölfflin, H.: Principles of Art History. Courier Corporation, Chelmsford (2012)Google Scholar
  32. 32.
    Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251. IEEE, October 2017Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Heidelberg Collaboratory for Image Processing, IWR, Heidelberg UniversityHeidelbergGermany

Personalised recommendations