Abstract
The classification of digital images is an essential task during the restoration and preservation of cultural heritage (CH). In computer vision, cultural heritage classification relies on the classification of asset images regarding a certain task such as type, artist, genre, style identification, etc. CH classification is challenging as various CH asset images have similar colors, textures, and shapes. In this chapter, the aim is to study and evaluate the use of pre-trained deep convolutional neural networks such as VGG16, VGG-19, ResNet50, and Inception-V3 for cultural heritage images classification using transfer learning techniques. The main idea is to start with CNN models previously trained for definite tasks with specific datasets and classes, instead of designing a full stand-alone CNN-based model. Two image datasets are used to validate the performance of these models in CH images classification. The experimental results showed that the Inception-V3-based model can achieve high levels of classification accuracy compared to other pre-trained CNNs studied in this chapter.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Saleh, B., Elgammal, A.: Large-scale classification of fine-art paintings: learning the right metric on the right feature. arXiv preprint arXiv:1505.00855 (2015)
Agarwal, S., Karnick, H., Pant, N., Patel, U.: Genre and style based painting classification. In: 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 588–594 (2015)
Van Noord, N., Hendriks, E., Postma, E.: Towards discovery of the artist‘s style: learning to recognize artists by their artworks. IEEE Signal Process. Mag. 32, 46–54 (2015)
Jboor, N.H., Belhi, A., Al-Ali, A.K., Bouras, A., Jaoua, A.: Towards an inpainting framework for visual cultural heritage. In: 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT), pp. 602–607. IEEE (2019)
Belhi, A., Bouras, A., Foufou, S.: Leveraging known data for missing label prediction in cultural heritage context. Appl. Sci. 8, 1768 (2018)
Belhi, A., Al-Ali, A.K., Bouras, A., Foufou, S., Yu, X., Zhang, H.: Investigating low-delay deep learning-based cultural image reconstruction. J. Real-Time Image Proc. 17, 1–16 (2020)
Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep Learning. MIT Press, Cambridge (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105. Morgan Kaufmann Publishers, San Mateo (2012)
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Mensink, T., Van Gemert, J.: The rijksmuseum challenge: museum-centered visual recognition. In: Proceedings of International Conference on Multimedia Retrieval, p. 451. ACM, New York (2014)
Shamir, L., Macura, T., Orlov, N., Eckley, D.M., Goldberg, I.G.: Impressionism, expressionism, surrealism: automated recognition of painters and schools of art. ACM Trans. Appl. Percept. 7, 8 (2010)
Puthenputhussery, A., Liu, Q., Liu, C.: Color multi-fusion fisher vector feature for fine art painting categorization and influence analysis. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–9. IEEE (2016)
Li, J., Yao, L., Hendriks, E., Wang, J.Z.: Rhythmic brushstrokes distinguish van Gogh from his contemporaries: findings via automated brushstroke extraction. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1159–1176 (2012)
Arora, R.S., Elgammal, A.: Towards automated classification of fine-art painting style: a comparative study. In: 2012 21st International Conference on Pattern Recognition (ICPR), pp. 3541–3544. IEEE (2012)
Sheng, J., Jiang, J.: Recognition of Chinese artists via windowed and entropy balanced fusion in classification of their authored ink and wash paintings (IWPs). Pattern Recogn. 47, 612–622 (2014)
Tseng, T.-E., Chang, W.-Y., Chen, C.-S., Wang, Y.-C.F.: Style retrieval from natural images. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1561–1565. IEEE (2016)
Chu, W.-T., Wu, Y.-L.: Deep correlation features for image style classification. In: Proceedings of the 2016 ACM on Multimedia Conference, pp. 402–406. ACM (2016)
Saleh, B., Elgammal, A.: A unified framework for painting classification. In: 2015 IEEE International Conference on Data Mining Workshop (ICDMW), pp. 1254–1261. IEEE (2015)
Hong, Y., Kim, J.: Art painting detection and identification based on deep learning and image local features. Multimed. Tools Appl. 2018, 1–16 (2018)
Karayev, S., Trentacoste, M., Han, H., Agarwala, A., Darrell, T., Hertzmann, A., Winnemoeller, H.: Recognizing image style. arXiv preprint arXiv:1311.3715 (2013)
Elgammal, A., Kang, Y., Leeuw, M.D.: Picasso, matisse, or a fake? Automated analysis of drawings at the stroke level for attribution and authentication. arXiv preprint arXiv:1711.03536 (2017)
Bianco, S., Mazzini, D., Schettini, R.: Deep multibranch neural network for painting categorization. In: International Conference on Image Analysis and Processing, pp. 414–423. World Scientific, Singapore (2017)
Mao, H., Cheung, M., She, J.: DeepArt: learning joint representations of visual arts. In: Proceedings of the 2017 ACM on Multimedia Conference, pp. 1183–1191. Association for Computing Machinery, New York (2017)
Seguin, B., Striolo, C., Kaplan, F.: Visual link retrieval in a database of paintings. In: European Conference on Computer Vision, pp. 753–767. Springer, Berlin/Heidelberg (2016)
Belhi, A., Bouras, A.: Towards a multimodal classification of cultural heritage. In: Qatar Foundation Annual Research Conference Proceedings Volume 2018 Issue 3, p. ICTPD1010. Hamad Bin Khalifa University Press (HBKU Press) (2018)
Huang, X., Zhong, S.-h., Xiao, Z.: Fine-art painting classification via two-channel deep residual network. In: Pacific Rim Conference on Multimedia, pp. 79–88. Springer, Berlin/New York (2017)
Llamas, J., M Lerones, P., Medina, R., Zalama, E., Gómez-GarcÃa-Bermejo, J.: Classification of architectural heritage images using deep learning techniques. Appl. Sci. 7, 992 (2017)
Cetinic, E., Lipic, T., Grgic, S.: Fine-tuning convolutional neural networks for fine art classification. Expert Syst. Appl. 114, 107–118 (2018)
Ma, D., Gao, F., Bai, Y., Lou, Y., Wang, S., Huang, T., Duan, L.-Y.: From part to whole: who is behind the painting? In: Proceedings of the 2017 ACM on Multimedia Conference, pp. 1174–1182. ACM, New York (2017)
Belhi, A., Bouras, A., Alfaqheri, T., Aondoakaa, A.S., Sadka, A.H.: Investigating 3D holoscopic visual content upsampling using super-resolution for cultural heritage digitization. Signal Process. Image Commun. 75, 188–198 (2019)
Belhi, A., Bouras, A., Foufou, S.: Towards a hierarchical multitask classification framework for cultural heritage. In: 2018 IEEE/ACS 15th International Conference on Computer Systems and Applications (AICCSA), pp. 1–7. IEEE, Piscataway (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Ha, I., Kim, H., Park, S., Kim, H.: Image retrieval using BIM and features from pretrained VGG network for indoor localization. Build. Environ. 140, 23–31 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. IEEE, Las Vegas (2016)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826. IEEE Computer Society, Los Alamitos (2016)
Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328. Morgan Kaufmann Publishers, San Mateo (2014)
Chollet, F.: Keras: deep learning library for theano and tensorflow. URL: https://keras.io 7, 8 (2015)
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M.: Tensorflow: a system for large-scale machine learning. In: OSDI, pp. 265–283. USENIX Association, Berkeley (2016)
Bisong, E.: Google colaboratory. In: Building Machine Learning and Deep Learning Models on Google Cloud Platform, pp. 59–64. Springer, New York (2019)
Acknowledgments
This work was made possible by NPRP grant 9-181-1-036 from the Qatar National Research Fund (a member of Qatar Foundation). The statements made herein are solely the responsibility of the authors (www.ceproqha.qa).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Belhi, A., Ahmed, H.O., Alfaqheri, T., Bouras, A., Sadka, A.H., Foufou, S. (2021). Study and Evaluation of Pre-trained CNN Networks for Cultural Heritage Image Classification. In: Belhi, A., Bouras, A., Al-Ali, A.K., Sadka, A.H. (eds) Data Analytics for Cultural Heritage. Springer, Cham. https://doi.org/10.1007/978-3-030-66777-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-66777-1_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66776-4
Online ISBN: 978-3-030-66777-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)