Abstract
Fine-art painting expresses the state of mind and social culture of mankind. Automatic fine-art painting classification is an important task to assist the analysis of fine-art paintings. In this paper, we propose a novel two-channel dual path networks for the task of style, artist and genre classification on fine-art painting image. It includes the RGB and the brush stroke information channels. Besides the RGB information channel is used to represent the color information in fine-art painting images, the brush stroke information channel is used to extract brush stroke information from fine-art painting images. And the four-directional gray-level co-occurrence matrix is used in deep learning to detect the brush stroke information, which has never been considered in the task of fine-art painting classification. Experiments on two datasets demonstrate that the four-directional gray-level co-occurrence matrix is effective in feature representation of fine-art painting images. And the proposed model achieves best classification accuracy and good generalization performance when compared with other methods.
Similar content being viewed by others
References
Štefko R, KrajĖák J (2013) An analytical view on fine arts marketing. Jerzy Kukuczka Academy of Physical Education Publishers Inc., Katowice
Alawa P (2017) Hegel and Gadamer on the contemporary understanding of art: an evaluation. Int J Philos Theol 5(2):58–65
Sultanova M, Mikhailova N, Amanzholova D (2016) Between the east and the west: Reflections on the contemporary art of Kazakhstan. Folklore 6:45–64
Polatkan G, Jafarpour S, Brasoveanu A, Hughes S, Daubechies I (2009) Detection of forgery in paintings using supervised learning. In: IEEE international conference on image processing (ICIP), pp 2921–2924
Crowley EJ, Zisserman A (2014) In search of art. In: European conference on computer vision (ECCV), pp 54–70
Crowley EJ, Zisserman A (2014) The state of the art: object retrieval in paintings using discriminative regions. In: British machine vision conference (BMVC), pp 1–12
Mensink T, Van Gemert J (2014) The Rijksmuseum challenge: Museumcentered visual recognition. In: Proceedings of international conference on multimedia retrieval (ICMR), pp 451–454
Gatys LA, Ecker AS, Bethge M (2015) A neural algorithm of artistic style. arXiv:1508.06576 (preprint)
Khan FS, Beigpour S, Van-De-Weijer J, Felsberg M (2014) Painting-91: a large scale database for computational painting categorization. Mach Vis Appl 25(6):1385–1397
Hentschel C, Wiradarma TP, Sack Harald (2016) Fine tuning CNNs with scarce training data-adapting ImageNet to art epoch classification. In: IEEE international conference on image processing (ICIP), pp 3693–3697
Krizhevsky A., Hinton G (2012) ImageNet classification with deep convolutional neural networks. In: Advances in neural information processing systems (NIPS), pp 1097–1105
Zhong S, Liu Y, Hua K (2016) Field effect deep networks for image recognition with incomplete data. ACM Trans Multimed Comput Commun Appl (TOMM) 12(4):52
Zhong S, Liu Y, Li B, Long J (2015) Query-oriented unsupervised multi-document summarization via deep learning. Expert Syst Appl (ESWA) 42(21):8146–8155
Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems (NIPS), pp 91–99
Banharnsakun A (2018) Towards improving the convolutional neural networks for deep learning using the distributed artificial bee colony method. Int J Mach Learn Cybern. https://doi.org/10.1007/s13042-018-0811-z
Pradhan R, Aygun RS, Maskey M, Ramachandran R, Daniel J (2018) Cecil.: tropical cyclone intensity estimation using a deep convolutional neural network. IEEE Trans Image Process 27(2):199–206
Xu X, Pan J, Zhang Y, Yang M (2018) Motion blur kernel estimation via deep learning. IEEE Trans Image Process 27(1):194–205
He K, Zhang X, Ren S, Jian S (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778
Huang G, Liu Z, Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 30th IEEE conference on computer vision and pattern recognition (CVPR), pp 4700–4708
Chen Y, Li J, Xiao H, Jin X, Yan S, Feng JS (2017) Dual path networks. In: 31st conference on neural information processing systems (NIPS), pp 4467–4475
Li L (2011) The relationship between the brush strokes and the image, the color, the emotion. J Yangtze Univ 34(9):181–182
Johnson CR, Hendriks E, Berezhnoy I, Brevdo E, Hughes S, Daubechies I, Li J, Postma E, Wang JZ (2008) Image processing for artist identification—computerized analysis of Vincent Van Gogh's painting brushstrokes. IEEE Signal Process Mag Special Issue Visual Cult Heritage 25:37–48
Li J, Yao L, Hendriks E, Wang JZ (2012) Rhythmic brushstrokes distinguish Van Gogh from his contemporaries: Findings via automated brush stroke extraction. IEEE Trans Pattern Anal Mach Intell 34:1159–1176
Santosh KC, Roy PP (2018) Arrow detection in biomedical images using sequential classifier. Int J Mach Learn Cybern 9(6):993–1006
Huang R, Zhang G, Chen J (2018) Semi-supervised discriminant Isomap with application to visualization, image retrieval and classification. Int J Mach Learn Cybern 2018:1–10. https://doi.org/10.1007/s13042-018-0809-6
Torresani L, Szummer M, Fitzgibbon A (2010) Efficient object category recognition using classemes. In: European conference on computer vision (ECCV), pp 776–789
Saleh B, Elgammal A (2015) Large-scale classification of fine-art paintings: learning the right metric on the right feature. Int J Digital Art Hist 2:70–93
Falomir Z, Museros L, Sanz I, Gonzalez-Abril L (2018) Categorizing paintings in art styles based on qualitative color descriptors, quantitative global features and machine learning (QArt-Learn). Expert Syst Appl 97:83–94
Lecoutre A, Negrevergne B, Yger F (2017) Recognizing art style automatically in painting with deep learning. In: Asian conference on machine learning (ACML), pp 327–342
Sabatelli M, Kestemont M, Daelemans W, Geurts P (2018) Deep transfer learning for art classification problems. In: European conference on computer vision (ECCV), pp 1–16.
Peng K, Chen T (2015) Cross-layer features in convolutional neural networks for generic classification tasks. In: IEEE international conference on image processing (ICIP), pp 3057–3061
Peng K, Chen T (2015) A framework of extracting multi-scale features using multiple convolutional neural networks. In: International conference on multimedia and expo (ICME), pp 1–6
Tan W, Chan CS, Aguirre H, Tanaka, K (2016) Ceci n’est pas une pipe: A deep convolutional network for fine-art paintings classification. In: IEEE international conference on image processing (ICIP), pp 3703–3707
Sun T, Wang Y, Yang J, Hu X (2017) Convolution neural networks with two pathways for image style recognition. IEEE Trans Image Process 26(9):4102–4113
Deac A. I, Lubbe J, Backer E (2006) Feature selection for paintings classification by optimal tree pruning. In: Multimedia content representation, classification and security, pp 354–361
Agarwal S, Karnick H, Pant N, Patel U (2015) Genre and style based painting classification. In: IEEE winter conference on applications of computer vision (WACV), pp 588–594
Cetinić E, Grgić S (2013) Automated painter recognition based on image feature extraction. In: Proceedings ELMAR-2013. Zadar, pp 19–22.
Huang X, Zhong S, Xiao Z (2017) Fine-art painting classification via two-channel deep residual network. In: Advances in multimedia information processing—Pacific-Rim conference on multimedia (PCM), pp 79–88
Ulyanov D, Lebedev V, Vedaldi A, Lempitsky V (2016) Texture networks: feed-forward synthesis of textures and stylized images. In: International conference on machine learning (ICML), pp 1349–1357
Acknowledgements
This work was supported by the National Natural Science Foundation of China (No. 61620106008), the Natural Science Foundation of Guangdong Province (Nos. 2016A030310053, 2016A030310039, 2017A030310521), the Science and Technology Innovation Commission of Shenzhen under Grant (No. JCYJ20160422151736824), the Shenzhen high-level overseas talents program, the National Engineering Laboratory for Big Data System Computing Technology, the Inlife-Handnet Open Fund, and the Postgraduate Innovation Development Fund Project of Shenzhen University (No. PIDFPZR2018001).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhong, Sh., Huang, X. & Xiao, Z. Fine-art painting classification via two-channel dual path networks. Int. J. Mach. Learn. & Cyber. 11, 137–152 (2020). https://doi.org/10.1007/s13042-019-00963-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-019-00963-0