Advertisement

Multimedia Tools and Applications

, Volume 78, Issue 22, pp 31893–31924 | Cite as

Artwork painting identification method for panorama based on adaptive rectilinear projection and optimized ASIFT

  • DaYou Jiang
  • Jongweon KimEmail author
Article
  • 46 Downloads

Abstract

In the paper, the authors present an artwork painting identification method for panorama based on adaptive rectilinear projection and optimized ASIFT (Affine Scale-Invariant Feature Transform). Firstly, the authors use the panorama dataset to train the artwork painting detection network to obtain the location information of artwork paintings. Secondly, the authors use the adaptive rectilinear projection to map the artwork painting into a square image with a fixed size. Then the authors use the image enhancement method to improve the image quality. Finally, the authors use the optimized ASIFT for features extraction and image matching. Several contrast experiments were conducted on the artwork paintings panorama dataset for artwork paintings identification. The results show that the proposed method can achieve 96% identification accuracy on average for the whole test artwork paintings panorama dataset. The proposed adaptive rectilinear based-method can improve at least 20% of the recognition accuracy. The proposed optimized ASIFT can improve at least 30% of the identification accuracy than SIFT. The authors also study other factors such as the size of the original artwork image, the image matching threshold, whether using image enhancement or not. The results show the size of the original artwork has little influence on the artwork identification in the panorama. The image matching threshold with 2.0 is better than 3.0. Furthermore, using the image enhancement method can improve about 2% of the identification accuracy.

Keywords

Artwork painting identification Panorama Object detection Rectilinear projection Feature extraction 

Notes

References

  1. 1.
    Alahi A, Ortiz R, Vandergheynst P (2012) Freak: Fast retina keypoint. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp 510-517.  https://doi.org/10.1109/CVPR.2012.6247715
  2. 2.
    Bay H, Tuytelaars T, Gool L V (2006) Surf: Speeded up robust features. In European conference on computer vision. Springer, Berlin, Heidelberg, pp 404-417.  https://doi.org/10.1007/11744023_32
  3. 3.
    Becattini F, Ferracani A, Landucci L (2016) Imaging Novecento: A Mobile App for Automatic Recognition of Artworks and Transfer of Artistic Styles. Euro-Mediterranean Conference EuroMed 2016, Digital Heritage. Progress in Cultural Heritage: Documentation, Preservation, and Protection, Springer, Cham, pp 781-791.  https://doi.org/10.1007/978-3-319-48496-9_62
  4. 4.
    Brown M, Lowe DG (2007) Automatic panoramic image stitching using invariant features. Int J Comput Vis 74(1):59–73.  https://doi.org/10.1007/s11263-006-0002-3 CrossRefGoogle Scholar
  5. 5.
    Crowley EJ (2016) Visual recognition in art using machine learning. University of Oxford, https://www.robots.ox.ac.uk/~vgg/publications/2016/Crowley16a/crowley16a-low-res.pdf. Accessed 25 April 2019
  6. 6.
    Dai J, Li Y, He KM (2016) R-fcn: Object detection via region-based fully convolutional networks. In Advances in neural information processing systems, pp 379-387. https://arxiv.org/abs/1605.06409
  7. 7.
    Dalens T, Sivic J, Laptev I (2014) Campedel, M. Painting recognition from wearable cameras. Painting recognition from wearable cameras. Technical Report hal-01062126, INRIA, 2014. https://hal.inria.fr/hal-01062126. Accessed 25 April 2019
  8. 8.
    Deng F, Zhu X, Ren J (2017) Object detection on panoramic images based on deep learning. In: IEEE 2017 3rd International Conference on Control, Automation and Robotics (ICCAR), pp 375-380.  https://doi.org/10.1109/ICCAR.2017.7942721
  9. 9.
    Fernando R, Kilgard MJ (2003) The Cg Tutorial: The definitive guide to programmable real-time graphics. Addison-Wesley Longman Publishing Co., Inc., BostonGoogle Scholar
  10. 10.
    Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Morgan Kaufmann Publishers Inc, San FranciscoGoogle Scholar
  11. 11.
    Florea C, Condorovici R, Vertan C, Butnaru R, Florea L, Vrânceanu R (2016) Pandora: Description of a painting database for art movement recognition with baselines and perspectives. In IEEE 2016 24th European Signal Processing Conference (EUSIPCO), pp 918-922. https://arxiv.org/abs/1602.08855
  12. 12.
    Fu H, Xie D, Zhong R (2017) An improved ASIFT algorithm for indoor panorama image matching. In Ninth International Conference on Digital Image Processing (ICDIP 2017), Vol. 10420, pp. 104201C. International Society for Optics and Photonics. 10.1117/12.2281615Google Scholar
  13. 13.
    Fu, X., Zeng, D., Huang, Y (2016) A weighted variational model for simultaneous reflectance and illumination estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2782-2790. 10.1109/CVPR.2016.304Google Scholar
  14. 14.
    Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580-587. https://arxiv.org/abs/1311.2524
  15. 15.
    He K M, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770-778. 10.1109/CVPR.2016.90Google Scholar
  16. 16.
    Jiang DY, Kim JW (2018) Artwork Recognition for Panorama Images Based on Optimized ASIFT and Cubic Projection. International Journal of Machine Learning and Computing 8(1):54–60.  https://doi.org/10.18178/ijmlc.2018.8.1.663 CrossRefGoogle Scholar
  17. 17.
    Jin X, Kim JW (2017) Artwork Identification for 360-Degree Panorama Images Using Polyhedron-Based Rectilinear Projection and Keypoint Shapes. Appl Sci 7(5):528.  https://doi.org/10.3390/app7050528 CrossRefGoogle Scholar
  18. 18.
    Johnson CR, Hendriks E, Berezhnoy IJ (2008) Image processing for artist identification. IEEE Signal Process Mag 25(4):37–48.  https://doi.org/10.1109/MSP.2008.923513 CrossRefGoogle Scholar
  19. 19.
    Lee C, Kim CS (2013) Contrast enhancement based on layered difference representation of 2D histograms. IEEE Trans Image Process 22(12):5372–5384.  https://doi.org/10.1109/TIP.2013.2284059 CrossRefGoogle Scholar
  20. 20.
    Levi G, Hassner T (2016) LATCH: learned arrangements of three patch codes. In 2016 IEEE winter conference on applications of computer vision (WACV), pp 1-9. https://arxiv.org/abs/1501.03719
  21. 21.
    Liu Y, Zhang, K Yao J, He T, Liu Y, Tu J (2015). An efficient method for text detection from indoor panorama images using extremal regions. In 2015 IEEE International Conference on Information and Automation, pp 781-786. 10.1109/ICInfA.2015.7279390Google Scholar
  22. 22.
    Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110.  https://doi.org/10.1023/B:VISI.0000029664.99615.94 CrossRefGoogle Scholar
  23. 23.
    Matas J, Chum O, Urban M, Pajdla T (2004) Robust wide-baseline stereo from maximally stable extremal regions. Image Vis Comput 22(10):761–767.  https://doi.org/10.1016/j.imavis.2004.02.006 CrossRefGoogle Scholar
  24. 24.
    mexopencv. Collection and a development kit of matlab mex functions for OpenCV library. https://kyamagu.github.io/mexopencv/. Accessed 25 April 2019
  25. 25.
    Morel JM, Yu G (2009) ASIFT: A new framework for fully affine invariant image comparison. SIAM Journal on Imaging Sciences (SIIMS) 2(2):438–469.  https://doi.org/10.1137/080732730 MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Pizer SM, Amburn EP, Austin JD (1987) Adaptive histogram equalization and its variations. Computer Vision, Graphics, and Image Processing 39(3):355–368.  https://doi.org/10.1016/S0734-189X(87)80186-X CrossRefGoogle Scholar
  27. 27.
    Redmon J, Farhadi A (2016) YOLO9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263-7271. 10.1109/CVPR.2017.690Google Scholar
  28. 28.
    Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: An efficient alternative to SIFT or SURF. ICCV 11(1):2.  https://doi.org/10.1109/ICCV.2011.6126544 CrossRefGoogle Scholar
  29. 29.
    Saleh, B., Elgammal, A (2015) Large-scale classification of fine-art paintings: Learning the right metric on the right feature. arXiv preprint arXiv:1505.00855. 10.11588/dah.2016.2.23376Google Scholar
  30. 30.
    Su H, Gong S, Zhu X (2018) Scalable deep learning logo detection. arXiv preprint arXiv:1803.11417. https://arxiv.org/pdf/1803.11417.pdf
  31. 31.
    Vedaldi A, Fulkerson B (2010) VLFeat: An open and portable library of computer vision algorithms. In: Proceedings of the 18th ACM international conference on Multimedia, pp 1469-1472. http://www.robots.ox.ac.uk/~vgg/publications-new/Public/2010/Vedaldi10a/vedaldi10a.pdf
  32. 32.
    Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. CVPR 1(1):511–518.  https://doi.org/10.1109/CVPR.2001.990517 CrossRefGoogle Scholar
  33. 33.
    Viswanathan N (2017) Artist Identification with Convolutional Neural Networks. Technical report, Stanford University. http://cs231n.stanford.edu/reports/2017/pdfs/406.pdf. Accessed 25 April 2019
  34. 34.
    Wang K. H., Lai SH (2019) Object Detection in Curved Space for 360-Degree Camera. In: 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3642-3646. 10.1109/ICASSP.2019.8683093Google Scholar
  35. 35.
    Yang W, Qian Y, Kämäräinen JK, Cricri F, Fan L (2018). Object detection in equirectangular panorama. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp 2190-2195. https://arxiv.org/pdf/1805.08009.pdf
  36. 36.
    Ying Z, Li G, Ren Y (2017) A new image contrast enhancement algorithm using exposure fusion framework. In: International Conference on Computer Analysis of Images and Patterns. Springer, Cham, pp 36-46. 10.1007/978-3-319-64698-54Google Scholar
  37. 37.
    Zuiderveld K (1994) Contrast limited adaptive histogram equalization. Graphics gems IV, Academic Press Professional, Inc., San DiegoCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Copyright ProtectionSangmyung UniversitySeoulSouth Korea
  2. 2.Department of Electronics EngineeringSangmyung UniversitySeoulSouth Korea

Personalised recommendations