Image classification by combining local and global features

Original Article
  • 62 Downloads

Abstract

Several techniques have recently been proposed to extract the features of an image. Feature extraction is one of the most important steps in various image processing and computer vision applications such as image retrieval, image classification, matching, object recognition. Relevant feature (global or local) contains discriminating information and is able to distinguish one object from others. Global features describe the entire image, whereas local features describe the image patches (small group of pixels). In this paper, we present a novel descriptor to extract the color-texture features via two information types. Our descriptor named concatenation of local and global color features is based on the fusion of global features using wavelet transform and a modified version of local ternary pattern, whereas, for the local features, speeded-up robust feature descriptor and bag of words model were used. All the features are extracted from the three color planes. To evaluate the effectiveness of our descriptor for image classification, we carried out experiments using the challenging datasets: New-BarkTex, Outex-TC13, Outex-TC14, MIT scene, UIUC sports event, Caltech 101 and MIT indoor scene. Experimental results showed that our descriptor outperforms the existing state-of-the-art methods.

Keywords

SURF BoW LBP LTP Wavelet transform Image classification 

References

  1. 1.
    Yu, J., Qin, Z., Wan, T., Zhang, X.: Feature integration analysis of bag of features model for image retrieval. Neurocomputing 120, 355–364 (2013)CrossRefGoogle Scholar
  2. 2.
    Banerji, S., Verma, A., Liu, C.: Cross disciplinary biometric systems. LBP and Color Descriptors for Image Classification, pp. 205–225. Springer, Berlin (2012)Google Scholar
  3. 3.
    Ledoux, A., Losson, O., Macaire, L.: Color local binary patterns: compact descriptors for texture classification. J Electron Imaging 25(6), 061404 (2016)CrossRefGoogle Scholar
  4. 4.
    Yongsheng, D., et al.: Multi-scale counting and difference representation for texture classification. Vis. Comput. (2017).  https://doi.org/10.1007/s00371-017-1415-4 Google Scholar
  5. 5.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  6. 6.
    Xuemei, H., Yan, D.: Image matching with an improved descriptor based on SIFT. In: Proceedings Volume 10322, Seventh International Conference on Electronics and Information Engineering, pp. 1-7 (2017).  https://doi.org/10.1117/12.2265595
  7. 7.
    Romero, A., Gatta, C., Camps-Valls, G.: Unsupervised deep feature extraction for remote sensing image classification. IEEE Trans. Geosci. Remote Sens. 54(3), 1349–1362 (2016)CrossRefGoogle Scholar
  8. 8.
    Berbar, M.: Three robust features extraction approaches for facial gender classification. Vis. Comput. 30(1), 19–31 (2014)CrossRefGoogle Scholar
  9. 9.
    Bay, H., et al.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)CrossRefGoogle Scholar
  10. 10.
    Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. 29(1), 51–59 (1996)CrossRefGoogle Scholar
  11. 11.
    Li, Y., Ruixi, Z., Nan, M., Yi, L.: Improved class-specific codebook with two-step classification for scene-level classification of high resolution remote sensing images. Remote Sens. 9(3), 1–24 (2017)Google Scholar
  12. 12.
    Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Proceedings of the International Workshop on Statistical Learning in Computer Vision, pp. 1–16 (2004)Google Scholar
  13. 13.
    Xiaoyong, B., Chen, C., Long, T., Qian, D.: Fusing local and global features for high-resolution scene classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10(6), 2889–2901 (2017)CrossRefGoogle Scholar
  14. 14.
    Daoyu, L., et al.: Marta gans: unsupervised representation learning for remote sensing image classification. IEEE Geosci. Remote Sens. Lett. 14(11), 1–5 (2017)CrossRefGoogle Scholar
  15. 15.
    Alex, K., Ilya, S., Geoffrey, E.H.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  16. 16.
    Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. CoRR, arXiv:1405.3531 (2014)
  17. 17.
    Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. In: Advances in Neural Information Processing Systems, pp. 487–495 (2014)Google Scholar
  18. 18.
    Lee, H., Grosse, R., Ranganath, R., Ng, A.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings Annual International Conference on Machine Learning, pp. 609–616 (2009)Google Scholar
  19. 19.
    Shuang, B., Zhaohong, L., Jianjun, H.: Learning two-pathway convolutional neural networks for categorizing scene images. Multimed. Tools Appl. 76(15), 16145–16162 (2017)CrossRefGoogle Scholar
  20. 20.
    Sandid, F., Douik, A.: Robust color texture descriptor for material recognition. Pattern Recognit. Lett. 80, 15–23 (2016)CrossRefGoogle Scholar
  21. 21.
    Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. IEEE Trans. Image Process. 19(6), 1657–1663 (2010)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Tan, X., Triggs, B.: Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans. Image Process. 19(6), 1635–1650 (2010)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Nishant, S., Vipin, T.: An effective scheme for image texture classification based on binary local structure pattern. Vis. Comput. 30(11), 1223–1232 (2014)CrossRefGoogle Scholar
  24. 24.
    Xiaosheng, W., Junding, S.: Joint-scale LBP: a new feature descriptor for texture classification. Vis. Comput. 33(3), 317–329 (2017)CrossRefGoogle Scholar
  25. 25.
    Rahman, M.M., et al.: DTCTH: a discriminative local pattern descriptor for image classification. EURASIP J. Image. Video Process. 2017, 1–24 (2017)Google Scholar
  26. 26.
    Khan, R., Muselet, D., Trémeau, A.: Texture classification across illumination color variations. Int. J. Comput. Theory Eng. 5, 65–70 (2013)CrossRefGoogle Scholar
  27. 27.
    Alvarez, S., Vanrell, M.: Texton theory revisited: a bag-of-words approach to combine textons. Pattern Recognit. 45(12), 4312–4325 (2012)CrossRefGoogle Scholar
  28. 28.
    Zhu, C., et al.: Image region description using orthogonal combination of local binary patterns enhanced with color information. Pattern Recognit. 46(7), 1949–1963 (2013)CrossRefGoogle Scholar
  29. 29.
    Porebski, A., et al.: A new benchmark image test suite for evaluating colour texture classification schemes. Multimed. Tools Appl. 70(1), 543–556 (2014)CrossRefGoogle Scholar
  30. 30.
    Cusano, C., Napoletano, P., Schettini, R.: Combining local binary patterns and local color contrast for texture classification under varying illumination. J. Opt. Soc. Am. A 31(7), 1453–1461 (2014)CrossRefGoogle Scholar
  31. 31.
    Sandid, F., Douik, A.: Texture descriptor based on local combination adaptive ternary pattern. IET Image Process. 9(8), 634–642 (2015)CrossRefGoogle Scholar
  32. 32.
    Kabbai, L., Abdellaoui, M., Douik, A.: Content based image retrieval using local and global features descriptor. In: IEEE International Conference on Advanced Technologies for Signal and Image Processing, pp. 151–154 (2016)Google Scholar
  33. 33.
    Kabbai, L., Abdellaoui, M., Douik, A.: Hybrid local and global descriptor enhanced with colour information. IET Image Process. 11(2), 109–117 (2016)CrossRefGoogle Scholar
  34. 34.
    Papadopoulos, G.Th, Mezaris, V., Kompatsiaris, I., Strintzis, M.G.: Combining global and local information for knowledge-assisted image analysis and classification. EURASIP J. Image Video Process. 2007, 1–15 (2007)MATHGoogle Scholar
  35. 35.
    Banerji, S., Sinha, A., Chengjun, L.: New image descriptors based on color, texture, shape, and wavelets for object and scene image classification. Neurocomputing 117, 173–185 (2013)CrossRefGoogle Scholar
  36. 36.
    Sinha, A., Banerji, S., Liu, C.: New color GPHOG descriptors for object and scene image classification. Mach. Vis. Appl. 25(2), 361–375 (2014)CrossRefGoogle Scholar
  37. 37.
    Khan, R., Barat, C., Muselet, D., Ducottet, C.: Spatial histograms of soft pairwise similar patches to improve the bag-of-visual words model. Comput. Vis. Image Underst. 132, 102–112 (2015)CrossRefGoogle Scholar
  38. 38.
    Amit, S., Xudong, J., How, L.E.: LBP-based edge-texture features for object recognition. IEEE Trans. Image Process. 23(5), 1953–1964 (2014)MathSciNetCrossRefMATHGoogle Scholar
  39. 39.
    Maji, S., Berg, A.C., Malik, J.: Efficient classification for additive kernel SVMs. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 66–77 (2013)CrossRefGoogle Scholar
  40. 40.
    Li, L.J., et al.: Object bank: A high-level image representation for scene classification & semantic feature sparsification. In: Advances in Neural Information Processing Systems, pp. 1378–1386 (2010)Google Scholar
  41. 41.
    Li, L.J., Li, F.F.: What, where and who? Classifying events by scene and object recognition. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  42. 42.
    Bo, L., Ren, X., Fox, D.: Hierarchical matching pursuit for image classification: architecture and fast algorithms. In: Advances in Neural Information Processing Systems, pp. 2115–2123 (2011)Google Scholar
  43. 43.
    Harris, C., Stephens, M.J.: A combined corner and edge detector. In: Proceedings of Fourth Alvey Vision Conference, pp. 147–151 (1988)Google Scholar
  44. 44.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)CrossRefGoogle Scholar
  45. 45.
    Ke, Y., Sukthankar, R.: PCA-SIFT: a more distinctive representation for local image descriptors. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 506–513 (2004)Google Scholar
  46. 46.
    Kabbai, L., Abdellaoui, M., Douik, A.: New robust descriptor for image matching. J. Theor. Appl. Inf. Technol. 87(3), 451–460 (2016)Google Scholar
  47. 47.
    Ai, D.N., et al.: Color independent components based SIFT descriptors for object/scene classification. IEICE Trans. Inf. Syst. 93(9), 2577–2586 (2010)CrossRefGoogle Scholar
  48. 48.
    Muralidharan, R., Chandrasekar, C.: Combining local and global feature for object recognition using SVM-KNN. In: IEEE International Conference on Pattern Recognition, Informatics and Medical Engineering, pp. 1–7 (2012)Google Scholar
  49. 49.
    Chaudhary, M.D., Upadhyay, A.B.: Fusion of local and global features using stationary wavelet transform for efficient content based image retrieval. In: IEEE Students’ Conference on Electrical, Electronics and Computer Science, pp. 1–6 (2014)Google Scholar
  50. 50.
    Li, L., et al.: Fusion framework for color image retrieval based on bag-of-words model and color local haar binary patterns. J. Electron. Imaging 25(2), 023022 (2016)CrossRefGoogle Scholar
  51. 51.
    Zou, J., et al.: Scene classification using local and global features with collaborative representation fusion. Inf. Sci. 348, 209–226 (2016)MathSciNetCrossRefGoogle Scholar
  52. 52.
    Mallat, S.G.: theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 11(7), 674–693 (1989)CrossRefMATHGoogle Scholar
  53. 53.
    Smith, J. R., Chang, S. F.: Automated binary texture feature sets for image retrieval. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 2239–2242 (1996)Google Scholar
  54. 54.
    Ojala, T., et al.: Outex new framework for empirical evaluation of texture analysis algorithms. In: Proceedings of IEEE International Conference on Pattern Recognition, pp. 701–706 (2002)Google Scholar
  55. 55.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vision 42(3), 145–175 (2001)CrossRefMATHGoogle Scholar
  56. 56.
    De Brabanter, K., et al.: LS-SVMlab toolbox user’s guide version 1.8. Internal Report, ESAT-SISTA, K.U. Leuven, Leuven, Belgium, pp. 10–14 (2010)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.National Engineering School of Sousse- ENISoUniversity of SousseSousseTunisia
  2. 2.High Institute of Applied Technologies of KairouanUniversity of KairouanKairouanTunisia

Personalised recommendations