Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Local jet pattern: a robust descriptor for texture classification

Abstract

Methods based on locally encoded image features have recently become popular for texture classification tasks, particularly in the existence of large intra-class variation due to changes in illumination, scale, and viewpoint. Inspired by the theories of image structure analysis, this work proposes an efficient, simple, yet robust descriptor namely local jet pattern (Ljp) for texture classification. In this approach, a jet space representation of a texture image is computed from a set of derivatives of Gaussian (DtGs) filter responses up to second order, so-called local jet vectors (Ljv), which also satisfy the Scale Space properties. The Ljp is obtained by using the relation of center pixel with its’ local neighborhoods in jet space. Finally, the feature vector of a texture image is formed by concatenating the histogram of Ljp for all elements of Ljv. All DtGs responses up to second order together preserves the intrinsic local image structure, and achieves invariance to scale, rotation, and reflection. This allows us to design a discriminative and robust framework for texture classification. Extensive experiments on five standard texture image databases, employing nearest subspace classifier (Nsc), the proposed descriptor achieves 100%, 99.92%, 99.75%, 99.16%, and 99.65% accuracy for Outex_TC10, Outex_TC12, KTH-TIPS, Brodatz, CUReT, respectively, which are better compared to state-of-the-art methods.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

References

  1. 1.

    Andrearczyk V, Whelan PF (2016) Using filter banks in convolutional neural networks for texture classification. Pattern Recogn Lett 84:63–69

  2. 2.

    Brodatz P (1966) Textures: a photographic album for artists and designers. Dover Pubns, New York

  3. 3.

    Chan TH, Jia K, Gao S, Lu J, Zeng Z, Ma Y (2015) Pcanet: a simple deep learning baseline for image classification? IEEE Trans Image Process 24 (12):5017–5032

  4. 4.

    Chang X, Ma Z, Lin M, Yang Y, Hauptmann AG (2017) Feature interaction augmented sparse learning for fast kinect motion detection. IEEE Trans Image Process 26 (8):3911–3920. https://doi.org/10.1109/TIP.2017.2708506

  5. 5.

    Chang X, Ma Z, Yang Y, Zeng Z, Hauptmann AG (2017) Bi-level semantic representation analysis for multimedia event detection. IEEE Transactions on Cybernetics 47(5):1180–1197. https://doi.org/10.1109/TCYB.2016.2539546

  6. 6.

    Chang X, Yu YL, Yang Y, Xing EP (2017) Semantic pooling for complex event analysis in untrimmed videos. IEEE Trans Pattern Anal Mach Intell 39 (8):1617–1632. https://doi.org/10.1109/TPAMI.2016.2608901

  7. 7.

    Chaudhuri BB, Sarkar N (1995) Texture segmentation using fractal dimension. IEEE Trans Pattern Anal Mach Intell 17(1):72–77

  8. 8.

    Chen JL, Kundu A (1994) Rotation and gray scale transform invariant texture identification using wavelet decomposition and hidden Markov model. IEEE Trans Pattern Anal Mach Intell 16(2):208–214

  9. 9.

    Cimpoi M, Maji S, Kokkinos I, Vedaldi A (2016) Deep filter banks for texture recognition, description, and segmentation. Int J Comput Vis 118(1):65–94

  10. 10.

    Cimpoi M, Maji S, Vedaldi A (2015) Deep filter banks for texture recognition and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3828–3836

  11. 11.

    Crosier M, Griffin LD (2010) Using basic image features for texture classification. Int J Comput Vis 88(3):447–460

  12. 12.

    Dana KJ, Van Ginneken B, Nayar SK, Koenderink JJ (1999) Reflectance and texture of real-world surfaces. ACM Transactions On Graphics (TOG) 18(1):1–34

  13. 13.

    Deng H, Clausi DA (2004) Gaussian MRF rotation-invariant features for image classification. IEEE Trans Pattern Anal Mach Intell 26(7):951–955

  14. 14.

    Dubey SR, Singh SK, Singh RK (2014) Rotation and illumination invariant interleaved intensity order-based local descriptor. IEEE Trans Image Process 23(12):5323–5333

  15. 15.

    Dubey SR, Singh SK, Singh RK (2015) Local wavelet pattern: a new feature descriptor for image retrieval in medical ct databases. IEEE Trans Image Process 24 (12):5892–5903

  16. 16.

    Dubey SR, Singh SK, Singh RK (2015) A multi-channel based illumination compensation mechanism for brightness invariant image retrieval. Multimedia Tools and Applications 74(24):11,223–11,253

  17. 17.

    Dubey SR, Singh SK, Singh RK (2016) Multichannel decoded local binary patterns for content-based image retrieval. IEEE Trans Image Process 25(9):4018–4032

  18. 18.

    Dubey SR, Singh SK, Singh RK (2017) Local svd based nir face retrieval. J Vis Commun Image Represent 49:141–152

  19. 19.

    Fathi A, Naghsh-Nilchi AR (2012) Noise tolerant local binary pattern operator for efficient texture analysis. Pattern Recogn Lett 33(9):1093–1100

  20. 20.

    Florack L, Romeny BTH, Viergever M, Koenderink J (1996) The Gaussian scale-space paradigm and the multiscale local jet. Int J Comput Vis 18(1):61–75

  21. 21.

    Guo Z, Wang X, Zhou J, You J (2016) Robust texture image representation by scale selective local binary patterns. IEEE Trans Image Process 25(2):687–699

  22. 22.

    Guo Z, Zhang L, Zhang D (2010) A completed modeling of local binary pattern operator for texture classification. IEEE Trans Image Process 19(6):1657–1663

  23. 23.

    Guo Z, Zhang L, Zhang D (2010) Rotation invariant texture classification using LBP variance (LBPV) with global matching. Pattern Recogn 43(3):706–719

  24. 24.

    Haralick RM, Shanmugam K, et al. (1973) Textural features for image classification. IEEE Transactions on systems, man, and cybernetics 3(6):610–621

  25. 25.

    Hayman E, Caputo B, Fritz M, Eklundh JO (2004) On the significance of real-world conditions for material classification. In: European conference on computer vision, Springer, pp 253–266

  26. 26.

    Heikkilä M, Pietikäinen M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern Recogn 42(3):425–436

  27. 27.

    Iversen GR, Norpoth H (1987) Analysis of variance. 1 Sage

  28. 28.

    Kashyap RL, Khotanzad A (1986) A model-based method for rotation invariant texture classification. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-8(4):472–481

  29. 29.

    Koenderink JJ, van Doorn AJ (1987) Representation of local geometry in the visual system. Biol Cybern 55(6):367–375

  30. 30.

    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  31. 31.

    Lazebnik S, Schmid C, Ponce J (2005) A sparse texture representation using local affine regions. IEEE Trans Pattern Anal Mach Intell 27(8):1265–1278

  32. 32.

    Li Z, Liu G, Yang Y, You J (2012) Scale-and rotation-invariant local binary pattern using scale-adaptive texton and subuniform-based circular shift. IEEE Trans Image Process 21(4):2130–2140

  33. 33.

    Li Z, Nie F, Chang X, Yang Y (2017) Beyond trace ratio: Weighted harmonic mean of trace ratios for multiclass discriminant analysis. IEEE Trans Knowl Data Eng 29(10):2100–2110. https://doi.org/10.1109/TKDE.2017.2728531

  34. 34.

    Liao S, Law MW, Chung AC (2009) Dominant local binary patterns for texture classification. IEEE Trans Image Process 18(5):1107–1118

  35. 35.

    Lindeberg T (1994) Scale-space theory: a basic tool for analyzing structures at different scales. J Appl Stat 21(1-2):225–270

  36. 36.

    Liu L, Fieguth P (2012) Texture classification from random features. IEEE Trans Pattern Anal Mach Intell 34(3):574–586

  37. 37.

    Liu L, Fieguth P, Wang X, Pietikäinen M., Hu D (2016) Evaluation of lbp and deep texture descriptors with a new robustness benchmark. In: European conference on computer vision, Springer, pp 69–86

  38. 38.

    Liu L, Long Y, Fieguth PW, Lao S, Zhao G (2014) Brint: binary rotation invariant and noise tolerant texture classification. IEEE Trans Image Process 23(7):3071–3084

  39. 39.

    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

  40. 40.

    Mao J, Jain AK (1992) Texture classification and segmentation using multiresolution simultaneous autoregressive models. Pattern Recogn 25(2):173–188

  41. 41.

    Martens JB (1990) The hermite transform-theory. IEEE Trans Acoust Speech Signal Process 38(9):1595–1606

  42. 42.

    Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans Pattern Anal Mach Intell 27(10):1615–1630

  43. 43.

    Ojala T, Maenpaa T, Pietikainen M, Viertola J, Kyllonen J, Huovinen S (2002) Outex-new framework for empirical evaluation of texture analysis algorithms. In: 16th international conference on Pattern recognition, 2002. Proceedings, vol 1, pp 701–706. IEEE

  44. 44.

    Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29(1):51–59

  45. 45.

    Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

  46. 46.

    Quan Y, Xu Y, Sun Y (2014) A distinct and compact texture descriptor. Image Vis Comput 32(4):250–259

  47. 47.

    Randen T, Husoy JH (1999) Filtering for texture classification: a comparative study. IEEE Trans Pattern Anal Mach Intell 21(4):291–310

  48. 48.

    Ren J, Jiang X, Yuan J (2013) Noise-resistant local binary pattern with an embedded error-correction mechanism. IEEE Trans Image Process 22(10):4049–4060

  49. 49.

    Roy SK, Chanda B, Chaudhuri BB, Banerjee S, Ghosh DK, Dubey SR (2018) Local directional zigzag pattern: a rotation invariant descriptor for texture classification. Pattern Recogn Lett 108:23–30

  50. 50.

    Roy SK, Chanda B, Chaudhuri BB, Ghosh DK, Dubey SR (2017) A complete Dual-Cross pattern for unconstrained texture classification. In: Proceedings of the 4th Asian conference on pattern recognition (ACPR 2017), Nanjing, China, pp 741–746. IEEE

  51. 51.

    Roy SK, Chanda B, Chaudhuri BB, Ghosh DK, Dubey SR (2018) Local morphological pattern: A scale space shape descriptor for texture classification. Digital Signal Processing, Elsevier (In Press)

  52. 52.

    Tan X, Triggs B (2007) Enhanced local texture feature sets for face recognition under difficult lighting conditions. In: International workshop on analysis and modeling of faces and gestures, pp 168–182. Springer

  53. 53.

    Umer S, Dhara BC, Chanda B (2017) A novel cancelable iris recognition system based on feature learning techniques. Inf Sci 406:102–118

  54. 54.

    Varma M, Garg R (2007) Locally invariant fractal features for statistical texture classification. In: 2007. ICCV 2007. IEEE 11th international conference on Computer vision, pp 1–8. IEEE

  55. 55.

    Varma M, Zisserman A (2005) A statistical approach to texture classification from single images. Int J Comput Vis 62(1-2):61–81

  56. 56.

    Varma M, Zisserman A (2009) A statistical approach to material classification using image patch exemplars. IEEE Trans Pattern Anal Mach Intell 31(11):2032–2047

  57. 57.

    Victor JD, Knight BW (1994) Simultaneously band and space limited functions in two dimensions, and receptive fields of visual neurons. In: Perspectives and problems in nolinear science, pp 375–419. Springer

  58. 58.

    Wang K, Bichot CE, Zhu C, Li B (2013) Pixel to patch sampling structure and local neighboring intensity relationship patterns for texture classification. IEEE Signal Processing Letters 20(9):853–856

  59. 59.

    WEISS I (1993) Geometric invariants and object recognition. Int J Comput Vis 10(3):207–231

  60. 60.

    Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31 (2):210–227

  61. 61.

    Xie L, Zhu L, Chen G (2016) Unsupervised multi-graph cross-modal hashing for large-scale multimedia retrieval. Multimedia Tools and Applications 75(15):9185–9204

  62. 62.

    Xie X, Mirmehdi M (2008) A Galaxy of Texture Features. In: Handbook of texture analysis, pp 375–406. World Scientific

  63. 63.

    Xu Y, Ji H, Fermuller C (2006) A projective invariant for textures. In: 2006 IEEE computer society conference on computer vision and pattern recognition, vol 2, pp 1932–1939. IEEE

  64. 64.

    Xu Y, Ji H, Fermüller C (2009) Viewpoint invariant texture description using fractal analysis. Int J Comput Vis 83(1):85–100

  65. 65.

    Yao CH, Chen SY (2003) Retrieval of translated, rotated and scaled color textures. Pattern Recogn 36(4):913–929

  66. 66.

    Young RA, Lesperance RM, Meyer WW (2001) The gaussian derivative model for spatial-temporal vision: i. cortical model. Spatial vision 14(3):261–319

  67. 67.

    Zand M, Doraisamy S, Halin AA, Mustaffa MR (2015) Texture classification and discrimination for region-based image retrieval. J Vis Commun Image Represent 26:305–316

  68. 68.

    Zhang B, Gao Y, Zhao S, Liu J (2010) Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor. IEEE Trans Image Process 19(2):533–544

  69. 69.

    Zhang J, Liang J, Zhao H (2013) Local energy pattern for texture classification using self-adaptive quantization thresholds. IEEE Trans Image Process 22(1):31–42

  70. 70.

    Zhang J, Marszałek M, Lazebnik S, Schmid C (2007) Local features and kernels for classification of texture and object categories: a comprehensive study. Int J Comput Vis 73(2):213–238

  71. 71.

    Zhu L, Jin H, Zheng R, Feng X (2014) Weighting scheme for image retrieval based on bag-of-visual-words. IET Image Process 8(9):509–518

  72. 72.

    Zhu L, Shen J, Liu X, Xie L, Nie L (2016) Learning compact visual representation with canonical views for robust mobile landmark search. In: International joint conferences on artificial intelligence

  73. 73.

    Zhu L, Shen J, Xie L (2015) Topic hypergraph hashing for mobile image retrieval. In: Proceedings of the 23rd ACM international conference on Multimedia, pp 843–846. ACM

Download references

Author information

Correspondence to Swalpa Kumar Roy.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Effects of similarity transforms

Appendix: Effects of similarity transforms

In computer vision problems, a group of transforms are typically chosen to analyse the effect of geometrical structure those are invariant with respect to scaling, translation, rotation and reflection of an image, a constant intensity addition, and multiplication of image intensity by a positive factor and their combinations. In order to cope with the translation effect, we choose the analyzing point as the origin of the co-ordinate system.

In the jet space, Gaussian derivatives are represented by Hermite polynomials multiplied with Gaussian window (in (1)). The details within the window is expanded over the basis of Hermite polynomials, therefore even if the scale (σ) is fixed, it provides a multi-scale hierarchical image structure for \(\mathscr{L}\)-jet [29]

Rotation of a surface is a more difficult problem in image analysis. Consider the rotation effect on jet about the origin with an angle \(\theta \). Here the zero order derivative term of the jet (J(0,0)) remains unaffected. The first order derivative terms are transformed as follows,

$$\left( \begin{array}{cc} J_{(1,0)}\\J_{(0,1)} \end{array}\right) \to \left( \begin{array}{cc} \cos\theta & \sin\theta\\ -\sin\theta & \cos\theta \end{array}\right) \left( \begin{array}{cc} J_{(0,1)}\\J_{(1,0)} \end{array}\right) $$

and the second order terms are transformed according to

$$\left( \begin{array}{ccc} J_{(2,0)}\\J_{(1,1)}\\J_{(0,2)} \end{array}\right) \to \frac{1}{2} \left( \begin{array}{ccc} 1 + b & 2c & 1 - b\\ -c & 2b & c\\1 - b & -2c & 1 + b \end{array}\right) \left( \begin{array}{ccc} J_{(0,2)}\\J_{(1,1)}\\J_{(2,0)} \end{array}\right) $$

where \(b = \cos 2\theta \) and \(c = \sin 2\theta \). So, to return the starting values, the first order derivative structure requires a full \(2\pi \) rotation, while the second order derivative structure returns after a rotation by \(\pi \).

In case of reflection (i.e. the effect of jet about the line \(y = x\)), the zero order term of the jet is not affected. However, the first order derivative term is transformed according to, \( J_{(1,0)} \longleftrightarrow J_{(0,1)}\) and the second order derivative term, according to \( J_{(2,0)} \longleftrightarrow J_{(0,2)}\). Considering all the DtGs responses together, the \(\mathscr{L}\)-jet (\(\mathscr{L}\) = 6) achieves invariance to scale, rotation, or reflection.

Image structure analysis may be appropriate if it considers that intensities are non-negative unconstrained real numbers, and also do not exceed a maximum value. Therefore, the image structure should not be affected by adding a constant \((\alpha )\) to all intensities as in case of uniform illumination change. When such changes arise, only the zero order derivative term is affected and it simply transforms according to, \(J_{(0,0)} \to J_{(0,0)} + \alpha \).

It is also required that the image local structure should be invariant to the multiplication by a non-zero positive factor (𝜖 > 0) with all image intensities. In this case, the factor is simply multiplied with all terms of jet vector i.e, \(\mathbf {J} \to \epsilon \mathbf {J}\). Note that, it does not require multiplication of intensities by a negative factor to make it invariant to physical constrains.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Roy, S.K., Chanda, B., Chaudhuri, B.B. et al. Local jet pattern: a robust descriptor for texture classification. Multimed Tools Appl (2018). https://doi.org/10.1007/s11042-018-6559-3

Download citation

Keywords

  • Derivative-of-Gaussian (DtGs)
  • Jet space
  • Local jet vector (Ljv)
  • Local jet pattern (Ljp)
  • Texture classification