Skip to main content
SpringerLink
Log in

Adequately hierarchical patterns based on pairwise regions

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

We propose a novel operator, called CHILOP, for capturing the hierarchical relationships of two adjacent supporting regions in a completed local encoding context. It can be seen as a generalization of typical CLBP, one of the most popular local operators. CHILOP is then considered in a multi-scale approach to forcefully capture multi-hierarchical patterns with more robustness. Moreover, multi-hierarchical Gaussian-filtered CHILOP properties are taken into account for a more discriminative descriptor inspired by filter-bank approaches. A comprehensive evaluation on different benchmark datasets has proven the benefit of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The datasets used in this paper are publicly available.

Notes

  1. A code to structure CHILOP-based patterns can be picked up at http://tpnguyen.univ-tln.fr/download/MATCodeCHILOP.

  2. Based on the model of data transformation from scheme Dynamics to Appearance in [18], a simple MATLAB code for correspondingly transferring results of encoding descriptors is available at http://tpnguyen.univ-tln.fr/download/MATCodeCHILOP. This code allows saving the performing time for future works because just encoding DT videos on Dynamics is taken into account, while on Appearance, it can be transferred from the obtained results.

References

  1. Kim, W., Moon, S., Lee, J.W., Nam, D., Jung, C.: Multiple player tracking in soccer videos: an adaptive multiscale sampling approach. Multimed. Syst. 24(6), 611–623 (2018)

    Google Scholar 

  2. Fang, M., Bai, X., Zhao, J., Yang, F., Hung, C., Liu, S.: Integrating Gaussian mixture model and dilated residual network for action recognition in videos. Multimed. Syst. 26(6), 715–725 (2020)

    Google Scholar 

  3. Kim, W.: Moving object detection using edges of residuals under varying illuminations. Multimed. Syst. 25(3), 155–163 (2019)

    Google Scholar 

  4. Barmpoutis, P., Dimitropoulos, K., Grammalidis, N.: Smoke detection using spatio-temporal analysis, motion modeling and dynamic texture recognition. In: 22nd European Signal Processing Conference, EUSIPCO 2014, Lisbon, Portugal, September 1–5, 2014, pp. 1078–1082 (2014)

  5. Nguyen, T.P., Manzanera, A., Garrigues, M., Vu, N.S.: Spatial motion patterns: action models from semi-dense trajectories. IJPRAI 28(7), 1460011 (2014)

    Google Scholar 

  6. Khan, F.A., Nawaz, M., Imran, M., Rahman, A.U., Qayum, F.: Foreground detection using motion histogram threshold algorithm in high-resolution large datasets. Multimed. Syst. 27(4), 667–678 (2021)

    Google Scholar 

  7. Dehghan, A., Shah, M.: Binary quadratic programing for online tracking of hundreds of people in extremely crowded scenes. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 568–581 (2018)

    PubMed  Google Scholar 

  8. Ullah, H., Islam, I.U., Ullah, M., Afaq, M., Khan, S.D., Iqbal, J.: Multi-feature-based crowd video modeling for visual event detection. Multimed. Syst. 27(4), 589–597 (2021)

    Google Scholar 

  9. Kushwaha, A.K.S., Srivastava, S., Srivastava, R.: Multi-view human activity recognition based on silhouette and uniform rotation invariant local binary patterns. Multimed. Syst. 23(4), 451–467 (2017)

    Google Scholar 

  10. Nguyen, X.S., Nguyen, T.P., Charpillet, F., Vu, N.S.: Local derivative pattern for action recognition in depth images. Multimed. Tools Appl. 77(7), 8531–8549 (2018)

    Google Scholar 

  11. Gavrilescu, M.: Recognizing human gestures in videos by modeling the mutual context of body position and hands movement. Multimed. Syst. 23(3), 381–393 (2017)

    Google Scholar 

  12. Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. IEEE Trans. IP 19(6), 1657–1663 (2010)

    MathSciNet  Google Scholar 

  13. Liu, L., Zhao, L., Long, Y., Kuang, G., Fieguth, P.W.: Extended local binary patterns for texture classification. Image Vis. Comput. 30(2), 86–99 (2012)

    Google Scholar 

  14. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Dynamic texture representation based on hierarchical local patterns. In: 20th International Conference on Advanced Concepts for Intelligent Vision Systems, ACIVS 2020, Auckland, New Zealand, February 10–14, 2020, pp. 277–289 (2020)

  15. Saisan, P., Doretto, G., Wu, Y.N., Soatto, S.: Dynamic texture recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2001, 8–14 December 2001, Kauai, HI, USA, pp 58–63 (2001)

  16. Péteri, R., Fazekas, S., Huiskes, M.J.: DynTex: a comprehensive database of dynamic textures. Pattern Recognit. Lett. 31(12), 1627–1632 (2010)

    ADS  Google Scholar 

  17. Ghanem, B., Ahuja, N.: Maximum margin distance learning for dynamic texture recognition. In: 11th European Conference on Computer Vision, ECCV 2010, Heraklion, Crete, Greece, September 5–11, 2010, vol. 6312, pp. 223–236 (2010)

  18. Hadji, I., Wildes, R.P.: A new large scale dynamic texture dataset with application to ConvNet understanding. In: 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany, September 8–14, 2018, pp. 334–351 (2018)

  19. Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. PAMI 24(7), 971–987 (2002)

    Google Scholar 

  20. Nguyen, T.P., Manzanera, A., Kropatsch, W.G., N’Guyen, X.S.: Topological attribute patterns for texture recognition. Pattern Recognit. Lett. 80, 91–97 (2016)

    ADS  Google Scholar 

  21. Zhao, Y., Huang, D.S., Jia, W.: Completed local binary count for rotation invariant texture classification. IEEE Trans. IP 21(10), 4492–4497 (2012)

    MathSciNet  Google Scholar 

  22. Jain, A.K., Farrokhnia, F.: Unsupervised texture segmentation using Gabor filters. Pattern Recognit. 24(12), 1167–1186 (1991)

    ADS  Google Scholar 

  23. Nguyen, T.P., Vu, N.S., Manzanera, A.: Statistical binary patterns for rotational invariant texture classification. Neurocomputing 173, 1565–1577 (2016)

    Google Scholar 

  24. Derpanis, K.G., Wildes, R.P.: Spacetime texture representation and recognition based on a spatiotemporal orientation analysis. IEEE Trans. PAMI 34(6), 1193–1205 (2012)

    Google Scholar 

  25. Arashloo, S.R., Kittler, J.: Dynamic texture recognition using multiscale binarized statistical image features. IEEE Trans. Multimed. 16(8), 2099–2109 (2014)

    Google Scholar 

  26. Jansson, Y., Lindeberg, T.: Dynamic texture recognition using time-causal and time-recursive spatio-temporal receptive fields. J. Math. Imaging Vis. 60(9), 1369–1398 (2018)

    MathSciNet  Google Scholar 

  27. Zhao, X., Lin, Y., Liu, L., Heikkilä, J., Zheng, W.: Dynamic texture classification using unsupervised 3D filter learning and local binary encoding. IEEE Trans. Multimed. 21(7), 1694–1708 (2019)

    Google Scholar 

  28. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Smooth-invariant Gaussian features for dynamic texture recognition. In: IEEE International Conference on Image Processing, ICIP 2019, Taipei, Taiwan, September 22–25, 2019, pp. 4400–4404 (2019)

  29. Nguyen, T.T., Nguyen, T.P., Bouchara, F., Vu, N.: Volumes of blurred-invariant gaussians for dynamic texture classification. In: 18th International Conference on Computer Analysis of Images and Patterns, CAIP 2019, Salerno, Italy, September 3–5, 2019, pp. 155–167 (2019)

  30. Chan, B.A.B., Vasconcelos, N.: Classifying video with kernel dynamic textures. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2007, 18–23 June 2007, Minneapolis, Minnesota, USA, pp. 1–6 (2007)

  31. Mumtaz, A., Coviello, E., Lanckriet, G.R.G., Chan, A.B.: Clustering dynamic textures with the hierarchical EM algorithm for modeling video. IEEE Trans. PAMI 35(7), 1606–1621 (2013)

    Google Scholar 

  32. Wang, Y., Hu, S.: Chaotic features for dynamic textures recognition. Soft Comput. 20(5), 1977–1989 (2016)

    Google Scholar 

  33. Ravichandran, A., Chaudhry, R., Vidal, R.: View-invariant dynamic texture recognition using a bag of dynamical systems. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2009, 20–25 June 2009, Miami, Florida, USA, pp. 1651–1657 (2009)

  34. Mumtaz, A., Coviello, E., Lanckriet, G.R.G., Chan, A.B.: A scalable and accurate descriptor for dynamic textures using bag of system trees. IEEE Trans. PAMI 37(4), 697–712 (2015)

    Google Scholar 

  35. Qi, X., Li, C.G., Zhao, G., Hong, X., Pietikainen, M.: Dynamic texture and scene classification by transferring deep image features. Neurocomputing 171, 1230–1241 (2016)

    Google Scholar 

  36. Andrearczyk, V., Whelan, P.F.: Convolutional neural network on three orthogonal planes for dynamic texture classification. Pattern Recognit. 76, 36–49 (2018)

    ADS  Google Scholar 

  37. Arashloo, S.R., Amirani, M.C., Noroozi, A.: Dynamic texture representation using a deep multi-scale convolutional network. J. Vis. Commun. Image Represent. 43, 89–97 (2017)

    Google Scholar 

  38. Hong, S., Ryu, J., Im, W., Yang, H.S.: D3: recognizing dynamic scenes with deep dual descriptor based on key frames and key segments. Neurocomputing 273, 611–621 (2018)

    Google Scholar 

  39. Quan, Y., Huang, Y., Ji, H.: Dynamic texture recognition via orthogonal tensor dictionary learning. In: IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7–13, 2015, pp. 73–81 (2015)

  40. Quan, Y., Bao, C., Ji, H.: Equiangular kernel dictionary learning with applications to dynamic texture analysis. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27–30, 2016, pp. 308–316 (2016)

  41. Xu, Y., Quan, Y., Ling, H., Ji, H.: Dynamic texture classification using dynamic fractal analysis. In: IEEE International Conference on Computer Vision, ICCV 2011, Barcelona, Spain, November 6–13, 2011, pp. 1219–1226 (2011)

  42. Xu, Y., Huang, S.B., Ji, H., Fermüller, C.: Scale-space texture description on SIFT-like textons. CVIU 116(9), 999–1013 (2012)

    Google Scholar 

  43. Ji, H., Yang, X., Ling, H., Xu, Y.: Wavelet domain multifractal analysis for static and dynamic texture classification. IEEE Trans. IP 22(1), 286–299 (2013)

    MathSciNet  Google Scholar 

  44. Baktashmotlagh, M., Harandi, M.T.A., Lovell, B.C., Salzmann, M.: Discriminative non-linear stationary subspace analysis for video classification. IEEE Trans. PAMI 36(12), 2353–2366 (2014)

    Google Scholar 

  45. Péteri, R., Chetverikov, D.: Dynamic texture recognition using normal flow and texture regularity. In: Second Iberian Conference on Pattern Recognition and Image Analysis, IbPRIA 2005, Estoril, Portugal, June 7–9, 2005, pp. 223–230 (2005)

  46. Péteri, R., Chetverikov, D.: Qualitative characterization of dynamic textures for video retrieval. In: International Conference on Computer Vision and Graphics, ICCVG 2004, Warsaw, Poland, September 2004, pp. 33–38 (2004)

  47. Peh, C.H., Cheong, L.F.: Synergizing spatial and temporal texture. IEEE Trans. IP 11(10), 1179–1191 (2002)

    MathSciNet  Google Scholar 

  48. Nguyen, T.T., Nguyen, T.P., Bouchara, F., Nguyen, X.S.: Directional beams of dense trajectories for dynamic texture recognition. In: 19th International Conference on Advanced Concepts for Intelligent Vision Systems, ACIVS 2018, Poitiers, France, September 24–27, 2018, pp. 74–86 (2018)

  49. Lu, Z., Xie, W., Pei, J., Huang, J.: Dynamic texture recognition by spatio-temporal multiresolution histograms. In: 7th IEEE Workshop on Applications of Computer Vision / IEEE Workshop on Motion and Video Computing (WACV/MOTION 2005), 5-7 January 2005, Breckenridge, CO, USA, pp. 241–246 (2005)

  50. Rivera, A.R., Chae, O.: Spatiotemporal directional number transitional graph for dynamic texture recognition. IEEE Trans. PAMI 37(10), 2146–2152 (2015)

    Google Scholar 

  51. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Completed statistical adaptive patterns on three orthogonal planes for recognition of dynamic textures and scenes. J. Electron. Imaging 27(05), 053044 (2018)

    ADS  Google Scholar 

  52. Zhao, G., Pietikäinen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. PAMI 29(6), 915–928 (2007)

    Google Scholar 

  53. Tiwari, D., Tyagi, V.: Dynamic texture recognition based on completed volume local binary pattern. MSSP 27(2), 563–575 (2016)

    Google Scholar 

  54. Tiwari, D., Tyagi, V.: A novel scheme based on local binary pattern for dynamic texture recognition. CVIU 150, 58–65 (2016)

    Google Scholar 

  55. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Completed local structure patterns on three orthogonal planes for dynamic texture recognition. In: Seventh International Conference on Image Processing Theory, Tools and Applications, IPTA 2017, Montreal, QC, Canada, November 28 - December 1, 2017, pp. 1–6 (2017)

  56. Mäenpää, T., Pietikäinen, M.: Multi-scale binary patterns for texture analysis. In: 13th Scandinavian Conference, SCIA 2003, Halmstad, Sweden, June 29 - July 2, 2003, pp. 885–892 (2003)

  57. Tiwari, D., Tyagi, V.: Improved Weber’s law based local binary pattern for dynamic texture recognition. Multimed. Tools Appl. 76(5), 6623–6640 (2017)

    Google Scholar 

  58. Xu, Y., Quan, Y., Zhang, Z., Ling, H., Ji, H.: Classifying dynamic textures via spatiotemporal fractal analysis. Pattern Recognit. 48(10), 3239–3248 (2015)

    ADS  Google Scholar 

  59. Dubois, S., Péteri, R., Ménard, M.: Characterization and recognition of dynamic textures based on the 2D+T curvelet transform. Signal Image Video Process. 9(4), 819–830 (2015)

    Google Scholar 

  60. Fan, R., Chang, K., Hsieh, C., Wang, X., Lin, C.: LIBLINEAR: a library for large linear classification. JMLR 9, 1871–1874 (2008)

    Google Scholar 

  61. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Directional dense-trajectory-based patterns for dynamic texture recognition. IET Comput. Vis. 14(4), 162–176 (2020)

    Google Scholar 

  62. Quan, Y., Sun, Y., Xu, Y.: Spatiotemporal lacunarity spectrum for dynamic texture classification. CVIU 165, 85–96 (2017)

    Google Scholar 

  63. Tiwari, D., Tyagi, V.: Dynamic texture recognition using multiresolution edge-weighted local structure pattern. Comput. Electr. Eng. 62, 485–498 (2017)

    Google Scholar 

  64. Zhao, X., Lin, Y., Heikkilä, J.: Dynamic texture recognition using volume local binary count patterns with an application to 2D face spoofing detection. IEEE Trans Multimed. 20(3), 552–566 (2018)

    Google Scholar 

  65. Nguyen, T.T., Nguyen, T.P., Bouchara, F., Nguyen, X.S.: Momental directional patterns for dynamic texture recognition. CVIU 194, 102882 (2020)

    Google Scholar 

  66. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Rubik Gaussian-based patterns for dynamic texture classification. Pattern Recognit. Lett. 135, 180–187 (2020)

    ADS  Google Scholar 

  67. Ren, J., Jiang, X., Yuan, J.: Dynamic texture recognition using enhanced LBP features. In: ICASSP, pp. 2400–2404 (2013)

  68. Ren, J., Jiang, X., Yuan, J., Wang, G.: Optimizing LBP structure for visual recognition using binary quadratic programming. IEEE Signal Process. Lett. 21(11), 1346–1350 (2014)

    ADS  Google Scholar 

  69. Hadji, I., Wildes, R.P.: A spatiotemporal oriented energy network for dynamic texture recognition. In: IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22–29, 2017, pp. 3085–3093 (2017)

  70. Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8–13 2014, Montreal, Quebec, Canada, pp. 568–576 (2014)

  71. Tran, D., Bourdev, L.D., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7–13, 2015, pp. 4489–4497 (2015)

Download references

Acknowledgements

We would like to express our sincere appreciation to HCMC University of Technology and Education, Faculty of IT, Thu Duc City, Ho Chi Minh City, Vietnam, who gave us crucial supports in high-performing computer systems for the experiments on the large-scale datasets. The work of Thanh Phuong Nguyen is partially supported by ANR ROV-Chasseur.

Author information

Authors and Affiliations

  1. Université de Toulon, Aix Marseille Univ, CNRS, LIS, Marseille, France

    Thanh Tuan Nguyen, Thanh Phuong Nguyen & Frédéric Bouchara

  2. HCMC University of Technology and Education, Faculty of IT, Thu Duc City, Ho Chi Minh City, Vietnam

    Thanh Tuan Nguyen

Authors
  1. Thanh Tuan Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thanh Phuong Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frédéric Bouchara
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Thanh Tuan Nguyen and Thanh Phuong Nguyen developed the theoretical formalism, performed the analytic calculations, performed the numerical simulations, designed and performed the experiments, and wrote the main manuscript text. All authors contributed to the final version of the manuscript.

Corresponding author

Correspondence to Thanh Tuan Nguyen.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article

Additional information

Communicated by J. Gao.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen, T.T., Nguyen, T.P. & Bouchara, F. Adequately hierarchical patterns based on pairwise regions. Multimedia Systems 30, 45 (2024). https://doi.org/10.1007/s00530-023-01217-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-023-01217-4

Keywords

Search

Navigation