Multimedia Tools and Applications

, Volume 70, Issue 1, pp 543–556 | Cite as

A new benchmark image test suite for evaluating colour texture classification schemes

  • A. PorebskiEmail author
  • N. Vandenbroucke
  • L. Macaire
  • D. Hamad


Several image test suites are available in the literature to evaluate the performance of classification schemes. In the framework of colour texture classification, OuTex-TC-00013 (OuTex) and Contrib-TC-00006 (VisTex) are often used. These colour texture image sets have allowed the accuracies reached by many classification schemes to be compared. However, by analysing the classification results obtained with these two sets of colour texture images, we have noticed that the use of colour histogram yields a higher rate of well-classified images compared to colour texture features. It does not take into account any texture information in the image, this incoherence leads us to question the relevance of these two benchmark colour texture sets for measuring the performances of colour texture classification algorithms. Indeed, the partitioning used to build these two sets consists of extracting training and validating sub-images of an original image. We show that such partitioning leads to biased classification results when it is combined with a classifier such as the nearest neighbour. In this paper a new relevant image test suite is proposed for evaluating colour texture classification schemes. The training and the validating sub-images come from different original images in order to ensure that the correlation of the colour texture images is minimized.


Benchmark colour texture test suite Supervised classification OuTex VisTex 


  1. 1.
    Aptoula E, Lefzèvre S (2007) A comparative study on multivariate mathematical morphology. Pattern Recogn 40(11):2914–2929CrossRefzbMATHGoogle Scholar
  2. 2.
    Arvis V, Debain C, Berducat M, Benassi A (2004) Generalization of the cooccurrence matrix for colour images: application to colour texture classification. Image Anal Stereol 23:63–72CrossRefGoogle Scholar
  3. 3.
    Dana KJ, Ginneken BV, Nayar SK, Koenderink JJ (1997) Reflectance and texture of real World surfaces. In: Proceedings of IEEE conference on Computer Vision and Pattern Recognition (CVPR). San Juan, Puerto Rico, pp 151–157Google Scholar
  4. 4.
    Drimbarean A, Whelan PF (2001) Experiments in colour texture analysis. Pattern Recogn Lett 22(10):1161–1167CrossRefzbMATHGoogle Scholar
  5. 5.
    Hable R (2013) Universal consistency of localized versions of regularized kernel methods. J Mach Learn Res 14:153–186zbMATHMathSciNetGoogle Scholar
  6. 6.
    Hernandez OJ, Cook J, Griffin M, De Rama C, McGovern M (2005) Classification of color textures with random field models and neural networks. J Comput Sci Technol 5(3):150–157Google Scholar
  7. 7.
    Hiremath PS, Shivashankar S, Pujari J (2006) Wavelet based features for color texture classification with application to CBIR. International Journal of Computer Science and Network Security (IJCSNS) 6(9):124–133Google Scholar
  8. 8.
    Iakovidis D, Maroulis D, Karkanis S (2005) A comparative study of color-texture image features. In: Proceedings of the 12th International Workshop on Systems, Signals & Image Processing (IWSSIP’05). Chalkida, Greece, pp 203–207Google Scholar
  9. 9.
    Khotanzad A, Hernandez OJ (2006) A classification methodology for color textures using multispectral random field mathematical models. Math Comput Appl 11(2):111–120zbMATHGoogle Scholar
  10. 10.
    Lakmann R (1998) Barktex benchmark database of color textured images. Koblenz-Landau University,
  11. 11.
    Mäenpää T, Pietikäinen M (2004) Classification with color and texture: jointly or separately? Pattern Recogn Lett 37(8):1629–1640CrossRefGoogle Scholar
  12. 12.
    Münzenmayer C, Wilharm S, Hornegger J, Wittenberg T (2005) Illumination invariant color texture analysis based on sum- and difference-histograms. In: Proceedings of the DAGM-Symposium. Editions Springer-Verlag, pp 17–24.Google Scholar
  13. 13.
    Münzenmayer C, Volk H, Küblbeck C, Spinnler K, Wittenberg T (2002) Multispectral texture analysis using interplane sum- and difference-histograms. In: Proceedings of the DAGM-Symposium. Editions Springer-Verlag, pp 42–49Google Scholar
  14. 14.
    Ojala T, Mäenpää T, Pietikäinen M, Viertola J, Kyllönen J, Huovinen S (2002) Outex new framework for empirical evaluation of texture analysis algorithms. In: Proceedings of the 16th International Conference on Pattern Recognition, vol 1. Quebec, Canada, pp 701–706Google Scholar
  15. 15.
    Palm C (2004) Color texture classification by integrative co-occurrence matrices. Pattern Recogn Lett 37(5):965–976CrossRefGoogle Scholar
  16. 16.
    Palm C, Lehmann TM (2002) Classification of color textures by gabor filtering. Mach Graph Vis 11(2):195–219Google Scholar
  17. 17.
    Picard R, Graczyk C, Mann S, Wachman J, Picard L, Campbell L (1995) Vision Texture Database. Media Laboratory, Massachusetts Institute of Technology (MIT), Cambridge
  18. 18.
    Pietikäinen M, Mäenpää T, Viertola J (2002) Color texture classification with color histograms and local binary patterns. In: Proceedings of the 2nd international workshop on texture analysis and synthesis, pp 109–112Google Scholar
  19. 19.
    Porebski A, Vandenbroucke N, Macaire L (2007) Iterative feature selection for color texture classification. In: Proceedings of the IEEE International Conference on Image Processing. San Antonio, USA, pp 509–512Google Scholar
  20. 20.
    Porebski A, Vandenbroucke N, Macaire L (2010) Comparison of feature selection schemes for color texture classification. In: Proceedings of the 2nd IEEE international Workshops on Image Processing Theory, Tools and Applications. Paris, France, pp 32–37Google Scholar
  21. 21.
    Porebski A, Vandenbroucke N, Macaire L (2013) Supervised texture classification: color space or texture feature selection? Pattern Anal and Appl 16(1):1–18.MathSciNetGoogle Scholar
  22. 22.
    Qazi IUH, Alata O, Burie JC, Moussa A, Fernandez-Maloigne C (2011) Choice of a pertinent color space for color texture characterization using parametric spectral analysis. Pattern Recogn 44(1):16–31CrossRefzbMATHGoogle Scholar
  23. 23.
    VanDen Broek EL, Van Rikxoort EM (2004) Evaluation of color representation for texture analysis. In: Proceedings of the Belgium-Dutch Conference on Artificial Intelligence, pp. 35–42. Groningen, The NetherlandsGoogle Scholar
  24. 24.
    Vandenbroucke N, Alata O, Lecomte C, Porebski A, Qazi I (2012) Color Texture Attributes, chap 6. Digital Color Imaging, ISTE Ltd/John Wiley & SonsGoogle Scholar
  25. 25.
    Van deWouwer G, Scheunders P, Livens S, Van Dyck D (1999) Wavelet correlation signatures for color texture characterization. Pattern Recogn 32:443–451CrossRefGoogle Scholar
  26. 26.
    Xu Q, Yang J, Ding S (2005) Color texture analysis using the wavelet-based hidden Markov model. Pattern Recogn Lett 26:1710–1719CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • A. Porebski
    • 1
    Email author
  • N. Vandenbroucke
    • 1
  • L. Macaire
    • 2
  • D. Hamad
    • 1
  1. 1.Laboratoire LISIC, EA 4491Université du Littoral Côte d’OpaleCalais CedexFrance
  2. 2.Laboratoire LAGIS, UMR CNRS 8219Université Lille 1, Sciences et Technologies, Cité ScientifiqueVilleneuve d’AscqFrance

Personalised recommendations