Significance of processing chrominance information for scene classification: a review

Abstract

The primary objective of this paper is to provide a detailed review of various works showing the role of processing chrominance information for color-to-grayscale conversion. The usefulness of perceptually improved color-to-grayscale converted images for scene classification is then studied as a part of this presented work. Various issues identified for the color-to-grayscale conversion and improved scene classification are presented in this paper. The review provided in this paper includes, review on existing feature extraction techniques for scene classification, various existing scene classification systems, different methods available in the literature for color-to-grayscale image conversion, benchmark datasets for scene classification and color-to-gray-scale image conversion, subjective evaluation and objective quality assessments for image decolorization. In the present work, a scene classification system is proposed using the pre-trained convolutional neural network and Support Vector Machines developed utilizing the grayscale images converted by the image decolorization methods. The experimental analysis on Oliva Torralba scene dataset shows that the color-to-grayscale image conversion technique has a positive impact on the performance of scene classification systems.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Notes

  1. 1.

    The terms, image classification and scene classification are interchangeably used in the context of the works presented in this paper.

  2. 2.

    This method is represented as ‘rgb2gray’ throughout the report.

References

  1. Akbas E, Ahuja N (2010) Low-level image segmentation based scene classification. In: Proc. IEEE int. conf. on pattern recognition (ICPR), pp 3623–3626

  2. Alsam A, Drew MS (2009) Fast multispectral2gray. J Imaging Sci Technol 53(6):1–19

    Google Scholar 

  3. Ancuti CO, Ancuti C, Bekaert P (2011) Enhancing by saliency-guided decolorization. In: Proc. IEEE conf. computer vision and pattern recognition (CVPR), pp 257 – 264

  4. Bala R, Eschbach R (2004) Spatial color-to-grayscale transform preserving chrominance edge information. In: Proc. IEEE int. conf. on pattern recognition (ICPR), pp 82–86

  5. Bay H, Tuytelaars T, Gool LV (2008) Speeded-up robust features (surf). Pattern Recognit 110(3):346–359

    Google Scholar 

  6. Bosch A, Munoz X, Zisserman A (2008) Scene classification using a hybrid generative/discriminative approach. IEEE Trans Pattern Anal Mach Intell (PAMI) 30(4):712–27

    Google Scholar 

  7. Bosch A, Zisserman A, Munoz X (2006) Scene classification via plsa. In: Proc. ECCV, LNCS, pp 517–530

  8. Cadik M (2008) Perceptual evaluation of color-to-grayscale image conversions. Comput. Graphics Forum 27(7):1745–1754

    Google Scholar 

  9. Dixit M, Rasiwasia N, Vasconcelos N (2011) Adapted gaussian models for image classification. In: Proc. int. conf. on computer vision and pattern recognition (CVPR), pp 937–943

  10. Dong G, Xie M (2005) Color clustering and learning for iamge segmentation based on neural networks. IEEE Trans Neural Netw 16(1):925–936

    Google Scholar 

  11. Douglas R, Thomas Q, Robert D (2000) Speaker verification using adapted gaussian mixture models. Int J Digital Signal Process 10(1):19–41

    Google Scholar 

  12. Faroudja YC (1988) NTSC and beyond. IEEE Trans Consum Electron 34(1):166–178

    Google Scholar 

  13. Gooch AA, Olsen SC, Tumblin J, Gooch B (2005) Color2gray: salience-preserving color removal. ACM Trans Graphics (TOG) 24(3):634–639

    Google Scholar 

  14. Grauman K, Darrell T (2005) Pyrmaid match kernels: discriminative classification with sets of image features. In: Proc. IEEE int. conf. on computer vision (ICCV), pp 1–8

  15. Grundland M, Dodgson NA (2007) Decolorize: fast, contrast enhancing, color to grayscale conversion. Int J Pattern Recognit 40(11):2891–2896

    Google Scholar 

  16. Gunes A, Kalkan H, Durmus E (2016) Optimizing the color-to-grayscale conversion for image classification. Int J Signal Image Video Process 10(5):853–860. https://doi.org/10.1007/s11760-015-0828-7

    Article  Google Scholar 

  17. Guo Z, Zhang D (2010) A completed modeling of local binary pattern operator for texture classification. IEEE Trans Image Process 19(6):1657–1663

    MathSciNet  MATH  Google Scholar 

  18. Hadjidemetriou E, Grossberg MD, Nayar SK (2004) Multiresolution histograms and their use in recognition. IEEE Trans Pattern Anal Mach Intell (PAMI) 26(7):831–847

    Google Scholar 

  19. Horiuchi T, Nohara F, Tominaga S (2010) Accurate reversible color-to-gray mapping algorithm without distortion conditions. Pattern Recognit Lett 31(1):2405–2414

    Google Scholar 

  20. Hua X (2012) Human computer interactions for converting color images to gray. Int J Neurocomputing 85(1):1–5

    Google Scholar 

  21. Ionescu RT, Popescu M (2015) Have a snak. encoding spatial information with the spatial non-alignment kernel. In: Int. conf. on image analysis and processing (ICIAP), pp 97–108

    Google Scholar 

  22. Ionescu RT, Popescu AL, Popescu D (2015) Texture classification with patch autocorrelation features. In: Proc. int.conf. on neural information processing (ICONIP), pp 1–11

    Google Scholar 

  23. Ionescu RT, Ionescu AL, Mothe J, Popescu D (2018) Patch autocorrelation features: a translation and rotation invariant approach for image classification. Artif Intell Rev 49(4):549–580

    Google Scholar 

  24. Ji Z, Fang M, Wang Y, Ma W (2016) Efficient decolorization preserving dominant distinctions. Visual Comput 32(12):1621–1631

    Google Scholar 

  25. Kadir T, Brady M (2001) Scale, saliency and image description. Int J Comput Vis (IJCV) 45(2):83–105

    MATH  Google Scholar 

  26. Kanan C, Cottrell G (2012) Color-to-grayscale: Does the method matter in image recognition? PLoS ONE 7(1):1–7

    Google Scholar 

  27. Kede M, Tiesong Z, Kai Z, Zhou W (2015) Objective quality assessment for color-to-gray image conversion. IEEE Trans Image Process 24(12):4673–4685

    MathSciNet  MATH  Google Scholar 

  28. Koenderink J, Doorn AV (1999) The structure of locally orderless images. Int J Comput Vis (IJCV) 31(2):159–168

    Google Scholar 

  29. Krapac J, Verbeek J, Jurie F (2011) Modeling spatial layout with fisher vectors for image categorization. In: Proc. IEEE int. conf. on computer vision (ICCV), pp 1487–1494

  30. Li FF, Pietro P (2005) A bayesian hierarchical model for learning natural scene categories. In: Proc. int. conf. on computer vision and pattern recognition, (CVPR), pp 524–531

  31. Li Z, Liu G, Yang Y, You J (2012) Scale- and rotation-invariant local binary pattern using scale-adaptive texton and subuniform-based circular shift. IEEE Trans Image Process 21(4):2130–2140

    MathSciNet  MATH  Google Scholar 

  32. Lim WH, Isa NAM (2011) Color to grayscale conversion based on neighborhood pixels effect approach for digital image. In: Proc. int. conf. on electrical and electronics engineering, pp 157–161

  33. Lissner I (2013) Image-difference prediction: from grayscale to color. IEEE Trans on Image Process 22(6):435–446

    MathSciNet  MATH  Google Scholar 

  34. Liu CW, Liu TL (2013) A sparse linear model for saliency-guided decolorization. In: Proc. twentieth IEEE int. conf. image processing (ICIP), pp 1105 – 1109

  35. Liu Q, Xiong J, ZhuMinghui L, Wang Z (2017) Extended rgb2gray conversion model for efficient contrast preserving decolorization. Multimed Tools Appl 76(12):14055–14074

    Google Scholar 

  36. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis (IJCV) 60(2):91–110

    Google Scholar 

  37. Lu C, Li X, Jia J (2012a) Real-time contrast preserving decolorization. In: Proc. int. conf. comput. graphics and interactive techniques (SIGGRAPH), pp 34:1–34:4

  38. Lu C, Xu L, Jia J (2012b) Contrast preserving decolorization. In: Proc. IEEE int. conf. computational photography (ICCP), pp 1–7

  39. Lu C, Xu L, Jia J (2014) Contrast preserving decolorization with perception-based quality metrics. Int J Comput Vis 110(2):222–239

    Google Scholar 

  40. Mantiuk R, Myszkowskia K, Seidel HP (2006) A perceptual framework for contrast processing of high dynamic range images. ACM Trans Appl Percept 3(3):286–308

    Google Scholar 

  41. Menesatti P, Angelini C, Pallottino F, Antonucci F, Aguzzi J, Costa C (2012) Rgb color calibration for quantitative image analysis: the 3d thin-plate spline warping approach. IEEE Sensors 12(1):7063–7079

    Google Scholar 

  42. Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans Pattern Anal Mach Intell (PAMI) 27(10):1615–1630

    Google Scholar 

  43. Morikawa S, Shibata T (2012) Scene image recognition based on the sequence of local image vectors represented by oriented edges. In: Proc. IEEE int. conf. on acoustics, speech, and signal processing, pp 1313–1316

  44. Neumann L, Cadik M, Nemcsics A (2007) An efficient perception-based adaptive color to gray transformation. In: Proc. third eurographics conf. computational aesthetics in graphics, visualization and imaging, pp 73–80

  45. Ojala T, Pietikainen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Int J. Comput Vis Image Underst 29(1):51–59

    Google Scholar 

  46. Oliva A, Torralba A (2001) Modelling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis (IJCV) 42(3):145–175

    MATH  Google Scholar 

  47. Pedram M, Abbas EM, Shahram S (2014) Subjective and objective quality assessment of image: a survey. Majlesi J Electr Eng 9(1):55–83

    Google Scholar 

  48. Qian X, Hua XS, Chen P, Ke L (2011) Plbp: an effective local binary patterns texture descriptor with pyramid representation. Pattern Recognit 44(10–11):2502–2515

    Google Scholar 

  49. Queiroz RLD, Braun KM (2006) Color to gray and back: color embedding into textured gray images. IEEE Trans Image Process 15(6):1464–1470

    Google Scholar 

  50. Rajan S, Sowmya V, Govind D, Soman KP (2017) Dependency of various color and intensity planes on cnn based image classification. In: Proc. third international symposium on signal processing and intelligent recognition systems (SIRS), pp 167–177. https://doi.org/10.1007/978-3-319-67934-1_15

    Google Scholar 

  51. Rangayyan RM, Acha B, Serrano C (2011) Color image processing with biomedical applications. In: SPIE

  52. Rasche K, Geist R, Westall J (2005) Re-coloring images for gamuts of lower dimension. Int J Comput Graphics Forum 24(3):423–432

    Google Scholar 

  53. Renninge LW, Malik J (2003) When is scene recognition just texture recognition? Int J Vis Res 44(1):2301–2311

    Google Scholar 

  54. Serrano N, Savakis A, Luo J (2004) Improved scene classification using efficient low-level features and semantic cues. Pattern Recognit 37(9):1773–1784

    MATH  Google Scholar 

  55. Smith K, Landes PE, Thollot J, Myszkowski K (2008) Apparent greyscale: a simple and fast conversion to perceptually accurate images and video. Int J Comput Graphics Forum 27(2):193–200

    Google Scholar 

  56. Sowmya V, Ajay A, Govind D, Soman KP (2017a) Improved color scene classification systemusing deep belief networks and support vector machines. In: Proc. IEEE int. conf. on signal and image processing applications (ICSIPA)

  57. Sowmya V, Govind D, Soman KP (2017b) Significance of contrast and structure features for an improved color image classification system. In: Proc. IEEE int. conf. on signal and image processing applications (ICSIPA)

  58. Sowmya V, Govind D, Soman KP (2017c) Significance of incorporating chrominance information for effective color-to-grayscale image conversion. Int J Signal Image Video Process 11(1):129–136. https://doi.org/10.1007/s11760-016-0911-8

    Article  Google Scholar 

  59. Suhre A, Kose K, Cetin AE, Gurcan MN (2010) Content-adaptive color transform for image compression. In: Proc. seventeenth int. conf. image processing, pp 189–192

  60. Svetlana L, Cordelia S, Jean P (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proc. int. conf. on computer vision and pattern recognition (CVPR), pp 2169–2178

  61. Vandenbroucke N, Macaire L, Postaire J (2003) Color image segementation by pixel classification in an adpated hybrid color space: application to soccer image analysis. Comput Vis Image Underst 90(1):190–216

    Google Scholar 

  62. Viswanathan S, Divakaran G, Soman KP (2017) Significance of perceptually relevant image decolorization for scene classification. J Electron Imaging SPIE 26(6):129–136

    Google Scholar 

  63. Vogel J, Schiele B (2007) Semantic modelling of natural scenes for content-based image retrieval. Int J Comput Vis (IJCV) 72(2):133–157

    Google Scholar 

  64. Wallraven C, Caputo B, Graf A (2003) Recognition with local features: the kernel recipe. In: Proc. IEEE int. conf. on computer vision (ICCV), pp 257–264

  65. Wang Z (2011) Applications of objective image quality assessment methods. IEEE Signal Process Mag 28(6):137–142

    Google Scholar 

  66. Wang Z, Bovik AC (2009) Mean sqaured error: Love it or leave it? A new look at signal fidelity measures. IEEE Signal Process Mag 26(1):98–117

    Google Scholar 

  67. Wang Z, Bovik AC (2011) Reduced- and no-reference image quality assessment. IEEE Signal Process Mag 28(6):29–40

    Google Scholar 

  68. Wang Z et al (2004) Image quality assessment: From error visibility to strcuture similarity. IEEE Trans. Image Processing 13(4):600–612

    Google Scholar 

  69. Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: Proc. IEEE int. conf. on computer vision and pattern recognition (CVPR), pp 3360–3367

  70. Wang L, Guo S, Huang W, Xiong Y, Qiao Y (2017) Knowledge guided disambiguation for large-scale scene classification with multi-resolution cnns. IEEE Trans Image Process 26(4):2055–2068

    MathSciNet  MATH  Google Scholar 

  71. Willamowski J, Arregui D, Csurka G, Dance CR, Fan L (2004) Categorizing nine visual classes using local appearance descriptors. In: Proc. ICPR workshop on learning for adaptable visual systems, pp 1–11

  72. Wu D, Sun DW (2013) Colour measurements by computer vision for food quality control. Trends Food Sci Technol 29(1):5–20

    Google Scholar 

  73. Wu T, Toet A (2014) Color-to-grayscale conversion through weighted multiresolution channel fusion. J Electron Imaging 23(4):1–6

    Google Scholar 

  74. Xie Z, Ling R, Wu K, Gao J (2012a) Learning robust independent bases for accurate scene categorization. In: Proc. IEEE int. conf. on image and signal processing (CISP), pp 459–463

  75. Xie Z, Ling R, Wu K, Gao J (2012b) Learning robust independent bases for accurate scene categorization. In: Proc. int. congress on image and signal processing (CISP), pp 459–463

  76. Xia J, Ehinger KA, Hays J, Torralba A, Oliva A (2016) Sun database: exploring a large collection of scene categories. Int J Comput Vis (IJCV) 119(1):3–22

    MathSciNet  Google Scholar 

  77. Xue W, Lam PS, Abdesselam B (2016) Visual descriptors for scene categorization: experimental evaluation. Artif Intell Rev 45(3):333–368

    Google Scholar 

  78. Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: Proc. IEEE int. conf. on computer vision and pattern recognition (CVPR), pp 1794–1801

  79. Zhang W, Deng H, Dietterich TG, Mortensen EN (2006) A hierarchical object recognition system based on multi-scale principal curvature regions. In: Proc. eighteenth int. conf. pattern recognition (ICPR), pp 778–782

  80. Zhou B, Lapedriza A, Xiao J, Torralba A, Oliva A (2014) Learning deep features for scene recognition using places database. Adv Neural Inf Process Syst 1(1):487–495

    Google Scholar 

  81. Zhou B, Khosla A, Lapedriza A, Torralba A, Oliva A (2016) Places: an image database for deep scene understanding. arXiv preprint arXiv:1610.02055

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to V. Sowmya.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sowmya, V., Govind, D. & Soman, K.P. Significance of processing chrominance information for scene classification: a review. Artif Intell Rev 53, 811–842 (2020). https://doi.org/10.1007/s10462-018-09678-0

Download citation

Keywords

  • Image decolorization
  • Scene classification
  • Color-to-grayscale
  • Image quality