Skip to main content
SpringerLink
Log in

Evaluation of quality measures for color quantization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The visual quality evaluation is one of the fundamental challenging problems in image processing. It plays a central role in the shaping, implementation, optimization, and testing of many methods. The existing image quality assessment methods centered mainly on images altered by common distortions while paying little attention to the distortion introduced by color quantization. This happens despite there is a wide range of applications requiring color quantization as a preprocessing step since many color-based tasks are more efficiently accomplished on an image with a reduced number of colors. To fill this gap, at least partially, we carry out a quantitative performance evaluation of nine currently widely-used full-reference image quality assessment measures. The evaluation runs on two publicly available and subjectively rated image quality databases for color quantization degradation by considering their appropriate combinations and subparts. The evaluation results indicate what are the quality measures that have closer performances in terms of their correlation to the subjective human rating and prove that the selected image database significantly impacts the evaluation of the quality measures, although a similar trend on each database is maintained. The detected strong trend similarity, both on individual databases and databases obtained by a proper combination, provides the ability to validate the database combination process and consider the quantitative performance evaluation on each database as an indicator for performance on the other databases. The experimental results are useful to address the choice of appropriate quality measures for color quantization and to improve their future employment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

The data presented in this study are openly available at the following Github link: https://github.com/gramella/IQA-CQ

References

  1. Bianco S, Celona L, Napoletano P, Schettini R (2018) On the use of deep learning for blind image quality assessment. Signal Image Video Process 12(2):355–362

    Article  Google Scholar 

  2. Bosse S, Maniry D, Müller KR, Wiegand T, Samek W (2018) Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans Image Process 27(1):206–219

    Article  MathSciNet  MATH  Google Scholar 

  3. Bruni V, Ramella G, Vitulano D (2015) Automatic Perceptual Color Quantization of Dermoscopic Images. In: Braz J et al (eds) VISAPP 2015 1. Scitepress Science and Technology Publications, pp 323–330

  4. Bruni V, Ramella G, Vitulano D (2017) Perceptual-based Color Quantization. In: Battiato S et al (eds) Image Analysis and Processing - ICIAP 2017, Lecture Notes in Computer Science vol. 10484. Springer, pp 671–681

  5. Chandler DM (2013) Seven challenges in image quality assessment: Past, present, and future research, vol. 2013, Article ID 905685. Hindawi Publishing Corporation ISRN Signal Processing, p 53

  6. Chandler DM, Hemami SS (2007) VSNR: a wavelet-based visual signal- to-noise ratio for natural images. IEEE Trans Image Process 1:2284–2298

    Article  MathSciNet  Google Scholar 

  7. Ciancio A, Targino da Costa ALN, da Silva EAB, Said A, Samadani R, Obrador P (2011) No-Reference Blur Assessment of Digital Pictures Based on Multifeature Classifiers, vol. 20, issue 1. IEEE Transactions on Image Processing

  8. Damera-Venkata N, Kite TD, Geisler WS, Evans BL, Bovik AC (2000) Image quality assessment based on a degradation model. IEEE Trans Image Process 9:636–650

    Article  Google Scholar 

  9. De K, Masilamani V (2016) No-reference image quality assessment for images degraded by color quantization in HSV Space. Proc. 2016 IEEE Students Technology Symposium, Kharagpur, India, 30 September–2 October 2016, pp 40–45

  10. Dekker A (1994) Kohonen neural networks for optimal colour quantization. Network Comp Neural Systems 5(3):351–367

    Article  MATH  Google Scholar 

  11. Eskicioglu AM, Fisher PS (1995) Image quality measures and their performance. IEEE Trans on Comm 43(12):2959–2965

    Article  Google Scholar 

  12. Ferzli R, Karam LJ (2009) A no-reference objective image sharpness metric based on the notion of just noticeable blur (jnb). IEEE Trans Imag Process 18(4):717–728

    Article  MathSciNet  MATH  Google Scholar 

  13. Frackiewicz M, Palus H (2017) New image quality metric used for the assessment of color quantization algorithms. In: Verikas A et al (eds) Proc. SPIE, Ninth International Conference on Machine Vision - ICMV 2016, vol. 10341. Nice, France, p 103411G

  14. Frackiewicz M, Palus H (2017) Further applications of the DSCSI metric for evaluating color quantization. In: Verikas A et al (eds) Proc. SPIE, Ninth International Conference on Machine Vision (ICMV 2016), vol. 10341. Nice, France, p 103411H

  15. Gao F, Yu J (2016) Biologically inspired image quality assessment. Signal Process 124:210–219

    Article  Google Scholar 

  16. Gaubatz M (2014) Metrix MUX Visual Quality Assessment Package. http://foulard.ece.cornell.edu/gaubatz/metrix_mux/

  17. Gervautz M, Purgathofer W (1990) A simple method for color quantization: Octree quantization. In: Graphics Gems. Academic Press Professional Inc, San Diego, CA, USA, pp 287–293

  18. Gibbons JD, Chakraborti S (2011) Nonparametric Statistical Inference, 5th edn. Chapman & Hall/CRC Press, Taylor & Francis Group, Boca Raton, FL

    MATH  Google Scholar 

  19. Golestaneh SA, Chandler DM (2014) No-reference quality assessment of jpeg images via a quality relevance map. IEEE Sign Process Lett 21(2):155–158

    Article  Google Scholar 

  20. Gu K, Zhai G, Lin W, Yang X, Zhang W (2015) No-reference image sharpness assessment in autoregressive parameter space. IEEE Trans Imag Process 24(10):3218–3231

    Article  MathSciNet  MATH  Google Scholar 

  21. Hasler D, Susstrunk S (2003) Measuring colourfulness in natural images. Proc. of the IS&T/SPIE Electronic Imaging: Human Vision and Electronic Imaging VIII, pp 87–95

  22. Hassan MA, Bashraheel MS (2017) Color-based structural similarity image quality assessment. Proc. 8th International Conference on Information Technology - ICIT 2017. Amman, Jordan, 17–18 May 2017, pp 691-606

  23. Hassan M, Bhagvati C (2015) Evaluation of image quality assessment metrics: Color quantization noise. Intern J Appl Inform Syst 9(1):1–8

    Google Scholar 

  24. Heckbert P (1982) Color image quantization for frame buffer display. ACM SIGGRAPH Comp Graphics 16(3):297–307

    Article  Google Scholar 

  25. Hou W, Gao X, Tao D, Li X (2015) Blind image quality assessment via deep learning. IEEE Trans Neural Netw Learn Syst 26(6):1275–1286

    Article  MathSciNet  Google Scholar 

  26. Huynh-Thu Q, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assessment. Electron Lett 44(13):800–801

    Article  Google Scholar 

  27. ISO/IEC 2010 (2010) ISO/IEC: ISO/IEC 29170: AIC technical report on evaluation methodologies - working draft v3, Tech. Rep. wg1n5598, ISO/IEC JTC1 SC29/WG1, Guangzhou, China

  28. ISO/IEC 29170-1. https://www.iso.org/standard/63637.html

  29. ISO/IEC 29170-2. https://www.iso.org/standard/66094.html

  30. ITU (2002) Methodology for the subjective assessment of the quality of television pictures, in Recommendation Gaubatz BT500-11, Geneva, Switzerland

  31. Kamble V, Bhurchandi KM (2015) No-reference image quality assessment algorithms: a survey. Optik 126:1090–1097

    Article  Google Scholar 

  32. Karunasekera SA, Kingsbury NG (1995) A distortion measure for blocking artifacts in images based on human visual sensitivity. IEEE Trans Image Process 4(6):713–724

    Article  Google Scholar 

  33. Kodak Lossless True Color Image Suite. http://r0k.us/graphics/kodak/

  34. Kingdom F, Prins N (2010) Psychometric functions. Psychophysics. In: A Practical Introduction, Pearson Prentice Hall

  35. Lahoulou A, Bouridane A, Viennet E, Haddadi M (2013) Full-Reference Image Quality Metrics Performance Evaluation Over Image Quality Databases. Arab J Sci Eng 38:2327–2356

    Article  Google Scholar 

  36. Larabi M-C, Montagne C, Lelandais S, Smolarz A, Fernandez-Maloigne C, Cornu P (2004) Color quantization: Objective and subjective comparisons of different algorithms. Traitement du signal 5(5):385–405

    MATH  Google Scholar 

  37. Larson E, Chandler D (2010) Most apparent distortion: full-reference image quality assessment and the role of strategy. J Electr Imaging 19(1):011006–011006

    Article  Google Scholar 

  38. Le Callet P, Autrusseau F (2005) Subjective quality assessment IRCCyN/IVC database. http://www.irccyn.ec-nantes.fr/ivcdb/

  39. Lee D, Rogers E (2014) Towards a novel perceptual color difference metric using circular processing of hue components. Proc. IEEE International Conference on Acoustics, Speech and Signal Processing. Florence, Italy, pp 166–170

  40. Lee D, Plataniotis KN (2015) Towards a full-reference quality assessment for color images using directional statistics. IEEE Trans Imag Process 24(11):3950–3965

    Article  MathSciNet  MATH  Google Scholar 

  41. Li L, Lin W, Wang X, Yang G, Bahrami K, Kot AC (2016) No-reference image blur assessment based on discrete orthogonal moments. IEEE Trans Cyb 46(1):39–50

    Article  Google Scholar 

  42. Lin W, Kuo CCJ (2011) Perceptual visual quality metrics: a survey. J Vis Commun Image R 22:297–312

    Article  Google Scholar 

  43. Linde Y, Buzo A, Gray RM (1980) An algorithm for vector quantizer desing. IEEE Trans Comm 28(1):84–95

    Article  Google Scholar 

  44. Lissner I, Preiss J, Urban P, Lichtenauer MS, Zolliker P (2013) Image-difference prediction: From grayscale to color. IEEE Trans Image Process 22(2):435–446

    Article  MathSciNet  MATH  Google Scholar 

  45. Liu M, Gu K, Zhai G, Le Callet P, Zhang W (2017) Perceptual reduced-reference visual quality assessment for contrast alteration. IEEE Trans Broadcasting 63(1):71–81

    Article  Google Scholar 

  46. Liu Y, Gu K, Zhai G, Liu X, Zhao D, Gao W (2017) Quality assessment for real out-of-focus blurred images. J Vis Commun Image R 46:70–80

    Article  Google Scholar 

  47. Luo MR, Cui G, Rigg B (2001) The development of the CIE 2000 colour-difference formula: CIEDE2000. Color Res Appl 26(5):340–350

    Article  Google Scholar 

  48. Ly D, Beucher S, Bilodeau M, Persa S, Damstra K, Pot R, Rooy J (2015) Automatic color correction: region-based approach and performance evaluation using full reference metrics. J Electron Imaging 24:1–9

    Article  Google Scholar 

  49. Marziliano P, Dufaux F, Winkler S, Ebrahimi T (2002) A No-reference perceptual blur metric. In: Proceedings of IEEE International Conference on Image Processing, vol. 3. Rochester, NY, USA, 22-Sept. 2002, pp III–57

  50. Massey FJ (1951) The Kolmogorov-Smirnov test for goodness of fit. J American Statist Assoc 46(253):68–78

    Article  MATH  Google Scholar 

  51. Mitsa T, Varkur KL (1993) Evaluation of contrast sensitivity functions for the formulation of quality measures incorporated in halftoning algorithms. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Minneapolis, USA, 1993, pp. 301–304

  52. Mojsilovic A, Jianying H, Soljanin E (2002) Extraction of perceptually important colors and similarity measurement for image matching, retrieval and analysis. IEEE Trans Image Process 11:1238–1248

    Article  MathSciNet  Google Scholar 

  53. Morovič J (2008) Desired color reproduction properties and their evaluation. In: Kriss MA (ed) Color Gamut Mapping. John Wiley & Sons Inc

  54. Ortiz-Jaramillo B, Kumcu A, Platisa L, Philips W (2019) Evaluation of color differences in natural scene color images. Signal Process Imag Comm 71:128–137

    Article  Google Scholar 

  55. Parvez Sazzad ZM, Kawayoke Y, Horita Y (2008) MICT image quality evaluation database. http://mict.eng.u-toyama.ac.jp/mictdb.html

  56. Peli E (1990) Contrast in complex images. J Opt Soc Amer A 7:2032–2039

    Article  Google Scholar 

  57. Pedersen M, Hardeberg JY (2012) Full-reference image quality metrics. Classification and evaluation. Foundations and Trends in Computer Graphics and Vision, vol. 7, issue 1. Now Publishers Inc, Hanover, USA, pp 1–80

  58. Plataniotis K, Venetsanopoulos N (2000) Color image processing and applications. Commun ACM 34:30–44

    Google Scholar 

  59. Ponomarenko N, Lukin V, Zelensky A, Egiazarian K, Carli M, Battisti F (2009) TID2008 - A database for evaluation of full-reference visual quality assessment metrics. Adv Mod Radioelectron 10:30–45

    Google Scholar 

  60. Ponomarenko N, Jin L, Ieremeiev O, Lukin V, Egiazarian K, Astola J, Vozel B, Chehdi K, Carli M, Battisti F, Jay-Kuo C (2015) Image database TID2013: Peculiarities, results and perspectives. Signal Process Image Commun 30:57–77

    Article  Google Scholar 

  61. Rajashekar U, Wang Z, Simoncelli E (2009) Quantifying color image distortions based on adaptive spatio-chromatic signal decompositions. Proc. IEEE International Conference on Image Processing. Cairo, Egypt, 7–10 Nov. 2009, pp 2213-2216

  62. Ramella G (2020) Automatic skin lesion segmentation based on saliency and color. In: Farinella GM et al (eds) VISAPP 2020 4. Scitepress Science and Technology Publications, pp 452–459

  63. Ramella G, Sanniti di Baja G (2010) Multiresolution histogram analysis for color reduction. In: Bloch I, Cesar, Jr RM (eds) Progress in Pattern Recognition, Image Analysis, Computer Vision and Applications - CIARP 2010, Lecture Notes in Computer Science vol. 6419. Springer, pp 22–29

  64. Ramella G, Sanniti di Baja G (2011) Color histogram-based image segmentation. In: Real P, Diaz-Pernil D, Molina-Abril H, Berciano A, Kropatsch W (eds) Computer Analysis of Images and Patterns - CAIP 2011, Lecture Notes in Computer Science vol. 6854. Springer I, pp 76–83

  65. Ramella G, Sanniti di Baja G (2013) A new technique for color quantization based on histogram analysis and clustering. Int J Patt Recog Artif Intell 27(3):1–17

    Article  MathSciNet  Google Scholar 

  66. Ramella G, Sanniti di Baja G (2013) Color quantization via spatial resolution reduction. In: Battiato S, Braz J (eds) VISAPP 2013. Scitepress Science and Technology Publications, pp 78–83

  67. Ramella G, Sanniti di Baja G (2016) A new method for color quantization. Yetongnon K et al (eds) Proc 12th Intern Conf Signal Imag Techn Internet-Based Syst. - SITIS 2016. IEEE Computer Society, pp 1–6

  68. Ramella G, Sanniti di Baja G (2016) From color quantization to image segmentation. In: Yetongnon K et al (eds) Proc. 12th International Conference on Signal-Image Technology & Internet-Based Systems - SITIS 2016. IEEE Computer Society, pp 798–804

  69. Rehman A, Wang Z (2012) Reduced-reference image quality assessment by structural similarity estimation. IEEE Trans Imag Process 21(8):3378–3389

    Article  MathSciNet  MATH  Google Scholar 

  70. Roberto e Souza M, Carlos Sousa e Santos A, Pedrini H (2020) A hybrid approach using the k means and genetic algorithms for image color quantization. In: De S, Dey S, Bhattacharyya S (eds) Recent advances in hybrid metaheuristics for data clustering, Chapter 9. Wiley online library

  71. Sertel O, Kong J, Lozanski G, Shana’ah A, Catalyurek U, Saltz J (2008) Texture classification using nonlinear color quantization: Application to histopathological image analysis. Proc. 2008 IEEE International Conference Acoustics, Speech and Signal Process. Las Vegas, 31 March–4 April, pp 597–600

  72. Silverstein DA, Farrell JE (1996) The relationship between image fidelity and image quality. Proc. IEEE International Conference Image Processing, Lausanne, Switzerland, 19–19 Sept. 1996, pp 881–884

  73. Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15:430–444

    Article  Google Scholar 

  74. Sheikh HR, Bovik AC, Cormack L (2005) No-reference quality assessment using natural scene statistics: Jpeg 2000. IEEE Trans Imag Process 14(11):1918–1927

    Article  Google Scholar 

  75. Sheikh HR, Wang Z, Cormack L, Bovik AC (2005) LIVE Image Quality Assessment Database Release 2. http://live.ece.utexas.edu/research/quality

  76. Shokrollahi A, Mahmoudi-Aznaveh A, Maybodi BMN (2017) Image quality assessment for contrast enhancement evaluation. AEU - Int J Electr Commun 77:61–66

    Article  Google Scholar 

  77. Soundararajan R, Bovik AC (2012) RRED indices: Reduced reference entropic differencing for image quality assessment. IEEE Trans Image Process 21(2):517–526

    Article  MathSciNet  MATH  Google Scholar 

  78. Sun W, Zhou F, Liao Q (2017) MDID: A multiply distorted image database for image quality assessment. Patt Recogn 61:153–168

    Article  Google Scholar 

  79. Tanchenko A (2014) Visual-PSNR measure of image quality. J Vis Commun Image R 25:874–878

    Article  Google Scholar 

  80. Temel D, AlRegib G (2016) CSV: Image quality assessment based on color, structure, and visual system. Sign Process: Image Commun 48:92–103

    Google Scholar 

  81. Thompson S, Celebi ME, Buck KH (2019) Fast color quantization using MacQueen k-means algorithm. J Real-Time Image Proc 1–16

  82. Toet A, Lucassen MP (2003) A new universal colour image fidelity metric. Displays 24(4):197–207

    Article  Google Scholar 

  83. Velho L, Frery AC, Gomes J (2009) Color quantization. In: Image processing for computer graphics and vision. Texts in Computer Science. Springer-Verlag: London, pp 293-311

  84. VQEG 1 (2008) Final report from the video quality experts group on the validation of objective models of multimedia quality assessment, Tech. Rep. PHASE I 2008, VQEG

  85. VQEG 2 (2003) Final report from the video quality experts group on the validation of objective models of video quality assessment, Tech. Rep. PHASE II 2003, VQEG

  86. Wallace G (1991) The jpeg still picture compression standard. Commun ACM 34(4):30–44

    Article  Google Scholar 

  87. Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Process Lett 9:81–84

    Article  Google Scholar 

  88. Wang Z, Bovik AC (2006) Modern image quality assessment. Synth Lect Image Video Multimed Process 2(1):1–15

    Article  Google Scholar 

  89. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Proc 13(4):600–612

    Article  Google Scholar 

  90. Wang Z, Li Q (2011) Information content weighting for perceptual image quality assessment. IEEE Trans Imag Process 20:1185–1198

    Article  MathSciNet  MATH  Google Scholar 

  91. Wang Z, Simoncelli EP (2008) Maximum differentiation (MAD) competition: a methodology for comparing computational models of perceptual quantities. J Vis 8(12):1–13

    Article  Google Scholar 

  92. Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment. Proceedings Thirty- Seventh Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, 9–12 November 2003, pp 1398–1402

  93. Wang X, Wang X, Wilkes DM (2020) A fast image retrieval method based on a quantization tree. In: Machine Learning-based Natural Scene Recognition for Mobile Robot Localization in An Unknown Environment. Springer, Singapore, pp 195–214

  94. Weeks AR (1996) Fundamentals of electronic image processing. SPIE/IEEE Series on Imaging Science and Engineering. SPIE Optical Engineering Press, Washington USA

    Google Scholar 

  95. Wena Y, Li Y, Zhang X, Shi W, Wang L, Chen J (2017) A weighted full-reference image quality assessment based on visual Saliency. J Vis Commun Image R 43:119–126

    Article  Google Scholar 

  96. Winkler S (2012) Analysis of public image and video databases for quality assessment. IEEE J Selected Topics Sig Process 6(6):1–10

    Google Scholar 

  97. Wu X (1991) Efficient statistical computations for optimal color quantization. In: Graphics Gems II, New York: Academic, James Arvo edition, pp 126–133

  98. Yan B, Bare B, Tan W (2019) Naturalness-aware deep no-reference image quality assessment. IEEE Trans on Multimedia 21(10):2603–2615

    Article  Google Scholar 

  99. Yang Y, Ming J, Yu N (2012) Color image quality assessment based on Ciede 2000. Adv Multimedia 1–6

  100. Zaric A, Tatalovic N, Brajkovic N, Hlevnjak H, Loncaric M, Dumic E, Grgic S (2012) VCL@FER image quality assessment database. Automatika 53(4):344–354

    Article  Google Scholar 

  101. Zhang L, Zhang L, Mou X, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work has been supported by the GNCS (Gruppo Nazionale di Calcolo Scientifico) of the INDAM (Istituto Nazionale di Alta Matematica).

Author information

Authors and Affiliations

  1. National Research Council (CNR), Institute for the Applications of Calculus, Via P. Castellino 111, Naples, 80131, Italy

    Giuliana Ramella

Authors
  1. Giuliana Ramella
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giuliana Ramella.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ramella, G. Evaluation of quality measures for color quantization. Multimed Tools Appl 80, 32975–33009 (2021). https://doi.org/10.1007/s11042-021-11385-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11385-y

Keywords

Search

Navigation