Abstract
The visual quality evaluation is one of the fundamental challenging problems in image processing. It plays a central role in the shaping, implementation, optimization, and testing of many methods. The existing image quality assessment methods centered mainly on images altered by common distortions while paying little attention to the distortion introduced by color quantization. This happens despite there is a wide range of applications requiring color quantization as a preprocessing step since many color-based tasks are more efficiently accomplished on an image with a reduced number of colors. To fill this gap, at least partially, we carry out a quantitative performance evaluation of nine currently widely-used full-reference image quality assessment measures. The evaluation runs on two publicly available and subjectively rated image quality databases for color quantization degradation by considering their appropriate combinations and subparts. The evaluation results indicate what are the quality measures that have closer performances in terms of their correlation to the subjective human rating and prove that the selected image database significantly impacts the evaluation of the quality measures, although a similar trend on each database is maintained. The detected strong trend similarity, both on individual databases and databases obtained by a proper combination, provides the ability to validate the database combination process and consider the quantitative performance evaluation on each database as an indicator for performance on the other databases. The experimental results are useful to address the choice of appropriate quality measures for color quantization and to improve their future employment.
Similar content being viewed by others
Data Availability
The data presented in this study are openly available at the following Github link: https://github.com/gramella/IQA-CQ
References
Bianco S, Celona L, Napoletano P, Schettini R (2018) On the use of deep learning for blind image quality assessment. Signal Image Video Process 12(2):355–362
Bosse S, Maniry D, Müller KR, Wiegand T, Samek W (2018) Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans Image Process 27(1):206–219
Bruni V, Ramella G, Vitulano D (2015) Automatic Perceptual Color Quantization of Dermoscopic Images. In: Braz J et al (eds) VISAPP 2015 1. Scitepress Science and Technology Publications, pp 323–330
Bruni V, Ramella G, Vitulano D (2017) Perceptual-based Color Quantization. In: Battiato S et al (eds) Image Analysis and Processing - ICIAP 2017, Lecture Notes in Computer Science vol. 10484. Springer, pp 671–681
Chandler DM (2013) Seven challenges in image quality assessment: Past, present, and future research, vol. 2013, Article ID 905685. Hindawi Publishing Corporation ISRN Signal Processing, p 53
Chandler DM, Hemami SS (2007) VSNR: a wavelet-based visual signal- to-noise ratio for natural images. IEEE Trans Image Process 1:2284–2298
Ciancio A, Targino da Costa ALN, da Silva EAB, Said A, Samadani R, Obrador P (2011) No-Reference Blur Assessment of Digital Pictures Based on Multifeature Classifiers, vol. 20, issue 1. IEEE Transactions on Image Processing
Damera-Venkata N, Kite TD, Geisler WS, Evans BL, Bovik AC (2000) Image quality assessment based on a degradation model. IEEE Trans Image Process 9:636–650
De K, Masilamani V (2016) No-reference image quality assessment for images degraded by color quantization in HSV Space. Proc. 2016 IEEE Students Technology Symposium, Kharagpur, India, 30 September–2 October 2016, pp 40–45
Dekker A (1994) Kohonen neural networks for optimal colour quantization. Network Comp Neural Systems 5(3):351–367
Eskicioglu AM, Fisher PS (1995) Image quality measures and their performance. IEEE Trans on Comm 43(12):2959–2965
Ferzli R, Karam LJ (2009) A no-reference objective image sharpness metric based on the notion of just noticeable blur (jnb). IEEE Trans Imag Process 18(4):717–728
Frackiewicz M, Palus H (2017) New image quality metric used for the assessment of color quantization algorithms. In: Verikas A et al (eds) Proc. SPIE, Ninth International Conference on Machine Vision - ICMV 2016, vol. 10341. Nice, France, p 103411G
Frackiewicz M, Palus H (2017) Further applications of the DSCSI metric for evaluating color quantization. In: Verikas A et al (eds) Proc. SPIE, Ninth International Conference on Machine Vision (ICMV 2016), vol. 10341. Nice, France, p 103411H
Gao F, Yu J (2016) Biologically inspired image quality assessment. Signal Process 124:210–219
Gaubatz M (2014) Metrix MUX Visual Quality Assessment Package. http://foulard.ece.cornell.edu/gaubatz/metrix_mux/
Gervautz M, Purgathofer W (1990) A simple method for color quantization: Octree quantization. In: Graphics Gems. Academic Press Professional Inc, San Diego, CA, USA, pp 287–293
Gibbons JD, Chakraborti S (2011) Nonparametric Statistical Inference, 5th edn. Chapman & Hall/CRC Press, Taylor & Francis Group, Boca Raton, FL
Golestaneh SA, Chandler DM (2014) No-reference quality assessment of jpeg images via a quality relevance map. IEEE Sign Process Lett 21(2):155–158
Gu K, Zhai G, Lin W, Yang X, Zhang W (2015) No-reference image sharpness assessment in autoregressive parameter space. IEEE Trans Imag Process 24(10):3218–3231
Hasler D, Susstrunk S (2003) Measuring colourfulness in natural images. Proc. of the IS&T/SPIE Electronic Imaging: Human Vision and Electronic Imaging VIII, pp 87–95
Hassan MA, Bashraheel MS (2017) Color-based structural similarity image quality assessment. Proc. 8th International Conference on Information Technology - ICIT 2017. Amman, Jordan, 17–18 May 2017, pp 691-606
Hassan M, Bhagvati C (2015) Evaluation of image quality assessment metrics: Color quantization noise. Intern J Appl Inform Syst 9(1):1–8
Heckbert P (1982) Color image quantization for frame buffer display. ACM SIGGRAPH Comp Graphics 16(3):297–307
Hou W, Gao X, Tao D, Li X (2015) Blind image quality assessment via deep learning. IEEE Trans Neural Netw Learn Syst 26(6):1275–1286
Huynh-Thu Q, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assessment. Electron Lett 44(13):800–801
ISO/IEC 2010 (2010) ISO/IEC: ISO/IEC 29170: AIC technical report on evaluation methodologies - working draft v3, Tech. Rep. wg1n5598, ISO/IEC JTC1 SC29/WG1, Guangzhou, China
ISO/IEC 29170-1. https://www.iso.org/standard/63637.html
ISO/IEC 29170-2. https://www.iso.org/standard/66094.html
ITU (2002) Methodology for the subjective assessment of the quality of television pictures, in Recommendation Gaubatz BT500-11, Geneva, Switzerland
Kamble V, Bhurchandi KM (2015) No-reference image quality assessment algorithms: a survey. Optik 126:1090–1097
Karunasekera SA, Kingsbury NG (1995) A distortion measure for blocking artifacts in images based on human visual sensitivity. IEEE Trans Image Process 4(6):713–724
Kodak Lossless True Color Image Suite. http://r0k.us/graphics/kodak/
Kingdom F, Prins N (2010) Psychometric functions. Psychophysics. In: A Practical Introduction, Pearson Prentice Hall
Lahoulou A, Bouridane A, Viennet E, Haddadi M (2013) Full-Reference Image Quality Metrics Performance Evaluation Over Image Quality Databases. Arab J Sci Eng 38:2327–2356
Larabi M-C, Montagne C, Lelandais S, Smolarz A, Fernandez-Maloigne C, Cornu P (2004) Color quantization: Objective and subjective comparisons of different algorithms. Traitement du signal 5(5):385–405
Larson E, Chandler D (2010) Most apparent distortion: full-reference image quality assessment and the role of strategy. J Electr Imaging 19(1):011006–011006
Le Callet P, Autrusseau F (2005) Subjective quality assessment IRCCyN/IVC database. http://www.irccyn.ec-nantes.fr/ivcdb/
Lee D, Rogers E (2014) Towards a novel perceptual color difference metric using circular processing of hue components. Proc. IEEE International Conference on Acoustics, Speech and Signal Processing. Florence, Italy, pp 166–170
Lee D, Plataniotis KN (2015) Towards a full-reference quality assessment for color images using directional statistics. IEEE Trans Imag Process 24(11):3950–3965
Li L, Lin W, Wang X, Yang G, Bahrami K, Kot AC (2016) No-reference image blur assessment based on discrete orthogonal moments. IEEE Trans Cyb 46(1):39–50
Lin W, Kuo CCJ (2011) Perceptual visual quality metrics: a survey. J Vis Commun Image R 22:297–312
Linde Y, Buzo A, Gray RM (1980) An algorithm for vector quantizer desing. IEEE Trans Comm 28(1):84–95
Lissner I, Preiss J, Urban P, Lichtenauer MS, Zolliker P (2013) Image-difference prediction: From grayscale to color. IEEE Trans Image Process 22(2):435–446
Liu M, Gu K, Zhai G, Le Callet P, Zhang W (2017) Perceptual reduced-reference visual quality assessment for contrast alteration. IEEE Trans Broadcasting 63(1):71–81
Liu Y, Gu K, Zhai G, Liu X, Zhao D, Gao W (2017) Quality assessment for real out-of-focus blurred images. J Vis Commun Image R 46:70–80
Luo MR, Cui G, Rigg B (2001) The development of the CIE 2000 colour-difference formula: CIEDE2000. Color Res Appl 26(5):340–350
Ly D, Beucher S, Bilodeau M, Persa S, Damstra K, Pot R, Rooy J (2015) Automatic color correction: region-based approach and performance evaluation using full reference metrics. J Electron Imaging 24:1–9
Marziliano P, Dufaux F, Winkler S, Ebrahimi T (2002) A No-reference perceptual blur metric. In: Proceedings of IEEE International Conference on Image Processing, vol. 3. Rochester, NY, USA, 22-Sept. 2002, pp III–57
Massey FJ (1951) The Kolmogorov-Smirnov test for goodness of fit. J American Statist Assoc 46(253):68–78
Mitsa T, Varkur KL (1993) Evaluation of contrast sensitivity functions for the formulation of quality measures incorporated in halftoning algorithms. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Minneapolis, USA, 1993, pp. 301–304
Mojsilovic A, Jianying H, Soljanin E (2002) Extraction of perceptually important colors and similarity measurement for image matching, retrieval and analysis. IEEE Trans Image Process 11:1238–1248
Morovič J (2008) Desired color reproduction properties and their evaluation. In: Kriss MA (ed) Color Gamut Mapping. John Wiley & Sons Inc
Ortiz-Jaramillo B, Kumcu A, Platisa L, Philips W (2019) Evaluation of color differences in natural scene color images. Signal Process Imag Comm 71:128–137
Parvez Sazzad ZM, Kawayoke Y, Horita Y (2008) MICT image quality evaluation database. http://mict.eng.u-toyama.ac.jp/mictdb.html
Peli E (1990) Contrast in complex images. J Opt Soc Amer A 7:2032–2039
Pedersen M, Hardeberg JY (2012) Full-reference image quality metrics. Classification and evaluation. Foundations and Trends in Computer Graphics and Vision, vol. 7, issue 1. Now Publishers Inc, Hanover, USA, pp 1–80
Plataniotis K, Venetsanopoulos N (2000) Color image processing and applications. Commun ACM 34:30–44
Ponomarenko N, Lukin V, Zelensky A, Egiazarian K, Carli M, Battisti F (2009) TID2008 - A database for evaluation of full-reference visual quality assessment metrics. Adv Mod Radioelectron 10:30–45
Ponomarenko N, Jin L, Ieremeiev O, Lukin V, Egiazarian K, Astola J, Vozel B, Chehdi K, Carli M, Battisti F, Jay-Kuo C (2015) Image database TID2013: Peculiarities, results and perspectives. Signal Process Image Commun 30:57–77
Rajashekar U, Wang Z, Simoncelli E (2009) Quantifying color image distortions based on adaptive spatio-chromatic signal decompositions. Proc. IEEE International Conference on Image Processing. Cairo, Egypt, 7–10 Nov. 2009, pp 2213-2216
Ramella G (2020) Automatic skin lesion segmentation based on saliency and color. In: Farinella GM et al (eds) VISAPP 2020 4. Scitepress Science and Technology Publications, pp 452–459
Ramella G, Sanniti di Baja G (2010) Multiresolution histogram analysis for color reduction. In: Bloch I, Cesar, Jr RM (eds) Progress in Pattern Recognition, Image Analysis, Computer Vision and Applications - CIARP 2010, Lecture Notes in Computer Science vol. 6419. Springer, pp 22–29
Ramella G, Sanniti di Baja G (2011) Color histogram-based image segmentation. In: Real P, Diaz-Pernil D, Molina-Abril H, Berciano A, Kropatsch W (eds) Computer Analysis of Images and Patterns - CAIP 2011, Lecture Notes in Computer Science vol. 6854. Springer I, pp 76–83
Ramella G, Sanniti di Baja G (2013) A new technique for color quantization based on histogram analysis and clustering. Int J Patt Recog Artif Intell 27(3):1–17
Ramella G, Sanniti di Baja G (2013) Color quantization via spatial resolution reduction. In: Battiato S, Braz J (eds) VISAPP 2013. Scitepress Science and Technology Publications, pp 78–83
Ramella G, Sanniti di Baja G (2016) A new method for color quantization. Yetongnon K et al (eds) Proc 12th Intern Conf Signal Imag Techn Internet-Based Syst. - SITIS 2016. IEEE Computer Society, pp 1–6
Ramella G, Sanniti di Baja G (2016) From color quantization to image segmentation. In: Yetongnon K et al (eds) Proc. 12th International Conference on Signal-Image Technology & Internet-Based Systems - SITIS 2016. IEEE Computer Society, pp 798–804
Rehman A, Wang Z (2012) Reduced-reference image quality assessment by structural similarity estimation. IEEE Trans Imag Process 21(8):3378–3389
Roberto e Souza M, Carlos Sousa e Santos A, Pedrini H (2020) A hybrid approach using the k means and genetic algorithms for image color quantization. In: De S, Dey S, Bhattacharyya S (eds) Recent advances in hybrid metaheuristics for data clustering, Chapter 9. Wiley online library
Sertel O, Kong J, Lozanski G, Shana’ah A, Catalyurek U, Saltz J (2008) Texture classification using nonlinear color quantization: Application to histopathological image analysis. Proc. 2008 IEEE International Conference Acoustics, Speech and Signal Process. Las Vegas, 31 March–4 April, pp 597–600
Silverstein DA, Farrell JE (1996) The relationship between image fidelity and image quality. Proc. IEEE International Conference Image Processing, Lausanne, Switzerland, 19–19 Sept. 1996, pp 881–884
Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15:430–444
Sheikh HR, Bovik AC, Cormack L (2005) No-reference quality assessment using natural scene statistics: Jpeg 2000. IEEE Trans Imag Process 14(11):1918–1927
Sheikh HR, Wang Z, Cormack L, Bovik AC (2005) LIVE Image Quality Assessment Database Release 2. http://live.ece.utexas.edu/research/quality
Shokrollahi A, Mahmoudi-Aznaveh A, Maybodi BMN (2017) Image quality assessment for contrast enhancement evaluation. AEU - Int J Electr Commun 77:61–66
Soundararajan R, Bovik AC (2012) RRED indices: Reduced reference entropic differencing for image quality assessment. IEEE Trans Image Process 21(2):517–526
Sun W, Zhou F, Liao Q (2017) MDID: A multiply distorted image database for image quality assessment. Patt Recogn 61:153–168
Tanchenko A (2014) Visual-PSNR measure of image quality. J Vis Commun Image R 25:874–878
Temel D, AlRegib G (2016) CSV: Image quality assessment based on color, structure, and visual system. Sign Process: Image Commun 48:92–103
Thompson S, Celebi ME, Buck KH (2019) Fast color quantization using MacQueen k-means algorithm. J Real-Time Image Proc 1–16
Toet A, Lucassen MP (2003) A new universal colour image fidelity metric. Displays 24(4):197–207
Velho L, Frery AC, Gomes J (2009) Color quantization. In: Image processing for computer graphics and vision. Texts in Computer Science. Springer-Verlag: London, pp 293-311
VQEG 1 (2008) Final report from the video quality experts group on the validation of objective models of multimedia quality assessment, Tech. Rep. PHASE I 2008, VQEG
VQEG 2 (2003) Final report from the video quality experts group on the validation of objective models of video quality assessment, Tech. Rep. PHASE II 2003, VQEG
Wallace G (1991) The jpeg still picture compression standard. Commun ACM 34(4):30–44
Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Process Lett 9:81–84
Wang Z, Bovik AC (2006) Modern image quality assessment. Synth Lect Image Video Multimed Process 2(1):1–15
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Proc 13(4):600–612
Wang Z, Li Q (2011) Information content weighting for perceptual image quality assessment. IEEE Trans Imag Process 20:1185–1198
Wang Z, Simoncelli EP (2008) Maximum differentiation (MAD) competition: a methodology for comparing computational models of perceptual quantities. J Vis 8(12):1–13
Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment. Proceedings Thirty- Seventh Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, 9–12 November 2003, pp 1398–1402
Wang X, Wang X, Wilkes DM (2020) A fast image retrieval method based on a quantization tree. In: Machine Learning-based Natural Scene Recognition for Mobile Robot Localization in An Unknown Environment. Springer, Singapore, pp 195–214
Weeks AR (1996) Fundamentals of electronic image processing. SPIE/IEEE Series on Imaging Science and Engineering. SPIE Optical Engineering Press, Washington USA
Wena Y, Li Y, Zhang X, Shi W, Wang L, Chen J (2017) A weighted full-reference image quality assessment based on visual Saliency. J Vis Commun Image R 43:119–126
Winkler S (2012) Analysis of public image and video databases for quality assessment. IEEE J Selected Topics Sig Process 6(6):1–10
Wu X (1991) Efficient statistical computations for optimal color quantization. In: Graphics Gems II, New York: Academic, James Arvo edition, pp 126–133
Yan B, Bare B, Tan W (2019) Naturalness-aware deep no-reference image quality assessment. IEEE Trans on Multimedia 21(10):2603–2615
Yang Y, Ming J, Yu N (2012) Color image quality assessment based on Ciede 2000. Adv Multimedia 1–6
Zaric A, Tatalovic N, Brajkovic N, Hlevnjak H, Loncaric M, Dumic E, Grgic S (2012) VCL@FER image quality assessment database. Automatika 53(4):344–354
Zhang L, Zhang L, Mou X, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386
Acknowledgements
This work has been supported by the GNCS (Gruppo Nazionale di Calcolo Scientifico) of the INDAM (Istituto Nazionale di Alta Matematica).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Ramella, G. Evaluation of quality measures for color quantization. Multimed Tools Appl 80, 32975–33009 (2021). https://doi.org/10.1007/s11042-021-11385-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-021-11385-y