Abstract
With the development of digital visual signal processing, efficient and reliable assessment of image quality becomes more and more important. Measuring the image quality is of fundamental importance for image processing applications, where the goal of image quality assessment (IQA) methods is to automatically evaluate the quality of images in agreement with human visual quality judgments. In the past decade, researchers have made great efforts to develop many IQAs to fulfill this goal. According to the availability of reference, these IQAs can be classified into full-reference, reduce-reference, and no-reference IQAs. For most of the applications, there is no reference available during visual signal processing, e.g., decoding of compressed visual signal in client’s terminal without the original signal stored in server. The absence of reference raises a great challenge for visual quality assessment (VQA). To address this challenge, machine learning was widely used to approximate the response of the human visual system (HVS) to visual quality perception. This chapter will present the fundamental knowledge of VQA, overview of the state-of-the-art IQAs in the literature, resource of VQA, standards developed for subjective quality assessment, and summary of all related items.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
ITU-R Recommendation BT.500-11 (2002) Methodology for the subjective assessment of the quality of television pictures. ITU, Geneva, Switzerland
ITU-R Recommendation BT.710-4 (1998) Subjective assessment methods for image quality in high-definition television. ITU, Geneva, Switzerland
ITU-T Recommendation P.910 (2008) Subjective video quality assessment methods for multimedia applications. ITU, Geneva, Switzerland
ITU-R Recommendation BT.814-1 (1994) Specification and alignment procedures for setting of brightness and contrast of displays. ITU
ITU-R Recommendation BT.1129-2 (1998) Subjective assessment of standard definition digital television (SDTV) systems. ITU
VQEG (2003) Final report from the video quality experts group on the validation of objective models of video quality assessment, phase II
Winkler S, Campos R (2003) Video quality evaluation for Internet streaming applications. In: Proceedings of SPIE/IS and T human vision and electronic imaging, vol 5007. Santa Clara, CA, 20–24 Jan 2003, pp 104–115
Muntean G-M, Perry P, Murphy L (2005) Subjective assessment of the quality-oriented adaptive scheme. IEEE Trans Broadcast 51(3):276–286
Ponomarenko N, Carli M, Lukin V, Egiazarian K, Astola J, Battisti F (2008) Color image database for evaluation of image quality metrics. In Proceedings of IEEE 10th workshop on multimedia signal processing, Cairns, AU, Oct 2008, pp 403–408
De Simone F, Naccari M, Tagliasacchi M, Dufaux F, Tubaro S, Ebrahimi T (2011) Subjective quality assessment of H.264/AVC video streaming with packet losses. EURASIP J Image Video Process 2011:1–12
Mantiuk R, Tomaszewska A, Mantiuk R (2012) Comparison of four subjective methods for image quality assessment. Comput Graph Forum 31:2478–2491
Gulliksen H, Tucker LR (1961) A general procedure for obtaining paired comparisons from multiple rank orders. Psychometrika 26:173–183
Silverstein DA, Farrell JE (2001) Efficient method for paired comparison. J Electron Image 10:394–398
Xu Q, Yao Y, Jiang T, Huang Q, Lin W, Yan B (2012) HodgeRank on random graphs for subjective video quality assessment. IEEE Trans Multimedia 14(3):844–857
van Dijk AM, Martens J-B, Watson AB (1995) Quality assessment of coded images using numerical category scaling. Proc SPIE 2451:90–101
LIVE Image Quality Assessment Database. http://live.ece.utexas.edu/research/quality/subjective.htm
Tampere Image Database (2008). http://www.ponomarenko.info/tid2008.htm
Ponomarenko N, Jin L, Ieremeiev O, Lukin V, Egiazarian K, Astola J, Vozel B, Chehdi K, Carli M, Battisti F, Kuo C-CJ (2014) Image database TID2013: peculiarities, results and perspectives. Signal Process Image Commun 50:57–77
Tampere Image Database (2013). http://www.ponomarenko.info/tid2013.htm
Categorical Image Quality (CSIQ) Database. http://vision.okstate.edu/csiq
IVC Image Quality Database. http://www2.irccyn.ec-nantes.fr/ivcdb
IVC-LAR Database. http://www.irccyn.ecnantes.fr/autrusse/Databases/LAR
Toyoma Database. http://mict.eng.utoyama.ac.jp/mictdb.html
Wireless Imaging Quality (WIQ) Database. http://www.bth.se/tek/rcg.nsf/pages/wiq-db
A57 Database. http://foulard.ece.cornell.edu/dmc27/vsnr/vsnr.html
MMSP 3D Image Quality Assessment Database. http://mmspg.epfl.ch/cms/page-58394.html
Image Retargeting Subjective Database. http://ivp.ee.cuhk.edu.hk/projects/demo/retargeting/index.html
Vqeg, FRTV Phase I Database (2000). ftp://ftp.crc.ca/crc/vqeg/TestSequences/
IRCCyN/IVC 1080i Database. http://www.irccyn.ec-nantes.fr/spip.php?article541
IRCCyN/IVC SD RoI Database. http://www.irccyn.ec-nantes.fr/spip.php?article551
EPFL-PoliMI Video Quality Assessment Database. http://vqa.como.polimi.it/
LIVE Video Quality Database. http://live.ece.utexas.edu/research/quality/live_video.html
LIVE Wireless Video Quality Assessment Database. http://live.ece.utexas.edu/research/quality/live_wireless_video.html
MMSP 3D Video Quality Assessment Database. http://mmspg.epfl.ch/3dvqa
MMSP Scalable Video Database. http://mmspg.epfl.ch/svd
Vqeg, HDTV Database. http://www.its.bldrdoc.gov/vqeg/projects/hdtv/
IVP Subjective Quality Video Database. http://ivp.ee.cuhk.edu.hk/research/database/subjective/
Lin JY, Song R, Wu C-H, Liu T-J, Wang HQ, Kuo C-CJ (2015) MCL-V: a streaming video quality assessment database. J Vis Commun Image Represent 30:1–9
MCL-V Video Streaming Database. http://mcl.usc.edu/mcl-v-database/
Consumer Digital Video Library. http://www.cdvl.org/
Liu T-J, Lin Y-C, Lin W, Kuo C-CJ (2013) Visual quality assessment: recent developments, coding applications and future trends. APSIPA Trans Sig Inf Process 2, e4:1–20
Engelke U, Zepernick HJ (2007) Perceptual-based quality metrics for image and video services: a survey. In: The 3rd EuroNGI conference on next generation internet networks, May 2007, pp 190–197
Winkler S, Mohandas P (2008) The evolution of video quality measurement: from PSNR to hybrid metrics. IEEE Trans Broadcast 54(3):660–668
Lin W, Kuo C-CJ (2011) Perceptual visual quality metrics: a survey. J Vis Commun Image Represent 22(4):297–312
Marziliano P, Dufaux F, Winkler S, Ebrahimi T (2002) A no-reference perceptual blur metric. In: Proceedings of IEEE ICIP, Sept 2002, pp 57–60
Pinson MH, Wolf S (2004) A new standardized method for objectively measuring video quality. IEEE Trans Broadcast 50(3):312–322
Wang Z, Bovik A, Sheikh H, Simoncelli E (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612
Teo PC, Heeger DJ (1994) Perceptual image distortion. In: Proceedings of IEEE international conference on image processing, vol 2, pp 982–986
Winkler S (2005) Digital video quality: vision models and metrics. Wiley, New York
Masry MA, Hemami SS (2004) A metric for continuous quality evaluation of compressed video with severe distortions. Signal Process Image Commun 19(2):133–146
Masry M, Hemami SS, Sermadevi Y (2006) A scalable wavelet-based video distortion metric and applications. IEEE Trans Circuits Syst Video Technol 16(2):260–273
Sheikh HR, Bovik AC, de Veciana G (2005) An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans Image Process 14(12):2117–2128
Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15(2):430–444
Luo H (2004) A training-based no-reference image quality assessment algorithm. In: Proceedings of IEEE international conference on image processing
Suresh S, Babu V, Sundararajan N (2006) Image quality measurement using sparse extreme learning machine classifier. In: Proceedings of IEEE ICARCV
Narwaria M, Lin W (2010) Objective image quality assessment based on support vector regression. IEEE Trans Neural Netw 21(3):515–519
Liu T-J, Lin W, Kuo C-CJ (2011) A multi-metric fusion approach to visual quality assessment. In: IEEE the 3rd international workshop on QoMEX, Sep 2011
Narwaria M, Lin W (2011) Video quality assessment using temporal quality variations and machine learning. In: IEEE ICME2011, July 2011
Wang Z, Bovik A (2009) Mean squared error: love it or leave it? IEEE Signal Process Mag 26:98–117
Winkler S, Mohandas P (2008) The evolution of video quality measurement: from PSNR to hybrid metrics. IEEE Trans Broadcast 54(3):660–668
Daly S (1993) The visible differences predictor: an algorithm for the assessment of image fidelity. In: Watson AB (ed) Digital image human visual. MIT Press, Cambridge, pp 179–206
Lubin J (1995) A visual discrimination model for imaging system design and evaluation. In: Peli E (ed) Vision models for target detection and recognition. World Scientific, Singapore
Watson AB, Hu J, McGowan JF III (2001) DVQ: a digital video quality metric based on human vision. J Electron Image 10(1):20–29
Winkler S (1999) A perceptual distortion metric for digital color video. Proc SPIE 3644:175–184
Wolf S (1997) Measuring the end-to-end performance of digital video systems. IEEE Trans Broadcast 43(3):320–328
Wang Z, Bovik AC, Evan BL (2002) Blind measurement of blocking artifacts in images. Proc Int Conf Image Process 3:981–984
Miyahara M, Kotani K, Algazi VR (1998) Objective picture quality scale (PQS) for image coding. IEEE Trans Commun 46(9):1215–1225
Marziliano P, Dufaux F, Winkler S, Ebrahimi T (2002) A no-reference perceptual blur metric. In: Proceedings of IEEE international conference on image processing (ICIP 02), vol 3, Sept 2002, pp 57–60
Wu HR, Yuen M (1997) A generalized block-edge impairment metric (GBIM) for video coding. IEEE Signal Process Lett 4(11):317–320
Tan KT, Ghanbari M (2000) A multimetric objective picture-quality measurement model for MPEG video. IEEE Trans Circuits Syst Video Technol 10(7):1208–1213
Yu Z, Wu HR, Winkler S, Chen T (2002) Vision-model-based impairment metric to evaluate blocking artifacts in digital video. Proc IEEE 90:154–169
Watson AB, Solomon JA (1997) Model of visual contrast gain control and pattern masking. J Opt Soc Am A 14(9):2379–2391
Sheikh HR, Bovik AC (2002) Image information and visual quality. IEEE Trans Image Process 15(2):430–444
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612
Sheikh HR, Bovik AC, de Veciana G (2005) An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans Image Process 14(12):2117–2128
Horita Y, Miyata T, Gunawan IP, Murai T, Ghanbari M (2003) Evaluation model considering static-temporal quality degradation and human memory for sscqe video quality. Proc SPIE Vis Commun Image Process 5150(11):1601–1611
Dijk J, van Grinkel M, van Asselt RJ, van Vliet LJ, Verbeek PW (2003) A new sharpness measure based on gaussian lines and edges. In: Proceedings of the internatioanl conference on computational analysis on images and patterns (CAIP). Lecture Notes in Computer Science, vol 2756. Springer, pp 149–156
Ong E, Lin W, Lu Z, Yao S, Yang X, Jiang L (2003) No reference JPEG-2000 image quality metric. In: Proceedings of IEEE international conference multimedia and expo (ICME), pp 545–548
Faugeras OD (1979) Digital color image processing within the framework of a human visual model. IEEE Trans Acoust Speech Signal Process 27:380–393
Mannos J, Sakrison D (1974) The effects of a visual fidelity criterion of the encoding of images. IEEE Trans Inform Theory 20(4):525–536
Lukas F, Budrikis Z (1982) Picture quality prediction based on a visual model. IEEE Trans Commun 30:1679–1692
Tong X, Heeger D, Lambrecht CVDB (1999) Video quality evaluation using STCIELAB. SPIE Proc Hum Vis Vis Process Digit Disp 3644:185–196
Sarnoff Corporation (1997) Sarnoff JND vision model. In: Lubin J (ed) Contribution to IEEE G-2.1.6 compression and processing subcommittee
Burt PJ, Adelson EH (1983) The laplacian pyramid as a compact image code. IEEE Trans Commun 31(4):532–540
van den Branden Lambrecht CJ (1996) Perceptual models and architectures for video coding applications. Ph.D. dissertation, Swiss Federal Institute of Technology, Lausanne, Switzerland
Ong E, Lin W, Lu Z, Yao S, Etoh M (2004) Visual distortion assessment with emphasis on spatially transitional regions. IEEE Trans Circuits Syst Video Technol 14(4):559–566
Masry MA, Hemami SS, Sermadevi Y (2006) A scalable wavelet-based video distortion metric and applications. IEEE Trans Circuit Syst Video Technol 16(2):260–273
Zhang Z, Zhang J, Wang X, Guan Q, Chen S (2014) Image quality assessment based on structural saliency. In: 19th International conference on digital signal processing (DSP), Aug 2014, pp 492–496
Hou W, Gao X (2014) Saliency-guided deep framework for image quality assessment. IEEE Trans Multimedia 16(3):785–795
Zhang L, Shen Y, Li H (2014) A visual saliency-induced index for perceptual image quality assessment. IEEE Trans Image Process 23(10):4270–4281
Watson AB, Solomon JA (1997) Model of visual contrast gain control and pattern masking. J Opt Soc Am A 14(9):2379–2391
Winkler S (2000) Vision models and quality metrics for image processing applications. Ecole Polytecnique Federale De Lausanne (EPFL), Swiss Federal Institute of Technology, Lausanne, Switzerland, Thesis 2313, Dec 2000
Winkler S, Dufaux F (2003) Video quality evaluation for mobile applications. In: Proceedings of SPIE/IS T visual communication image processing, vol 5150, pp 593–603
Caviedes J, Gurbuz S (2002) No-reference sharpness metric based on local edge kurtosis. Proc IEEE Int Conf Image Process (ICIP) 3:53–56
Lin W, Dong L, Xue P (2005) Visual distortion gauge based on discrimination of noticeable contrast changes. IEEE Trans Circuits Syst Video Technol 15(7):900–909
Wang Z, Bovik AC (2004) A universal image quality index. IEEE Signal Process Lett 9(3):81–84
Dosselmann R, Yang XD (2010) A comprehensive assessment of the structural similarity index. Signal Image Video Process 5(1):81–91
Liu A, Lin W, Narwaria M (2012) Image quality assessment based on gradient similarity. IEEE Trans Image Process 21(4):1500–1512
Zhang L, Zhang L, Mou X, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386
Chen Q, Xu Y, Li C, Liu N (2013) An image quality assessment metric based on quaternion wavelet transform. In: IEEE international conference on multimedia and expo worshops (ICMEW), July 2013
Zhang XD, Feng XC, Wang WW, Xue WF (2013) Edge strength similarity for image quality assessment. IEEE Signal Process Lett 20(4):319–322
Wang Y, Jiang TT, Ma SW, Gao W (2012) Novel spatio-temporal structural information based video quality metric. IEEE Trans Circuits Syst Video Technol 22(7):989–998
Qi H, Jiao S, Lin W, Tang L, Shen W (2014) Content-based image quality assessment using semantic information and luminance differences. Electron Lett 50(20):1435–1436
Eskicioglu AM, Gusev A, Shnayderman A (2006) An SVD-based gray-scale image quality measure for local and global assessment. IEEE Trans Image Process 15(2):422–429
Narwaria M, Lin W (2009) Scalable image quality assessment based on structural vectors. In: Proceedings of IEEE workshop on multimedia signal processing (MMSP)
Wolf S, Pinson MH (2002) Video quality measurement techniques. NTIA Report 02-392, June 2002. http://www.its.bldrdoc.gov/publications/2423.aspx
Video Quality Expert Group (VQEG) (2003) Final report from the video quality experts group on the validation of objective models of video quality assessment. Phase II. http://www.vqeg.org. Accessed Aug 2003
Tan KT, Ghanbari M (2000) Blockiness detection for MPEG2-coded video. IEEE Signal Proc Lett 7(8):213–215
Wu S, Lin W, Xie S, Lu Z, Ong E, Yao S (2009) Blind blur assessment for visionbased applications. J Vis Commun Image Represent 20(4):231–241
Chen JYC, Thropp JE (2007) Review of low frame rate effects on human performance. IEEE Trans Systems Man Cybern 37:1063–1076
Yang KC, Guest CC, El-Maleh K, Das PK (2007) Perceptual temporal quality metric for compressed video. IEEE Trans Multimedia 9:1528–1535
Marziliano P, Winkler S, Dufaux F, Ebrahimi T (2004) Perceptual blur and ringing metrics: application to JPEG2000. Signal Process Image Commun 19:163–172
Mei T, Hua X-S, Zhu C-Z, Zhou H-Q, Li S (2007) Home video visual quality assessment with spatiotemporal factors. IEEE Trans Circuits Syst Video Technol 17(6):699–706
Oelbaum T, Keimel C, Diepold K (2009) Rule-based no-reference video quality evaluation using additionally coded videos. IEEE J Sel Top Signal Process 3(2):294–303
Zhai G, Zhang W, Yang X, Lin W, Xu Y (2008) No-reference noticeable blockiness estimation in images. Signal Process Image Commun 23(6):417–432
Ferzli R, Karam LJ (2009) A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB). IEEE Trans Image Process 18(4):717–728
Coudoux F-X, Gazalet MG, Derviaux C, Corlay P (2001) Picture quality measurement based on block visibility in discrete cosine transform coded video sequences. J Electron Imaging 10(2):498–510
Liang L, Wang S, Chen J, Ma S, Zhao D, Gao W (2010) No-reference perceptual image quality metric using gradient profiles for JPEG2000. Signal Process Image Commun 25(7):502–516
Moorthy AK, Bovik AC (2011) Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans Image Process 20(12):3350–3364
Tang H, Joshi N, Kapoor A (2011) Learning a blind measure of perceptual image quality. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), Colorado Springs
Zhang M, Muramastsu C, Zhou X, Hara T, Fujita H (2015) Blind image quality assessment using the joint statistics of generalized local binary patter. IEEE Trans Signal Process (Lett) 22(2):207–210
Xue W, Mou X, Zhang L, Bovik AC, Feng X (2014) Blind image quality assessment using joint statistics of gradient magnitude and Laplacian features. IEEE Trans Image Process 23(11):4850–4862
Gao X, Cao F, Tao D, Li X (2013) Universal blind image quality assessment metrics via natural scene statistics and multiple kernel learning. IEEE Trans Neural Netw Learn Syst 24(12):2013–2026
Liu T-J, Lin W, Kuo C-CJ (2013) Image quality assessment using multi-metric fusion (MMF). IEEE Trans Image Process 22(5):1793–1807
Liu T-J, Lin W, Kuo C-CJ (2011) A multi-metric fusion approach to visual quality assessment. In: QoMEX, Sept 2011, pp 72–77
Liu T-J, Lin Y-C, Lin W, Kuo C-CJ. Image quality assessment using paraboosting ensemble. Submitted to Neural Netw Learn Syst
Saad M, Bovik AC, Charrier C (2012) Blind image quality assessment: a natural scene statistics approach in the DCT domain. IEEE Trans Image Process 21(8):3339–3352
Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708
Mittal A, Muralidhar GS, Ghosh J, Bovik AC (2011) Blind image quality assessment without human training using latent quality factors. IEEE Signal Process Lett 19:75–78
Mittal A, Soundararajan R, Bovik AC (2013) Making a “completely blind” image quality analyzer. IEEE Signal Process Lett 20(3):209–212
Narwaria M, Lin W, Cetin E (2011) Scalable image quality assessment with 2D mel-cepstrum and machine learning approach. Pattern Recogn 45(1):299–313
Narwaria M, Lin W (2010) Objective image quality assessment based on support vector regression. IEEE Trans Neural Netw 21(3):515–519
Narwaria M, Lin W (2012) SVD-based quality metric for image and video using machine learning. IEEE Trans Syst Man Cybern Part B 42(2):347–364
Xu L, Lin W, Li J, Wang X, Yan Y, Fang Y (2014) Rank learning on training set selection and image quality assessment. In: ICME2014
Xu Q, Yao Y, Jiang T, Huang Q, Lin W, Yan B (2012) HodgeRank on random graphs for subjective video quality assessment. IEEE Trans Multimedia 14(3):844–857
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2015 The Author(s)
About this chapter
Cite this chapter
Xu, L., Lin, W., Kuo, CC.J. (2015). Introduction. In: Visual Quality Assessment by Machine Learning. SpringerBriefs in Electrical and Computer Engineering(). Springer, Singapore. https://doi.org/10.1007/978-981-287-468-9_1
Download citation
DOI: https://doi.org/10.1007/978-981-287-468-9_1
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-287-467-2
Online ISBN: 978-981-287-468-9
eBook Packages: EngineeringEngineering (R0)