Advertisement

Multimedia Tools and Applications

, Volume 78, Issue 22, pp 31019–31033 | Cite as

metrics and methods of video quality assessment: a brief review

  • Qiang Fan
  • Wang LuoEmail author
  • Yuan Xia
  • Guozhi Li
  • Daojing He
Article

Abstract

With rapid development of video acquisition devices and wide applications of video data, more and more requirements are established to use video data query. video quality assessment and improvement become popular and important research issues which attract lots of researchers. The video quality can influence technique application, user experience, and application results. This paper firstly reviews research work on video query based applications. Then, various metrics of video quality assessment are reviewed according to the requirement of reference video, including full-reference metrics, no-reference metrics and reduced-reference metrics. In addition, methods of video quality assessment are reviewed by the features, which include visual features, bitstream-based or packet-based features, data features. Finally, a number of video quality improvement methods are briefly introduced.

Keywords

Quality assessment Assessment metrics Assessment method Video query 

Notes

Acknowledgment

This research is supported by state grid corporation science and technology project “The pilot application on network access security for patrol data captured by unmanned planes and robots and intelligent recognition base on big data platform”.

References

  1. 1.
    Agrawal R, Faloutsos C, Swami A (1993) Efficient similarity search in sequence data basesGoogle Scholar
  2. 2.
    Aloshious AB, Sreelekha G (2011) Quality improvement of h. 264/avc frext by incorporating perceptual models. In: Proceedings of annual IEEE conference on india conference (INDICON), pp 1–6Google Scholar
  3. 3.
    Andrade H, Kurc T, Sussman A, Saltz J (2004) Optimizing the execution of multiple data analysis queries on parallel and distributed environments. IEEE Trans Parallel Distrib Syst 15(6):520–532Google Scholar
  4. 4.
    Anegekuh L, Sun L, Jammeh E, ISHM, Ifeachor E (2015) Content-based video quality prediction for hevc encoded videos streamed over packet networks. IEEE Trans Multimed 17(8):1323–1334Google Scholar
  5. 5.
    Atenas M, Garcia M, Canovas A, Lloret J (2010) A mpeg-2/mpeg-4 quantizer to improve the video quality in iptv services. In: Proceedings of the 6th international conference on networking and services (ICNS), pp 49–54Google Scholar
  6. 6.
    Bhat A, Kannangara S, Zhao Y, Richardson I (2012) A full reference quality metric for compressed video based on mean squared error and video content. IEEE Trans Circuits Syst Video Technol 22(2):165–173Google Scholar
  7. 7.
    Boujut H, Benois-Pineau J, Ahmed T, Hadar O, Bonnet P (2011) A metric for no-reference video quality assessment for hd tv delivery based on saliency maps. In: IEEE international conference on multimedia and expo, pp 1–5Google Scholar
  8. 8.
    Chan SSM, Li Q, Wu Y, Zhuang Y (2002) Accommodating hybrid retrieval in a comprehensive video data base management system. IEEE Trans Multimed 4(2):146–159Google Scholar
  9. 9.
    Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: A classification, review, and performance comparison. IEEE Trans Broadcast 57(2):165–182Google Scholar
  10. 10.
    Choudary C, Liu T, Huang C-T (2007) Semantic retrieval of instructional videos. In: Proceedings of the 9th IEEE international symposium on multimedia workshops, pp 277–282Google Scholar
  11. 11.
    Dubravko U, Milan M, Vladimir Z, Maja P, Vladimir C, Dragan K (2011) Salient motion features for video quality assessment. IEEE Trans Image Process 20(4):948–958MathSciNetzbMATHGoogle Scholar
  12. 12.
    Espina F, Morato D, Izal M, Magana E (2011) Improving video quality in network paths with bursty losses. In: Proceedings of IEEE conference on global telecommunications conference (GLOBECOM 2011), pp 1–6Google Scholar
  13. 13.
    Feng X, Liu T, Yang D, Wang Y (2008) Saliency based objective quality assessment of decoded video affected by packet losses. In: Proceedings of the 15th IEEE international conference on image processing (ICIP), pp 2560–2563Google Scholar
  14. 14.
    Feng X, Liu T, Yang D, Wang Y (2011) Saliency inspired full-reference quality metrics for packet-loss-impaired video. IEEE Trans Broadcast 57(1):81–88Google Scholar
  15. 15.
    Gunawan IP, Ghanbari M (2008) Reduced-reference video quality assessment using discriminative local harmonic strength with motion consideration. IEEE Trans Circuits Syst Video Technol 18(1):71–83Google Scholar
  16. 16.
    Ha K, Kim M (2011) A perceptual quality assessment metric using temporal complexity and disparity information for stereoscopic video. In: Proceedings of the 18th IEEE international conference on image processing (ICIP), pp 2525–2528Google Scholar
  17. 17.
    Hee M, Ik YY, Kim KC (1999) Hybrid video system supporting content-based retrieval. In: Proceedings of the 3rd international conference on computational intelligence and multimedia applications (ICCIMA), pp 258–262Google Scholar
  18. 18.
    Hewage CTER, Martini MG (2010) Reduced-reference quality evaluation for compressed depth maps associated with colour plus depth 3d video. In: Proceedings of the 17th IEEE international conference on image processing (ICIP), pp 4017–4020Google Scholar
  19. 19.
    Hirai K, Tumurtogoo J, Kikuchi A, Tsumura N, Nakaguchi T, Miyake Y (2010) Video quality assessment using spatio-velocity contrast sensitivity function. IEICE Trans Inf Syst 93(5):1253–1262Google Scholar
  20. 20.
    Jain A, Bhateja V (2011) A full-reference image quality metric for objective evaluation in spatial domain. In: International conference on communication and industrial application (ICCIA), pp 1–5Google Scholar
  21. 21.
    Jenkins C, Inman D (2000) Server-side automatic metadata generation using qualified dublin core and rdf. In: Proceedings of international conference on digital libraries: research and practice, pp 262– 269Google Scholar
  22. 22.
    Kalpana S, Conrad BA (2010) Motion tuned spatio-temporal quality assessment of natural videos. IEEE Trans Image Process 19(2):335–350MathSciNetzbMATHGoogle Scholar
  23. 23.
    Kalpana S, Rajiv S, Conrad BA, Cormack LK (2010) Study of subjective and objective quality assessment of video. IEEE Trans Image Process 19(6):1427–1441MathSciNetzbMATHGoogle Scholar
  24. 24.
    Karacali B, Krishnakumar AS (2012) Measuring video quality degradation using face detection. In: Proceedings of the 35th IEEE transactions on sarnoff symposium (SARNOFF), pp 1–5Google Scholar
  25. 25.
    Keimel C, Klimpke M, Habigt J, Diepold K (2011) No-reference video quality metric for hdtv based on h.264/avc bitstream features. In: IEEE international conference on image processing, pp 3325– 3328Google Scholar
  26. 26.
    Kim D, Ryu S, Sohn K (2012) Depth perception and motion cue based 3d video quality assessment. In: Proceedings of IEEE international symposium on broadband multimedia systems and broadcasting (BMSB), pp 1–4Google Scholar
  27. 27.
    Kriegel H-P, Kroger P, Kunath P, Pryakhin A (2006) Effective similarity search in multimedia data bases using multiple representations. In: Proceedings of the 12th international multi-media modelling conference proceedings, p 4Google Scholar
  28. 28.
    Kwok SH, Leon Zhao J (2006) Content-based object organization for efficient image retrieval in image data bases. Decis Support Syst 42(3):1901–1916Google Scholar
  29. 29.
    Lee S-O, Jung K-S, Sim D-G (2010) Real-time objective quality assessment based on coding parameters extracted from h. 264/avc bitstream. IEEE Trans Consum Electron 56(2):1071–1078Google Scholar
  30. 30.
    Leszczuk M, Janowski L, Romaniak P, Papir Z (2013) Assessing quality of experience for high definition video streaming under diverse packet loss patterns. Signal Process Image Commun 28(8):903–916Google Scholar
  31. 31.
    Leszczuk M, Kowalczyk K, Janowski L, Papir Z (2015) Lightweight implementation of no-reference (nr) perceptual quality assessment of h. 264/avc compression. Signal Process Image Commun 39:457– 465Google Scholar
  32. 32.
    Li C, Bovik AC (2010) Content-weighted video quality assessment using a three-component image model. J Electron Imaging 19(1):143–153Google Scholar
  33. 33.
    Li J, Xia Y, Shan Z, Liu Y (2015) Scalable constrained spectral clustering. IEEE Trans Knowl Data Eng 27(2):589–593Google Scholar
  34. 34.
    Ma Q, Zhang L, Wang B (2010) New strategy for image and video quality assessment. J Electron Imaging 19(1):011019–011019Google Scholar
  35. 35.
    Maalouf A, Larabi MC (2010) A no-reference color video quality metric based on a 3d multispectral wavelet transform. In: Proceedings of international workshop on quality of multimedia experience (QoMEX), pp 11–16Google Scholar
  36. 36.
    Martínez JM, Pereira F (2002) Mpeg-7: the generic multimedia content description standard, part 1. IEEE Trans Multimed 9(2):78–87Google Scholar
  37. 37.
    Matsubara FM, Hanada T, Imai S, Miura S, Akatsu S (2009) Managing a media server content directory in absence of reliable metadata. IEEE Trans Consum Electron 55(2):873–877Google Scholar
  38. 38.
    Narvekar ND, Karam LJ (2009) A no-reference perceptual image sharpness metric based on a cumulative probability of blur detection. In: Proceedings of international workshop on quality of multimedia experience (QoMEx), pp 87–91Google Scholar
  39. 39.
    Narwaria M, Lin W, Liu A (2012) Low-complexity video quality assessment using temporal quality variations. IEEE Trans Multimed 14(3):525–535Google Scholar
  40. 40.
    Nezhadarya E, Ward RK (2013) Semi-blind quality estimation of compressed videos using digital water marking. Digital Signal Process 23(5):1483–1495MathSciNetGoogle Scholar
  41. 41.
    Quan H-T, Ghanbari M (2010) Modelling of spatio–temporal interaction for video quality assessment. Signal Process Image Commun 25(7):535–546Google Scholar
  42. 42.
    Rasli RM, Haw S-C, Wong C-O (2010) A survey on optimizing video and audio query retrieval in multimedia data bases. In: Proceedings of the 3rd international conference on advanced computer theory and engineering (ICACTE), vol 2, pp V2–302Google Scholar
  43. 43.
    Ribeiro C, David G, Calistru C (2004) A multimedia data base workbench for content and context retrieval. In: Proceedings of the 6th IEEE workshop on multimedia signal processing, pp 430– 433Google Scholar
  44. 44.
    Sadat ABMRI, Rubaiyat Islam BM, Lecca P (2009) On the performances in simulation of parallel data bases: an overview on the most recent techniques for query optimization. In: Proceedings of international workshop on high performance computational systems biology (HIBI), pp 113–117Google Scholar
  45. 45.
    Seshadrinathan K, Bovik AC (2009) Motion-based perceptual quality assessment of video. In: IS&T/SPIE electronic imaging, pp 72400X–72400XGoogle Scholar
  46. 46.
    Shan Z, Xia Y, Hou P, He J (2016) Fusing incomplete multisensor heterogeneous data to estimate urban traffic. IEEE MultiMedia 23(3):56–63Google Scholar
  47. 47.
    Shen HT, Ooi BC, Zhou X (2005) Towards effective indexing for very large video sequence data base. In: Proceedings of ACM SIGMOD international conference on management of data, pp 730–741Google Scholar
  48. 48.
    Shenoy ST, Ozsoyoglu ZM (1989) Design and implementation of a semantic query optimizer. IEEE Trans Knowl Data Eng 1(3):344–361Google Scholar
  49. 49.
    Steinacker A, Ghavam A, Steinmetz R (2001) Metadata standards for web-based resources. IEEE Trans Multimed 8(1):70–76Google Scholar
  50. 50.
    Tagliasacchi M, Valenzise G, Naccari M, Tubaro S (2010) A reduced-reference structural similarity approximation for videos corrupted by channel errors. Multimed Tools Appl 48(3):471–492Google Scholar
  51. 51.
    Torkamani-Azar F, Imani H, Fathollahian H (2015) Video quality measurement based on 3-d. singular value decomposition. J Vis Commun Image Represent 27:1–6Google Scholar
  52. 52.
    Vranješ M, Rimac-Drlje S, Grgić K (2013) Review of objective video quality metrics and performance comparison using different data bases. Image Commun 28(1):1–19Google Scholar
  53. 53.
    Wang A, Jiang G, Wang X, Yu M (2009) New no-reference blocking artifacts metric based on human visual system. In: Proceedings of international conference on wireless communications & signal processing (WCSP), pp 1–5Google Scholar
  54. 54.
    Wang S-H, Chen W-L, Chiang T (2007) An efficient fgs to mpeg-1/2/4 single layer transcoder with r-d optimized multi-layer streaming technique for video quality improvement. J Chin Inst Eng 30(6):1059–1070Google Scholar
  55. 55.
    Wang Z, Lu L, Bovik AC (2004) Video quality assessment based on structural distortion measurement. Signal Process Image Commun 19(2):121–132Google Scholar
  56. 56.
    Weibel S (1997) The dublin core: a simple content description model for electronic resources. Bull Am Soc Inf Sci Technol 24(1):9–11Google Scholar
  57. 57.
    Wichterich M, Assent I, Kranen P, Seidl T (2008) Efficient emd-based similarity search in multimedia data bases via flexible dimensionality reduction. In: Proceedings of ACM SIGMOD international conference on management of data, pp 199–212Google Scholar
  58. 58.
    Xia T, Mei T, Hua G, Zhang YD, Hua XS (2010) Visual quality assessment for web videos. J Vis Commun Image Represent 21(8):826–837Google Scholar
  59. 59.
    Xia Y, Chen J, Li J, Zhang Y (2016) Geometric discriminative features for aerial image retrieval in social media. Multimed Syst 22(4):497–507Google Scholar
  60. 60.
    Xia Y, Chen J, Lu X, Wang C, Xu C (2016) Big traffic data processing framework for intelligent monitoring and recording systems. Neurocomputing 181:139–146Google Scholar
  61. 61.
    Xia Y, Nie L, Zhang L, Yang Y, Hong R, Li X (2016) Weakly supervised multilabel clustering and its applications in computer vision. IEEE Trans Cybern 46 (12):3220–3232Google Scholar
  62. 62.
    Xia Y, Chen W, Liu X, Zhang L, Li X, Xiang Y (2017) Adaptive multimedia data forwarding for privacy preservation in vehicular ad-hoc networks. IEEE Trans Intell Transp SystGoogle Scholar
  63. 63.
    Xia Y, Liu Z, Yan Y, Chen Y, Zhang L, Zimmermann R (2017) Media quality assessment by perceptual gaze-shift patterns mining. IEEE Trans MultimedGoogle Scholar
  64. 64.
    Xia Y, Ren X, Peng Z, Zhang J, She L (2016) Effectively identifying the influential spreaders in large-scale social networks. Multimed Tools Appl 75(15):8829–8841Google Scholar
  65. 65.
    Xia Y, Shi X, Song G, Geng Q, Liu Y (2016) Towards improving quality of video-based vehicle counting method for traffic flow estimation. Signal Process 120:672–681Google Scholar
  66. 66.
    Xia Y, Zhang L, Hong R, Nie L, Yan Y, Shao L (2017) Perceptually guided photo retargeting. IEEE Trans Cybern 47(3):566–578Google Scholar
  67. 67.
    Xia Y, Zhang L, Tang S (2014) Large-scale aerial image categorization by multi-task discriminative topologies discovery. In: Proceedings of the first international workshop on internet-scale multimedia management, pp 53–58. ACMGoogle Scholar
  68. 68.
    Xia Y, Zhang L, Xu W, Shan Z, Liu Y (2015) Recognizing multi-view objects with occlusions using a deep architecture. Inf Sci 320:333–345MathSciNetGoogle Scholar
  69. 69.
    Xia Y, Zhu M, Gu Q, Zhang L, Li X (2016) Toward solving the steiner travelling salesman problem on urban road maps using the branch decomposition of graphs. Inf Sci 374:164–178Google Scholar
  70. 70.
    Xiang X, Shi Y, Guo L (2003) A conformance test suite of localized lom model. In: Proceedings of the 3rd IEEE international conference on advanced learning technologies, pp 288–289Google Scholar
  71. 71.
    Yamamura Y, Iwasaki S, Matsuo Y, Katto J (2013) Quality assessment of compressed video sequenceSHaving blocking artifacts by cepstrum analysis. In: Proceedings of IEEE international conference on consumer electronics (ICCE), pp 494–495Google Scholar
  72. 72.
    Yao J, Xie Y, Tan J, Li Z, Qi J, Gao L (2012) No-reference video quality assessment using statistical features along temporal trajectory. Procedia Eng 29:947–951Google Scholar
  73. 73.
    Yoon J, Jayant N (2001) Relevance feedback for semantics based image retrieval. In: Proceedings of international conference on image processing, vol 1, pp 42–45Google Scholar
  74. 74.
    You J, Korhonen J, Perkis A (2010) Spatial and temporal pooling of image quality metrics for perceptual video quality assessment on packet loss streams. In: Proceedings of IEEE international conference on acoustics speech and signal processing (ICASSP), pp 1002–1005Google Scholar
  75. 75.
    You J, Korhonen J, Perkis A, Ebrahimi T (2011) Balancing attended and global stimuli in perceived video quality assessment. IEEE Trans Multimed 13 (6):1269–1285Google Scholar
  76. 76.
    Zeng K, Wang Z (2010) Temporal motion smoothness measurement for reduced-reference video quality assessment. In: ICASSP, pp 1010–1013Google Scholar
  77. 77.
    Zhang L, Gao Y, Ji R, Xia Y, Dai Q, Li X (2014) Actively learning human gaze shifting paths for semantics-aware photo cropping. IEEE Trans Image Process 23(5):2235–2245MathSciNetzbMATHGoogle Scholar
  78. 78.
    Zhang L, Gao Y, Xia Y, Dai Q, Li X (2015) A fine-grained image categorization system by cellet-encoded spatial pyramid modeling. IEEE Trans Ind Electron 62(1):564–571Google Scholar
  79. 79.
    Zhang L, Gao Y, Xia Y, Lu K, Shen J, Ji R (2014) Representative discovery of structure cues for weakly-supervised image segmentation. IEEE Trans Multimed 16 (2):470–479Google Scholar
  80. 80.
    Zhang L, Li X, Nie L, Yang Y, Xia Y (2016) Weakly supervised human fixations prediction. IEEE Trans Cybern 46(1):258–269Google Scholar
  81. 81.
    Zhang L, Liu X, Lu K (2014) Svd-based 3d image quality assessment by using depth information. In: Proceedings of IEEE 17th international conference on computational science and engineering (CSE)Google Scholar
  82. 82.
    Zhang L, Liu Z, Nie L, Li X et al (2016) Weakly-supervised multimodal kernel for categorizing aerial photographs. IEEE Trans Image ProcessGoogle Scholar
  83. 83.
    Zhang L, Xia Y, Ji R, Li X (2015) Spatial-aware object-level saliency prediction by learning graphlet hierarchies. IEEE Trans Ind Electron 62(2):1301–1308Google Scholar
  84. 84.
    Zhang L, Xia Y, Mao K, Ma H (2015) An effective video summarization framework toward handheld devices. IEEE Trans Ind Electron 62(2):1309–1316Google Scholar
  85. 85.
    Zhou W, Dao S, Kuo C-CJ (2002) On-line knowledge-and rule-based video classification system for video indexing and dissemination. Inf Syst 27(8):559–586zbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.State Grid Electric Power Research InstituteNARI Group CorporationNanjingChina
  2. 2.School of Computer Science and Software EngineeringEast China Normal UniversityShanghaiChina

Personalised recommendations