Skip to main content
SpringerLink
Log in

Blind image quality assessment for anchor-assisted adaptation to practical situations

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Directly conveying human visual perception of image quality, the mean opinion score (MOS) has traditionally been the cornerstone of blind image quality assessment (BIQA) training. However, attempt to solve the inter-dataset differences of MOS was anecdotal so far. Scores from the local raters are inevitably affected by the habits, preferences and environments at that time, leading each dataset has its own characteristics, and the absolute values will not be comparable once separated from the specific IQA dataset. Towards overcoming the limitations, we propose a BIQA approach that combines ranking and anchor-assisted regression, which can be easily adapted to different goal-oriented environments or populations. The comparison with anchors abandons the traditional practice of using one score to fit all standards, and avoids the difference between datasets in principle. We also demonstrate that a receiver-based distortion classification method can simplify the cluttered existing distortions as well as support the ranking method. Experiments have proved that our approach achieves flexible adaptation to target situations by freely replacing anchors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The source data that support the findings of this study are available from Waterloo, and the generation method of distorted series is provided in the paper, so that readers can generate the dataset according to this.

References

  1. Bosse S, Maniry D, Wiegand T, Samek W (2016) A deep neural network for image quality assessment. In: 2016 IEEE international conference on image processing (ICIP) (pp. 3773-3777). IEEE

  2. Chen J, Zhang Y, Liang L, Ma S, Wang R, Gao W (2008) A no-reference blocking artifacts metric using selective gradient and plainness measures. In Pacific-rim conference on multimedia (pp. 894-897). Springer, Berlin, Heidelberg

  3. Chiu TY, Zhao Y, Gurari D (2020) Assessing image quality issues for real-world problems. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 3646-3656).

  4. Chopra S, Hadsell R, LeCun Y (2005, June) Learning a similarity metric discriminatively, with application to face verification. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05) (Vol. 1, pp. 539-546). IEEE.

  5. Fang Y, Zhu H, Zeng Y, Ma K, Wang Z (2020) Perceptual quality assessment of smartphone photography. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 3677-3686)

  6. Gao F, Tao D, Gao X, Li X (2015) Learning to rank for blind image quality assessment. IEEE Trans Neural Networks Learn Syst 26(10):2275–2290

    Article  MathSciNet  Google Scholar 

  7. Ghadiyaram D, Bovik AC (2015) Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans Image Process 25(1):372–387

    Article  MathSciNet  MATH  Google Scholar 

  8. Glickman ME, Jones AC (1999) Rating the chess rating system[J]. Chance-berlin then New York, 12:21–28

  9. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, … Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Proces Syst 27

  10. Gu K, Zhai G, Yang X, Zhang W (2014) Deep learning network for blind image quality assessment. In 2014 IEEE international conference on image processing (ICIP) (pp. 511-515). IEEE.

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778)

  12. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  MATH  Google Scholar 

  13. Hosu V, Goldlucke B, Saupe D (2019) Effective aesthetics prediction with multi-level spatially pooled features. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 9375-9383).

  14. Hosu V, Lin H, Sziranyi T, Saupe D (2020) KonIQ-10k: an ecologically valid database for deep learning of blind image quality assessment. IEEE Trans Image Process 29:4041–4056

    Article  MATH  Google Scholar 

  15. Hou W, Gao X, Tao D, Li X (2014) Blind image quality assessment via deep learning. IEEE Trans Neural Networks Learn Syst 26(6):1275–1286

    MathSciNet  Google Scholar 

  16. Kim J, Lee S (2016) Fully deep blind image quality predictor. IEEE J Select Topics Signal Process 11(1):206–220

    Article  Google Scholar 

  17. Kim J, Nguyen AD, Lee S (2018) Deep CNN-based blind image quality predictor. IEEE Trans Neural Networks Learn Syst 30(1):11–24

    Article  Google Scholar 

  18. Larson EC, Chandler DM (2010) Most apparent distortion: full-reference image quality assessment and the role of strategy. J Electronic Imaging 19(1):011006

    Article  Google Scholar 

  19. Li C, Bovik AC, Wu X (2011) Blind image quality assessment using a general regression neural network. IEEE Trans Neural Netw 22(5):793–799

    Article  Google Scholar 

  20. Li Y, Po LM, Xu X, Feng L, Yuan F, Cheung CH, Cheung KW (2015) No-reference image quality assessment with shearlet transform and deep neural networks. Neurocomputing 154:94–109

    Article  Google Scholar 

  21. Lin KY, Wang G (2018) Hallucinated-IQA: no-reference image quality assessment via adversarial learning. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 732-741)

  22. Liu W, Wang Z, Liu X, … Alsaadi FE (2017) A survey of deep neural network architectures and their applications[J]. Neurocomputing 234:11–26

    Article  Google Scholar 

  23. Liu X, Van De Weijer J, Bagdanov AD (2017) Rankiqa: learning from rankings for no-reference image quality assessment. In proceedings of the IEEE international conference on computer vision (pp. 1040-1049)

  24. Liu X, Van De Weijer J, Bagdanov AD (2019) Exploiting unlabeled data in cnns by self-supervised learning to rank. IEEE Trans Pattern Anal Mach Intell 41(8):1862–1878

    Article  Google Scholar 

  25. Lv Y, Jiang G, Yu M, Xu H, Shao F, Liu S (2015) Difference of Gaussian statistical features based blind image quality assessment: a deep learning approach. In 2015 IEEE international conference on image processing (ICIP) (pp. 2344-2348). IEEE.

  26. Ma K, Liu W, Liu T, Wang Z, Tao D (2017) dipIQ: blind image quality assessment by learning-to-rank discriminable image pairs. IEEE Trans Image Process 26(8):3951–3964

    Article  MathSciNet  MATH  Google Scholar 

  27. Ma K, Liu X, Fang Y, Simoncelli EP (2019) Blind image quality assessment by learning from multiple annotators. In 2019 IEEE international conference on image processing (ICIP) (pp. 2344-2348). IEEE.

  28. Meesters L, Martens JB (2002) A single-ended blockiness measure for JPEG-coded images. Signal Process 82(3):369–387

    Article  MATH  Google Scholar 

  29. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708

    Article  MathSciNet  MATH  Google Scholar 

  30. Mittal A, Soundararajan R, Bovik AC (2012) Making a “completely blind” image quality analyzer. IEEE Signal Process Lett 20(3):209–212

    Article  Google Scholar 

  31. Moorthy AK, Bovik AC (2010) A two-step framework for constructing blind image quality indices. IEEE Signal Process Lett 17(5):513–516

    Article  Google Scholar 

  32. Moorthy AK, Bovik AC (2011) Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans Image Process 20(12):3350–3364

    Article  MathSciNet  MATH  Google Scholar 

  33. Ong E, Lin W, Lu Z, Yao S, Yang X, Jiang L (2003) No-reference JPEG-2000 image quality metric. In 2003 international conference on multimedia and expo. ICME'03. Proceedings (cat. No. 03TH8698) (Vol. 1, pp. I-545). IEEE

  34. Ponomarenko N, Ieremeiev O, Lukin V, Egiazarian K, Jin L, Astola J, ..., Kuo CCJ (2013). Color image database TID2013: Peculiarities and preliminary results. In European workshop on visual information processing (EUVIP) (pp. 106–111). IEEE

  35. Prashnani E, Cai H, Mostofi Y, Sen P (2018) Pieapp: perceptual image-error assessment through pairwise preference. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1808-1817)

  36. Ruderman DL (1994) The statistics of natural images. Netw Comput Neural Syst 5(4):517–548

    Article  MATH  Google Scholar 

  37. Saad MA, Bovik AC, Charrier C (2010) A DCT statistics-based blind image quality index. IEEE Signal Process Lett 17(6):583–586

    Article  Google Scholar 

  38. Saad MA, Bovik AC, Charrier C (2012) Blind image quality assessment: a natural scene statistics approach in the DCT domain. IEEE Trans Image Process 21(8):3339–3352

    Article  MathSciNet  MATH  Google Scholar 

  39. Sheikh HR, Bovik AC, Cormack L (2005) No-reference quality assessment using natural scene statistics: JPEG2000. IEEE Trans Image Process 14(11):1918–1927

    Article  Google Scholar 

  40. Sheikh HR, Sabir MF, Bovik AC (2006) A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans Image Process 15(11):3440–3451

    Article  Google Scholar 

  41. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  42. Specht DF (1991) A general regression neural network. IEEE Trans Neural Netw 2(6):568–576

    Article  Google Scholar 

  43. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, ..., Rabinovich A (2015) Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1–9)

  44. Tang H, Joshi N, Kapoor A (2014) Blind image quality assessment using semi-supervised rectifier networks. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2877-2884)

  45. Thorpe S, Fize D, Marlot C (1996) Speed of processing in the human visual system[J]. Nature 381(6582):520–522

    Article  Google Scholar 

  46. Wang Z, Sheikh HR, Bovik AC (2002) No-reference perceptual quality assessment of JPEG compressed images. In proceedings. International conference on image processing (Vol. 1, pp. I-I). IEEE

  47. Xu L, Li J, Lin W, Zhang Y, Ma L, Fang Y, Yan Y (2016) Multi-task rank learning for image quality assessment. IEEE Trans Circ Syst Video Technol 27(9):1833–1843

    Article  Google Scholar 

  48. Xu J, Ye P, Li Q, Du H, Liu Y, Doermann D (2016) Blind image quality assessment based on high order statistics aggregation. IEEE Trans Image Process 25(9):4444–4457

    Article  MathSciNet  MATH  Google Scholar 

  49. Ying Z, Mandal M, Ghadiyaram D, Bovik A (2021) QPatch-VQ:'Patching Up'the video quality problem. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 14019-14029)

  50. Yu X, Bampis CG, Gupta P, Bovik AC (2019) Predicting the quality of images compressed after distortion in two steps. IEEE Trans Image Process 28(12):5757–5770

    Article  MathSciNet  MATH  Google Scholar 

  51. Zoph B, Le QV (2016) Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578

Download references

Acknowledgments

This work is supported by the Specialized Research Fund for the Doctoral Program of Higher Education of China,” Research of Visual Perception for Impairments of Color Information in High-Definition Images” (No.20110018110001).

Author information

Authors and Affiliations

  1. Information and Telecommunication Engineering Institute, Communication University of China, Beijing, China

    Li Xu & Xiuhua Jiang

  2. PengCheng Laboratory, Shenzhen, China

    Li Xu & Xiuhua Jiang

Authors
  1. Li Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiuhua Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Xu.

Ethics declarations

Conflicts of interest / competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 9.38 MB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, L., Jiang, X. Blind image quality assessment for anchor-assisted adaptation to practical situations. Multimed Tools Appl 82, 17929–17946 (2023). https://doi.org/10.1007/s11042-022-14225-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14225-9

Keywords

Search

Navigation