Skip to main content
SpringerLink
Log in

Blind Quality Assessment of Multiply Distorted Images Using Deep Neural Networks

  • Conference paper
  • First Online:
Image Analysis and Recognition (ICIAR 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11662))

Included in the following conference series:

Abstract

In real-world visual content acquisition and distribution systems, a vast majority of visual content undergoes multiple distortions between the source and the end user. However, traditional image quality assessment (IQA) algorithms are usually validated and at times trained on image databases with a single distortion stage. Existing IQA methods for multiply distorted images remain limited in their scope and performance. In this work we design a first-of-its-kind blind IQA model for multiply distorted visual content based on a deep end-to-end convolutional neural network. The network is trained on a newly developed dataset which is composed of millions of multiply distorted images annotated with synthetic quality scores. Our tests on three publicly available subject-rated multiply distorted image databases show that the proposed model outperforms state-of-the-art blind IQA methods in terms of both accuracy and speed.

This work is supported in part by the Natural Sciences and Engineering Research Council of Canada

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Wang, Z., Bovik, A.C.: Mean squared error: love it or leave it? A new look at signal fidelity measures. IEEE Signal Process. Mag. 26(1), 98–117 (2009)

    Article  Google Scholar 

  2. Wang, Z., Bovik, A.C.: Modern image quality assessment. Synth. Lect. Image Video Multimedia Process. 2(1), 1–156 (2006)

    Article  Google Scholar 

  3. Sheikh, H.R., Wang, Z., Cormack, L., Bovik, A.C.: LIVE Image Quality Assessment Database Release 2. http://live.ece.utexas.edu/research/Quality/subjective.htm

  4. Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 3440–3451 (2006)

    Article  Google Scholar 

  5. Ponomarenko, N., et al.: TID2008-a database for evaluation of full-reference visual quality assessment metrics. Adv. Modern Radioelectron. 10(4), 30–45 (2009)

    Google Scholar 

  6. Ponomarenko, N., et al.: Image database TID2013: peculiarities, results and perspectives. Signal Process Image Commun. 30, 57–77 (2015)

    Article  Google Scholar 

  7. Larson, E.C., Chandler, D.M.: Most apparent distortion: full-reference image quality assessment and the role of strategy. J. Electron. Imag. 19(1), 011 006:1–011 006:21 (2010)

    Google Scholar 

  8. Zarić, A., et al.: VCL@FER image quality assessment database. AUTOMATIKA 53(4), 344–354 (2012)

    Article  Google Scholar 

  9. Liu, X., Pedersen, M., Hardeberg, J.Y.: CID:IQ – a new image quality database. In: Elmoataz, A., Lezoray, O., Nouboud, F., Mammass, D. (eds.) ICISP 2014. LNCS, vol. 8509, pp. 193–202. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07998-1_22

    Chapter  Google Scholar 

  10. Chandler, D.M.: Seven challenges in image quality assessment: past, present, and future research. ISRN Signal Process. 2013, 1–53 (2013). Article ID 905685

    Article  Google Scholar 

  11. Jayaraman, D., Mittal, A., Moorthy, A.K., Bovik, A.C.: Objective quality assessment of multiply distorted images. In: Conference Record Asilomar Conference Signals, Systems, and Computers (ASILOMAR), Pacific Grove, CA, USA, pp. 1693–1697, November 2012

    Google Scholar 

  12. Gu, K., Zhai, G., Yang, X., Zhang, W.: Hybrid no-reference quality metric for singly and multiply distorted images. IEEE Trans. Broadcast. 60(3), 555–567 (2014)

    Article  Google Scholar 

  13. Sun, W., Zhou, F., Liao, Q.: MDID: a multiply distorted image database for image quality assessment. Pattern Recognit. 61, 153–168 (2017)

    Article  Google Scholar 

  14. Corchs, S., Gasparini, F.: A multidistortion database for image quality. In: Bianco, S., Schettini, R., Trémeau, A., Tominaga, S. (eds.) CCIW 2017. LNCS, vol. 10213, pp. 95–104. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56010-6_8

    Chapter  Google Scholar 

  15. Virtanen, T., Nuutinen, M., Vaahteranoksa, M., Oittinen, P., Häkkinen, J.: CID2013: a database for evaluating no-reference image quality assessment algorithms. IEEE Trans. Image Process. 24(1), 390–402 (2015)

    Article  MathSciNet  Google Scholar 

  16. Ghadiyaram, D., Bovik, A.C.: Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans. Image Process. 25(1), 372–387 (2016)

    Article  MathSciNet  Google Scholar 

  17. Lu, Y., Xie, F., Liu, T., Jiang, Z., Tao, D.: No reference quality assessment for multiply-distorted images based on an improved bag-of-words model. IEEE Signal Process. Lett. 22(10), 1811–1815 (2015)

    Article  Google Scholar 

  18. Li, C., Zhang, Y., Wu, X., Fang, W., Mao, L.: Blind multiply distorted image quality assessment using relevant perceptual features. In: Proceedings of IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, pp. 4883–4886, September 2015

    Google Scholar 

  19. Li, C., Zhang, Y., Wu, X., Zheng, Y.: A multi-scale learning local phase and amplitude blind image quality assessment for multiply distorted images. IEEE Access 6, 64 577–64 586 (2018)

    Article  Google Scholar 

  20. Li, Q., Lin, W., Fang, Y.: No-reference quality assessment for multiply-distorted images in gradient domain. IEEE Signal Process. Lett. 23(4), 541–545 (2016)

    Article  Google Scholar 

  21. Hadizadeh, H., Bajić, I.V.: Color Gaussian jet features for no-reference quality assessment of multiply-distorted images. IEEE Signal Process. Lett. 23(12), 1717–1721 (2016)

    Article  Google Scholar 

  22. Zhang, Y., Chandler, D.M.: Opinion-unaware blind quality assessment of multiply and singly distorted images via distortion parameter estimation. IEEE Trans. Image Process. 27(11), 5433–5448 (2018)

    Article  MathSciNet  Google Scholar 

  23. Ma, K., Liu, W., Zhang, K., Duanmu, Z., Wang, Z., Zuo, W.: End-to-end blind image quality assessment using deep neural networks. IEEE Trans. Image Process. 27(3), 1202–1213 (2018)

    Article  MathSciNet  Google Scholar 

  24. Kim, J., Lee, S.: Fully deep blind image quality predictor. IEEE J. Sel. Topics Signal Process. 11(1), 206–220 (2017)

    Article  Google Scholar 

  25. Talebi, H., Milanfar, P.: NIMA: neural image assessment. IEEE Trans. Image Process. 27(8), 3998–4011 (2018)

    Article  MathSciNet  Google Scholar 

  26. Bosse, S., Maniry, D., Müller, K., Wiegand, T., Samek, W.: Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans. Image Process. 27(1), 206–219 (2018)

    Article  MathSciNet  Google Scholar 

  27. Li, J., Zou, L., Yan, J., Deng, D., Qu, T., Xie, G.: No-reference image quality assessment using Prewitt magnitude based on convolutional neural networks. Signal Image Video Process. (SIViP) 10(4), 609–616 (2016)

    Article  Google Scholar 

  28. Zeng, H., Zhang, L., Bovik, A.C.: Blind image quality assessment with a probabilistic quality representation. In: Proceedings of IEEE International Conference on Image Processing (ICIP), Athens, Greece, pp. 609–613, October 2018

    Google Scholar 

  29. Gao, F., Yu, J., Zhu, S., Huang, Q., Tian, Q.: Blind image quality prediction by exploiting multi-level deep representations. Pattern Recognit. 81, 432–442 (2018)

    Article  Google Scholar 

  30. Bianco, S., Celona, L., Napoletano, P., Schettini, R.: On the use of deep learning for blind image quality assessment. Signal Image Video Process. (SIViP) 12(2), 355–362 (2018)

    Article  Google Scholar 

  31. Kang, L., Ye, P., Li, Y., Doermann, D.: Convolutional neural networks for no-reference image quality assessment. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, pp. 1733–1740, June 2014

    Google Scholar 

  32. Kang, L., Ye, P., Li, Y., Doermann, D.: Simultaneous estimation of image quality and distortion via multi-task convolutional neural networks. In: Proceedings of IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, pp. 2791–2795, September 2015

    Google Scholar 

  33. Fu, J., Wang, H., Zuo, L.: Blind image quality assessment for multiply distorted images via convolutional neural networks. In: Proceedings of IEEE International Conference on Acoustics, Speech, Signal Processing (ICASSP), Shanghai, China, pp. 1075–1079, March 2016

    Google Scholar 

  34. Li, J., Yan, J., Deng, D., Shi, W., Deng, S.: No-reference image quality assessment based on hybrid model. Signal Image Video Process. (SIViP) 11(6), 985–992 (2017)

    Article  Google Scholar 

  35. Li, Q., Wang, Z.: Reduced-reference image quality assessment using divisive normalization-based image representation. IEEE J. Sel. Topics Signal Process. 3(2), 202–211 (2009)

    Article  Google Scholar 

  36. Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of International Conference on Machine Learning (ICML), Haifa, Israel, pp. 807–814, June 2010

    Google Scholar 

  37. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, pp. 1026–1034, December 2015

    Google Scholar 

  38. Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: Proceedings of International Conference on Learning Representations (ICLR), San Diego, CA, USA, May 2015

    Google Scholar 

  39. Moorthy, A.K., Bovik, A.C.: A two-step framework for constructing blind image quality indices. IEEE Signal Process. Lett. 17(5), 513–516 (2010)

    Article  Google Scholar 

  40. Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)

    Article  MathSciNet  Google Scholar 

  41. Ye, P., Kumar, J., Kang, L., Doermann, D.: Unsupervised feature learning framework for no-reference image quality assessment. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, pp. 1098–1105, June 2012

    Google Scholar 

  42. Ma, K., Liu, W., Liu, T., Wang, Z., Tao, D.: dipIQ: blind image quality assessment by learning-to-rank discriminable image pairs. IEEE Trans. Image Process. 26(8), 3951–3964 (2017)

    Article  MathSciNet  Google Scholar 

  43. Xue, W., Mou, X., Zhang, L., Bovik, A.C., Feng, X.: Blind image quality assessment using joint statistics of gradient magnitude and laplacian features. IEEE Trans. Image Process. 23(11), 4850–4862 (2014)

    Article  MathSciNet  Google Scholar 

  44. Xu, J., Ye, P., Li, Q., Du, H., Liu, Y., Doermann, D.: Blind image quality assessment based on high order statistics aggregation. IEEE Trans. Image Process. 25(9), 4444–4457 (2016)

    Article  MathSciNet  Google Scholar 

  45. Zhang, L., Zhang, L., Bovik, A.C.: A feature-enriched completely blind image quality evaluator. IEEE Trans. Image Process. 24(8), 2579–2591 (2015)

    Article  MathSciNet  Google Scholar 

  46. Wu, Q., Wang, Z., Li, H.: A highly efficient method for blind image quality assessment. In: Proceedings of IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, pp. 339–343, September 2015

    Google Scholar 

  47. Mittal, A., Soundararajan, R., Bovik, A.C.: Making a “Completely Blind” image quality analyzer. IEEE Signal Process. Lett. 20(3), 209–212 (2013)

    Article  Google Scholar 

  48. Li, Q., Lin, W., Xu, J., Fang, Y.: Blind image quality assessment using statistical structural and luminance features. IEEE Trans. Multimedia 18(12), 2457–2469 (2016)

    Article  Google Scholar 

  49. Xue, W., Zhang, L., Mou, X.: Learning without human scores for blind image quality assessment. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA, pp. 995–1002, June 2013

    Google Scholar 

  50. Wang, Z., Li, Q.: Information content weighting for perceptual image quality assessment. IEEE Trans. Image Process. 20(5), 1185–1198 (2011)

    Article  MathSciNet  Google Scholar 

  51. Video Quality Experts Group and others, Final report from the Video Quality Experts Group on the validation of objective models of video quality assessment, Phase II (2003)

    Google Scholar 

  52. Sheskin, D.J.: Handbook of Parametric and Nonparametric Statistical Procedures. Chapman & Hall/CRC, Boca Raton (2011)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, ON, N2L 3G1, Canada

    Zhongling Wang, Shahrukh Athar & Zhou Wang

Authors
  1. Zhongling Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shahrukh Athar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhou Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shahrukh Athar .

Editor information

Editors and Affiliations

  1. University of Waterloo, Waterloo, ON, Canada

    Fakhri Karray

  2. University of Porto, Porto, Portugal

    Aurélio Campilho

  3. University of Waterloo, Waterloo, ON, Canada

    Alfred Yu

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, Z., Athar, S., Wang, Z. (2019). Blind Quality Assessment of Multiply Distorted Images Using Deep Neural Networks. In: Karray, F., Campilho, A., Yu, A. (eds) Image Analysis and Recognition. ICIAR 2019. Lecture Notes in Computer Science(), vol 11662. Springer, Cham. https://doi.org/10.1007/978-3-030-27202-9_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-27202-9_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-27201-2

  • Online ISBN: 978-3-030-27202-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation