Skip to main content
SpringerLink
Log in

Blind image quality assessment by simulating the visual cortex

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Objective assessment of image quality seeks to predict image quality without human perception. Given that the ultimate goal of a blind/no-reference image quality assessment (BIQA) algorithm is to provide a score consistent with the subject’s prediction, it makes sense to design an algorithm that resembles human behavior. Recently, a large number of image features have been introduced to image quality assessment. However, only a few of these features are generated by using the computational mechanisms of the visual cortex. In this paper, we propose bioinspired algorithms to extract image features for BIQA by simulating the visual cortex. We extract spatial features like texture and energy from images by mimicking the retinal circuit. We extract spatial-frequency features from images by imitating the simple cell of the primary visual cortex. And we extract color features from images by employing the color opponent mechanism of the biological vision system. Then, by using the statistical features derived from these physiologically plausible features, we train a support vector regression model to predict image quality. The experimental results show that the proposed algorithm is more consistent with subjective evaluations than the comparison algorithms in predicting image quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability statements

All data generated or analyzed during this study are included in this published article and are freely available to any researcher wishing to use them for non-commercial purposes, without breaching participant confidentiality.

References

  1. Wang, Z., Bovik, A.C.: Modern image quality assessment. Synth. Lectures Image Video Multimedia Process. 2(1), 156 (2006). https://doi.org/10.2200/S00010ED1V01Y200508IVM003

    Article  Google Scholar 

  2. Thung, K.H., Raveendran, P.: A survey of image quality measures. Int. Conf. Tech. Postgraduates (TECHPOS) 2009, 1–4 (2009). https://doi.org/10.1109/TECHPOS.2009.5412098

    Article  Google Scholar 

  3. Ruderman, D. L.:The statistics of natural images. Netw. Comput. Neural Syst. 5(4), 517–548 (1994). https://doi.org/10.1088/0954-898X_5_4_006

  4. Moorthy, A.K., Bovik, A.C.: A two-step framework for constructing blind image quality indices. IEEE Signal Process. Lett. 17(5), 513–516 (2010). https://doi.org/10.1109/LSP.2010.2043888

    Article  Google Scholar 

  5. Moorthy, A.K., Bovik, A.C.: Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans. Image Process. 20(12), 3350–3364 (2011). https://doi.org/10.1109/TIP.2011.2147325

    Article  MathSciNet  MATH  Google Scholar 

  6. Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012). https://doi.org/10.1109/TIP.2012.2214050

    Article  MathSciNet  MATH  Google Scholar 

  7. Liu, L., Wang, T., Huang, H.: Pre-attention and spatial dependency driven no-reference image quality assessment. IEEE Trans. Multimedia 21(9), 2305–2318 (2019). https://doi.org/10.1109/TMM.2019.2900941

    Article  Google Scholar 

  8. Gu, K., Zhai, G., Lin, W., Yang, X., Zhang, W.: No-reference image sharpness assessment in autoregressive parameter space. IEEE Trans. Image Process. 24(10), 3218–31 (2015). https://doi.org/10.1109/TIP.2015.2439035

    Article  MathSciNet  MATH  Google Scholar 

  9. Joshi, P., Prakash, S., Rawat, S.: Continuous wavelet transform-based no-reference quality assessment of deblocked images. Vis. Comput. 2934(12), 1739–1748 (2018). https://doi.org/10.1007/s00371-017-1460-z

    Article  Google Scholar 

  10. Ji, J., Xiang, K., Wang, X.: Scvs: blind image quality assessment based on spatial correlation and visual saliency. Vis. Comput. (2022). https://doi.org/10.1007/s00371-021-02340-x

    Article  Google Scholar 

  11. Mittal, A., Soundararajan, R., Bovik, A.C.: Making a “completely blind” image quality analyzer. IEEE Signal Process. Lett. 20(3), 209–212 (2013). https://doi.org/10.1109/LSP.2012.2227726

  12. Zhang, L., Zhang, L., Bovik, A.C.: A feature-enriched completely blind image quality evaluator. IEEE Trans. Image Process. 24(8), 2579–2591 (2015). https://doi.org/10.1109/TIP.2015.2426416

    Article  MathSciNet  MATH  Google Scholar 

  13. Saad, M.A., Bovik, A.C., Charrier, C.: Blind image quality assessment: a natural scene statistics approach in the dct domain. IEEE Trans. Image Process. 21(8), 3339–3352 (2012). https://doi.org/10.1109/TIP.2012.2191563

    Article  MathSciNet  MATH  Google Scholar 

  14. Nizami, I. F., Rehman, M. U., Majid, M., Anwar, S. M.: Natural scene statistics model independent no-reference image quality assessment using patch based discrete cosine transform. Multimedia Tools Appl. 79(2), (2020). https://doi.org/10.1007/s11042-020-09229-2

  15. Gupta, P., Moorthy, A. K., Soundararajan, R., Bovik,A. C.: Generalized gaussian scale mixtures: A model for wavelet coefficients of natural images. Signal Process. Image Commun. (2018) S0923596518303710 https://doi.org/10.1016/j.image.2018.05.009

  16. Li, C., Guan, T., Zheng, Y., Zhong, X., Bovik, A.: Blind image quality assessment in the contourlet domain. Signal Process. Image Commun. 91(5), 116064 (2021). https://doi.org/10.1016/j.image.2020.116064

    Article  Google Scholar 

  17. Kang, L., Ye, P., Li, Y., Doermann, D.: Convolutional neural networks for no-reference image quality assessment. IEEE Conf. Comput. Vis. Pattern Recog. 2014, 1733–1740 (2014). https://doi.org/10.1109/CVPR.2014.224

    Article  Google Scholar 

  18. Kim, J., Nguyen, A.-D., Lee, S.: Deep cnn-based blind image quality predictor. IEEE Trans. Neural Netw. Learn. Syst. 30(1), 11–24 (2019). https://doi.org/10.1109/TNNLS.2018.2829819

    Article  Google Scholar 

  19. Zhang, W., Ma, K., Yan, J., Deng, D., Wang, Z.: Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Trans. Circuits Syst. Video Technol. 30(1), 36–47 (2020). https://doi.org/10.1109/TCSVT.2018.2886771

    Article  Google Scholar 

  20. Zhu, H., Li, L., Wu, J., Dong, W., Shi, G.: Metaiqa: Deep meta-learning for no-reference image quality assessment. IEEE/CVF Conf. Comput. Vis. Pattern Recogn. (CVPR) 2020, 14131–14140 (2020). https://doi.org/10.1109/CVPR42600.2020.01415

    Article  Google Scholar 

  21. Ma, J., Wu, J., Li, L., Dong, W., Xie, X.: Active inference of gan for no-reference image quality assessment. IEEE Int. Conf. Multimedia Expo (ICME) 2020, 1–6 (2020). https://doi.org/10.1109/ICME46284.2020.9102895

    Article  Google Scholar 

  22. Wu, J., Lin, W., Shi, G., Li, L., Fang, Y.: Orientation selectivity based visual pattern for reduced-reference image quality assessment. Inf. Sci. 351, 18–29 (2016). https://doi.org/10.1016/j.ins.2016.02.043

    Article  Google Scholar 

  23. Wu, J., Zeng, J., Dong, W., Shi, G., Lin, W.: Blind image quality assessment with hierarchy: Degradation from local structure to deep semantics. J. Vis. Commun. Image Rep. 58, 353–362 (2019). https://doi.org/10.1016/j.jvcir.2018.12.005

  24. Gu, K., Zhai, G., Yang, X., Zhang, W.: Using free energy principle for blind image quality assessment. IEEE Trans. Multimedia 17(1), 50–63 (2015). https://doi.org/10.1109/TMM.2014.2373812

    Article  Google Scholar 

  25. Kaneko, Akimichi: Receptive field organization of bipolar and amacrine cells in the goldfish retina. J. Physiol. 235(1), 133–53 (1973). https://doi.org/10.1113/jphysiol.1973.sp010381

    Article  MathSciNet  Google Scholar 

  26. Dacey, D., Packer, O.S., Diller, L., Brainard, D., Lee, B.: Center surround receptive field structure of cone bipolar cells in primate retina. Vis. Res. 40(14), 1801–1811 (2000). https://doi.org/10.1016/S0042-6989(00)00039-0

    Article  Google Scholar 

  27. Hubel, D.H., Wiesel, T.N.: Receptive fields of single neurones in the cat’s striate cortex. J. Physiol. 148(3), 574–591 (1959). https://doi.org/10.1113/jphysiol.1959.sp006308

    Article  Google Scholar 

  28. Hubel, D.H., Wiesel, T.N.: Receptive fields and functional architecture of monkey striate cortex. J. Physiol. 195(1), 215–243 (1968). https://doi.org/10.1113/jphysiol.1968.sp008455

    Article  Google Scholar 

  29. Ts’O, D., Gilbert, C.: The organization of chromatic and spatial interactions in the primate striate cortex. J. Neurosci. 8(5), 1712–1727 (1988). https://doi.org/10.1523/jneurosci.08-05-01712.1988

    Article  Google Scholar 

  30. Lennie, P., Krauskopf, J., Sclar, G.: Chromatic mechanisms in striate cortex of macaque. J. Neurosci. 10(2), 649–69 (1990). https://doi.org/10.1523/JNEUROSCI.10-02-00649.1990

    Article  Google Scholar 

  31. David Berga, X. O. V. L. X. M. P., Xose R.: Fdez-Vidal, Psychophysical evaluation of individual low-level feature influences on visual attention. Vis. Res. 154, 60–79 (2019). https://doi.org/10.1016/j.visres.2018.10.006

  32. Laurent Itti, C. K., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998). https://doi.org/10.1109/34.730558

  33. Berman, M.G., Hout, M.C., Kardan, O., Hunter, M.R., Yourganov, G., Henderson, J.M., Hanayik, T., Karimi, H., Jonides, J.: The perception of naturalness correlates with low-level visual features of environmental scenes. PloS One 9(12),(2014). https://doi.org/10.1371/journal.pone.0114572

  34. Kardan, O., Demiralp, E., Hout, M. C., Hunter, M. R., Karimi, H., Hanayik, T., Yourganov, G., Jonides, J., Berman, M. G.: Is the preference of natural versus man-made scenes driven by bottom-up processing of the visual features of nature?. Front. Psychol. 6 (2015). https://doi.org/10.3389/fpsyg.2015.00471

  35. Valeton, J.: Photoreceptor light adaptation models: an evaluation. Vis. Res. 23(12), 1549–1554 (1983). https://doi.org/10.1016/0042-6989(83)90168-2

    Article  Google Scholar 

  36. Beaudot, H. A.: William, Sensory coding in the vertebrate retina: towards an adaptive control of visual sensitivity. Netw. Comput. Neural Syst. 7(2), 317–323 (1996). https://doi.org/10.1088/0954-898X_7_2_012

  37. Kolb, Helga: How the retina works. Am. Sci. 91(1), 28–35 (2003)

    Article  Google Scholar 

  38. Rajalakshmi, T., Prince, S.: Retinal model-based visual perception: applied for medical image processing. Biol. Insp. Cognit. Arch. 18, 95–104 (2016). https://doi.org/10.1016/j.bica.2016.09.005

    Article  Google Scholar 

  39. Benoit, A., Caplier, A., Durette, B., Herault, J.: Using human visual system modeling for bio-inspired low level image processing. Comput. Vis. Image Understand. 114(7), 758–773 (2010). https://doi.org/10.1016/j.cviu.2010.01.011

    Article  Google Scholar 

  40. Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160(1), 106–154 (1962). https://doi.org/10.1113/jphysiol.1962.sp006837

    Article  Google Scholar 

  41. Field, D.J.: Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A 4(12), 2379–2394 (1987). https://doi.org/10.1364/JOSAA.4.002379

    Article  Google Scholar 

  42. Zhang, Y., Chandler, D.M.: No-reference image quality assessment based on log-derivative statistics of natural scenes. J. Electron. Imag. 22(4), 1–23 (2013). https://doi.org/10.1117/1.JEI.22.4.043025

    Article  Google Scholar 

  43. Pridmore, R.W.: A new transformation of cone responses to opponent color responses. Attention Percep. Psychophys. 83(1218), 1797–1803 (2021). https://doi.org/10.3758/s13414-020-02216-7

    Article  Google Scholar 

  44. Ruderman, D.L., Cronin, T.W., Chiao, C.-C.: Statistics of cone responses to natural images: implications for visual coding. J. Opt. Soc. Am. A 15(8), 2036–2045 (1998). https://doi.org/10.1364/JOSAA.15.002036

    Article  Google Scholar 

  45. Lasmar,N.-E., Stitou, Y., Berthoumieu, Y.: Multiscale skewed heavy tailed model for texture analysis. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 2281–2284 (2009). https://doi.org/10.1109/ICIP.2009.5414404

  46. Sheikh, H., Sabir, M., Bovik, A.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 3440–3451 (2006). https://doi.org/10.1109/TIP.2006.881959

    Article  Google Scholar 

  47. Chandler, L.D.M.: Most apparent distortion: full-reference image quality assessment and the role of strategy. J. Electronic Imag. 19(1), 011006 (2010). https://doi.org/10.1117/1.3267105

    Article  Google Scholar 

  48. Ponomarenko, N., Jin, L., Ieremeiew, O., et al.: Image database tid2013: peculiarities, results and perspectives. Signal Process. Image Commun. 30, 57–77 (2015). https://doi.org/10.1016/j.image.2014.10.009

    Article  Google Scholar 

  49. Ghadiyaram, D., Bovik, A.C.: Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans. Image Process. 25(1), 372–387 (2016). https://doi.org/10.1109/TIP.2015.2500021

    Article  MathSciNet  MATH  Google Scholar 

  50. Hosu, V., Lin, H., Sziranyi, T., Saupe, D.: Koniq-10k: an ecologically valid database for deep learning of blind image quality assessment. IEEE Trans. Image Process. 29, 4041–4056 (2020). https://doi.org/10.1109/TIP.2020.2967829

    Article  MATH  Google Scholar 

  51. Fang, Y., Zhu, H., Zeng, Y., Ma, K., Wang, Z.: Perceptual quality assessment of smartphone photography. IEEE/CVF Conf. Comput. Vis. Pattern Recogn. (CVPR) 2020, 3674–3683 (2020). https://doi.org/10.1109/CVPR42600.2020.00373

    Article  Google Scholar 

  52. Li, Q., Lin, W., Gu, K., Zhang, Y., Fang, Y.: Blind image quality assessment based on joint log-contrast statistics. Neurocomputing 331, 189–198 (2019). https://doi.org/10.1016/j.neucom.2018.11.015

    Article  Google Scholar 

  53. Ye, P., Kumar, J., Kang, L., Doermann, D.: Unsupervised feature learning framework for no-reference image quality assessment, in. IEEE Conf. Comput. Vis. Pattern Recogn. 2012, 1098–1105 (2012). https://doi.org/10.1109/CVPR.2012.6247789

    Article  Google Scholar 

  54. Xu, J., Ye, P., Li, Q., Du, H., Liu, Y., Doermann, D.: Blind image quality assessment based on high order statistics aggregation. IEEE Trans. Image Process. 25(9), 4444–4457 (2016). https://doi.org/10.1109/TIP.2016.2585880

    Article  MathSciNet  MATH  Google Scholar 

  55. Ghadiyaram, D., Bovik, A.C.: Perceptual quality prediction on authentically distorted images using a bag of features approach. J. Vis. 17(1), 32–32 (2017). https://doi.org/10.1167/17.1.32

    Article  Google Scholar 

  56. Yang, S., Jiang, Q., Lin, W., Wang, Y.: Sgdnet: An end-to-end saliency-guided deep neural network for no-reference image quality assessment. In: the 27th ACM International Conference, (2019). https://doi.org/10.1145/3343031.3350990

  57. Su, S., Yan, Q., Zhu, Y., Zhang, C., Ge, X., Sun, J., Zhang, Y.: Blindly assess image quality in the wild guided by a self-adaptive hyper network. IEEE/CVF Conf. Comput. Vis. Pattern Recogn. (CVPR) 2020, 3664–3673 (2020). https://doi.org/10.1109/CVPR42600.2020.00372

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the anonymous reviewers for their valuable suggestions on improving the quality of the paper.

Author information

Authors and Affiliations

  1. College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou, Fujian, 350117, People’s Republic of China

    Rongtai Cai

  2. Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, and Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Normal University, Fuzhou, 350117, People’s Republic of China

    Ming Fang

Authors
  1. Rongtai Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ming Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rongtai Cai.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cai, R., Fang, M. Blind image quality assessment by simulating the visual cortex. Vis Comput 39, 4639–4656 (2023). https://doi.org/10.1007/s00371-022-02614-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02614-y

Keywords

Search

Navigation