Skip to main content
SpringerLink
Log in

Blind quality assessment of screen content images via edge histogram descriptor and statistical moments

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

With the growth in utilizing desktop sharing and remote control applications in recent years for many purposes like online education and remote working, quality assessment (QA) of screen images has become a hot topic. It could be used to enhance the user’s quality experience. Currently, most screen image QA methods require a reference image, and the existing blind/no-reference methods do not consider both the image’s content and chrominance degradations. This paper proposes a novel blind quality assessment method for screen content images (SCIs) through block-based content representation, which extracts content- and chromatic-based features on local, semi-global, and global scales. Our proposed edge histogram descriptor- and statistical moment-based (EHDSM) method divides the image into 16 blocks and then describes each block using its local edge and semi-global chrominance features. It also takes the global chrominance features into account to investigate how the image’s color information is changed in the presence of chrominance distortions. Local features are extracted using edge histogram descriptor, while the semi-global and global features are measured by computing the statistical moments. Next, the quality assessment is achieved by training a support vector regression (SVR) model. Extensive experiments on three commonly used SCI datasets have verified the superiority of our proposed EHDSM method compared with the state-of-the-art blind screen content image quality assessment methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

Data sharing is not applicable to this article as no new datasets were generated or analyzed during the current study.

References

  1. Wang, S., Gu, K., Ma, S., Gao, W.: Joint chroma downsampling and upsampling for screen content image. IEEE Trans. Circuits Syst. Video Technol. 26, 1595–609 (2016)

    Article  Google Scholar 

  2. Gu, K., Zhai, G., Lin, W., Yang, X., Zhang, W.: Learning a blind quality evaluation engine of screen content images. Neurocomputing 196, 140–149 (2016)

    Article  Google Scholar 

  3. Gu, K., Zhou, J., Qiao, J.F., Zhai, G., Lin, W., Bovik, A.C.: No-reference quality assessment of screen content pictures. IEEE Trans. Image Process. 26, 4005–4018 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  4. Gu, K., et al.: Saliency-guided quality assessment of screen content images. IEEE Trans. Multimedia 18, 1098–1110 (2016)

    Article  Google Scholar 

  5. Mittal, A., Soundararajan, R., Bovik, A.C.: Making a “completely blind" image quality analyzer. IEEE Signal Process. Lett. 20, 209–212 (2013)

    Article  Google Scholar 

  6. Zhang, L., Zhang, L., Bovik, A.C.: A feature-enriched completely blind image quality evaluator. IEEE Trans. Image Process. 24, 2579–2591 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  7. Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21, 4695–4708 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. Xue, W., Mou, X., Zhang, L., Bovik, A.C., Feng, X.: Blind image quality assessment using joint statistics of gradient magnitude and Laplacian features. IEEE Trans. Image Process. 23, 4850–4862 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  9. Fang, Y., Du, R., Zuo, Y., Wen, W., Li, L.: Perceptual quality assessment for screen content images by spatial continuity. IEEE Trans. Circuits Syst. Video Technol. 30, 4050–4063 (2020)

    Article  Google Scholar 

  10. Tolie, H.F., Faraji, M.R.: Screen content image quality assessment using distortion-based directional edge and gradient similarity maps. Signal Process. Image Commun. 101, 116562 (2022)

    Article  Google Scholar 

  11. Yang, H., Fang, Y., Lin, W.: Perceptual quality assessment of screen content images. IEEE Trans. Image Process. 24, 4408–4421 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  12. Fang, Y., Yan, J., Li, L., Wu, J., Lin, W.: No reference quality assessment for screen content images with both local and global feature representation. IEEE Trans. Image Process. 27, 1600–1610 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  13. Lu, N., Li, G.: Blind quality assessment for screen content images by orientation selectivity mechanism. Signal Process. 145, 225–232 (2018)

    Article  Google Scholar 

  14. Zheng, L., Shen, L., Chen, J., An, P., Luo, J.: No-reference quality assessment for screen content images based on hybrid region features fusion. IEEE Trans. Multimedia 21, 2057–2070 (2019)

    Article  Google Scholar 

  15. Bai, Y., Zhu, Z., Zhu, C., Wang, Y.: Blind image quality assessment of screen content images via fisher vector coding. IEEE Access (Early Access) 10, 13174–13181 (2022)

    Article  Google Scholar 

  16. Zhai, G., Wu, X., Yang, X., Lin, W., Zhang, W.: A psychovisual quality metric in freeenergy principle. IEEE Trans. Image Process. 21, 41–52 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  17. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24, 971–987 (2002)

    Article  MATH  Google Scholar 

  18. Wu, J., Lin, W., Shi, G., Zhang, Y., Dong, W., Chen, Z.: Visual orientation selectivity based structure description. IEEE Trans. Image Process. 24(11), 4602–4613 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  19. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)

    Article  Google Scholar 

  20. Sheikh, H., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 3440–3451 (2006)

    Article  Google Scholar 

  21. Sheikh, H.R. , Wang, Z., Cormack, L., Bovik, A.C.: Live image quality assessment database release 2. http://live.ece.utexas.edu/research/quality (2005)

  22. Bai, Y., Yu, M., Jiang, Q., Jiang, G., Zhu, Z.: Learning content-specific codebooks for blind quality assessment of screen content images. Signal Process. 161, 248–258 (2019)

    Article  Google Scholar 

  23. Bai, Y., Zhu, Z., Jiang, G., Sun, H.: Blind quality assessment of screen content images via macro-micro modeling of tensor domain dictionary. IEEE Trans. Multimedia 161, 248–258 (2020)

    Google Scholar 

  24. Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54, 4311–4322 (2006)

    Article  MATH  Google Scholar 

  25. Chen, J., Shen, L., Zheng, L., Jiang, X.: Naturalization module in neural networks for screen content image quality assessment. IEEE Signal Process. Lett. 25, 1685–1689 (2018)

    Article  Google Scholar 

  26. Zheng, L., Shen, L., Chen, J., An, P., Luo, J.: No reference quality assessment for screen content images using stacked autoencoders in pictorial and textual regions. IEEE Trans. Cybern. 52, 2798–2810 (2020)

    Google Scholar 

  27. Jiang, X., Shen, L., Feng, G., Yu, L., An, P.: “An optimized cnn-based quality assessment model for screen content image,” Signal Processing: Image Communication, vol. 94, May (2021)

  28. Wang, R., Yang, H., Pan, Z., Huang, B., Hou, G.: Screen content image quality assessment with edge features in gradient domain. IEEE Access. 7, 4818–4831 (2019)

    Google Scholar 

  29. Fang, Y., Yan, J., Du, R., Zuo, Y., Wen, W., Zeng, Y., Li, L.: Blind quality assessment for tone-mapped images by analysis of gradient and chromatic statistics. IEEE Trans. Multimedia 23, 955–966 (2020)

    Article  Google Scholar 

  30. Weijer,J. V. D., Schmid, C.: “Coloring local feature extraction,” Lecture Notes in Computer Science, pp. 334–348, (2006)

  31. Gerhard, H.E., Wichmann, F.A., Bethge, M.: How sensitive is the human visual system to the local statistics of natural images? PLoS Comput. Biol. 9(1), e1002873 (2013)

    Article  Google Scholar 

  32. Marr, D., Hildreth, E.: Theory of edge detection. Proc. R. Soc. Lond. B 207(1167), 187–217 (1980)

    Article  Google Scholar 

  33. Loh,W. T., Bong, D. B. L.: “Quality assessment for natural and screen visual contents,” IEEE International Conference on Image Processing, Taipei, Taiwan, p. 3025-3026, September (2019)

  34. Martini, m. G., Hewage, c. T.E.R., Villarini, B.: “Image quality assessment based on edge preservation,” Signal Processing: Image Communication, vol. 27, no. 8, pp. 875–882, (2012). Special issue on: pervasive mobilemultimedia

  35. Sadykova,D., James, A. P.: “Quality assessment metrics for edge detection and edge-aware filtering: A tutorial review,” in Quality assessment metrics for edge detection and edge-aware filtering: A tutorial review, pp. 2366–2369, (2017)

  36. Ni, Z., Ma, L., Zeng, H., Cai, C., Ma, K.: Gradient direction for screen content image quality assessment. IEEE Signal Process. Lett. 23, 1394–1398 (2016)

    Article  Google Scholar 

  37. Ni, Z., Ma, L., Zeng, H., Chen, J., Cai, C., Ma, K.-K.: ESIM: Edge similarity for screen content image quality assessment. IEEE Trans. Image Process. 26, 4818–4831 (2017)

    Article  MathSciNet  Google Scholar 

  38. Fu, Y., Zeng, H., Ma, L., Ni, Z., Zhu, J., Ma, K.: Screen content image quality assessment using multi-scale difference of gaussian. IEEE Trans. Circuits Syst. Video Technol. 28, 2428–2432 (2018)

    Article  Google Scholar 

  39. Swain, M.J., Ballard, D.H.: Color indexing. Int. J. Comput. Vision 7–1, 11–32 (1991)

    Article  Google Scholar 

  40. Jain, A., Vailaya, A.: Image retrieval using color and shape. Pattern Recogn. 29(8), 1233–1244 (1966)

    Article  Google Scholar 

  41. Won, C., Park, D., Park, S.-J.: Efficient use of mpeg7 edge histogram descriptor. Electronics and Telecommunications Research Institute (ETRI) 24, 23–30 (2002)

    Google Scholar 

  42. Kabbai, L., Abdellaoui, M., Douik, A.: Image classification by combining local and global features. Vis. Comput. 35, 679–693 (2019)

    Article  Google Scholar 

  43. Stricker, M.A., Orengo, M.: Similarity of color images. Proc. SPIE 2420, 381–392 (1995)

    Article  Google Scholar 

  44. Kusumoto, R., Han, X., Chen, Y.-W.: “Hybrid aggregation of sparse coded descriptors for food recognition,” in 2014 22nd International Conference on Pattern Recognition, pp. 1490–1495, (2014)

  45. Cai,R., Fang, M.: “Blind image quality assessment by simulating the visual cortex,” The Visual Computer, pp. 1–18, (2022)

  46. Chang, C.C., Lin, C.J.: Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2(3), 1–27 (2011)

    Article  Google Scholar 

  47. Ni, Z., Ma, L., Zeng, H., Fu, Y., Xing, L., Ma, K.: “SCID: a database for screen content images quality assessment,” International Scientific Publications and Consulting Services, pp. 774–779, (2016)

  48. Wang, S., Gu, K., Zhang, X., Lin, W., Zhang, L., Ma, S., Gao, W.: Subjective and objective quality assessment of compressed screen content images. IEEE J. Emerg. Sel. Top. Circuits Syst. 4, 532–543 (2016)

    Article  Google Scholar 

  49. VQEG: Final report from the video quality experts group on the validation of objective models of video quality assessment. http://www.its.bldrdoc.gov/vqeg/vqeg-home.aspx, August 2015

  50. Ji, J., Xiang, K., Wang, X.: SCVS: blind image quality assessment based on spatial correlation and visual saliency. Vis. Comput. 39, 443–458 (2022)

    Article  Google Scholar 

  51. Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11), 2579–2605 (2008)

    MATH  Google Scholar 

  52. Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

No funding was received for conducting this study. The authors have no relevant financial or non-financial interests to disclose.

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Technology, Institute for Advanced Studies in Basic Sciences (IASBS), Zanjan, 45137-66731, Iran

    Hamidreza Farhadi Tolie & Mohammad Reza Faraji

  2. Department of Computer Science, Utah State University, Logan, UT, 84322-4205, USA

    Xiaojun Qi

Authors
  1. Hamidreza Farhadi Tolie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Reza Faraji
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaojun Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamidreza Farhadi Tolie.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tolie, H.F., Faraji, M.R. & Qi, X. Blind quality assessment of screen content images via edge histogram descriptor and statistical moments. Vis Comput (2023). https://doi.org/10.1007/s00371-023-03108-1

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00371-023-03108-1

Keywords

Search

Navigation