Advertisement

Retargeted Image Quality Assessment: Current Progresses and Future Trends

  • Lin Ma
  • Chenwei Deng
  • Weisi Lin
  • King Ngi Ngan
  • Long Xu
Chapter

Abstract

The diversity and versatility of the display devices have imposed new demands on digital image processing. Variant devices of different resolution screens need to display the same image for human visual experience. The retargeting methods are proposed to adapt the source image into arbitrary sizes and simultaneously keep the salient content of the source signal of high visual quality. Therefore, there is a new challenge of objectively evaluating the retargeted image perceptual quality, where variant resolutions may be presented, the objective shape may be distorted, and some content information may be discarded. In this chapter, recent progresses in quality assessment of retargeted images are reviewed. Firstly, we will review and discuss the recently developed retargeting methods for images. Afterwards, the subjective approaches to assess the retargeted image are reviewed, as well as the constructed subjective databases. Thirdly, some objective quality metrics developed recently are reviewed and compared based on the databases. Finally, future trends are discussed on retargeted image quality assessment in terms of both subjective and objective approaches.

Keywords

Source Image Scale Invariant Feature Transform Shape Descriptor Mean Opinion Score Perceptual Quality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

The work described in this chapter was partially supported by a grant from the Research Grants Council of the Hong Kong SAR, China (Project CUHK 415913); the National Nature Science Foundation of China under Grant No. 61301090; the ROSE Lab grant from the Singapore National Research Foundation; the Supporting Program for Beijing Excellent Talents under Grant No. 2013D009011000001, and the National Natural Science Foundation of China under Grant No. 61202242.

References

  1. 1.
    Shamir A., and Sorkine O.: Visual media retargeting. ACM SIGGRAPH Asia Courses, (2009).Google Scholar
  2. 2.
    Wolf L., Guttmann M., and Cohen-Or D.: Non-homogeneous content-driven video-retargeting. Proceedings of International Conference on Computer Vision, (2007).Google Scholar
  3. 3.
    Krahenbuhl P., Lang M., Hornung A., and Gross M.: A system for retargeting of streaming video. Proceedings of SIGGRAPH Asia, (2009).Google Scholar
  4. 4.
    Avidan S., and Shamir A.: Seam carving for content-aware image resizing. Proceedings of SIGGRAPH, (2007).Google Scholar
  5. 5.
    Rubinstein M., Shamir A., and Avidan A.: Improved seam carving for video retargeting. Proceedings of SIGGRAPH, (2008).Google Scholar
  6. 6.
    Shamir A., and Avidan S.: Seam-carving for media retargeting. Communications of the ACM, 52(1), 77–85, Jan. (2009).Google Scholar
  7. 7.
    Rubinstein M., Shamir A., and Avidan S.: Multi-operator media retargeting. Proceedings of SIGGRAPH, (2009).Google Scholar
  8. 8.
    Wang Y., Tai C., Sorkine O., and Lee T.: Optimized scale-and-stretch for image resizing. Proceedings of SIGGRAPH Asia, (2008).Google Scholar
  9. 9.
    Pritch Y., Kav-Venaki E., and Peleg S.: Shift-map image editing. Proceedings of International Conference on Computer Vision, (2009).Google Scholar
  10. 10.
    Qi A., and Ho J.: Shift-map based stereo image retargeting with disparity adjustment. Proceedings of Asian Conference on Computer Vision, (2013).Google Scholar
  11. 11.
    Dekel T., Moses Y., and Avidan S.: Stereo seam carving: a geometrically consistent approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(10), 2513–2525, Oct. (2013).Google Scholar
  12. 12.
    Dekel T., Moses Y., and Avidan S.: Geometrically consistent stereo seam carving. Proceedings of International Conference on Computer Vision, (2011).Google Scholar
  13. 13.
    Itti L., Koch C., and Niebur E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254–1259, Nov. (1998).Google Scholar
  14. 14.
    Karni Z., Freedman D., and Gotsman C.: Energy-based image deformation. Proceedings of Symposium on Geometry Processing, (2009).Google Scholar
  15. 15.
    Dong W., Zhou N., Paul J. C., and Zhang X.: Optimized image resizing using seam carving and scaling. Proceedings of SIGGRAPH, (2009).Google Scholar
  16. 16.
    Ma L., Deng C., Lin W., and Ngan K. N.: Image retargeting subjective quality database. Available. http://ivp.ee.cuhk.edu.hk/projects/demo/retargeting/index.html.
  17. 17.
    Rubinstein M., Gutierrez D., Sorkine O., and Shamir A.: A comparative study of image retargeting. Proceedings of SIGGRAPH Asia (2010). Available. http://people.csail.mit.edu/mrub/retargetme/.
  18. 18.
    Kendall M. G., and Babington Smith B.: On the method of paired comparisons. Biometrika, 31, 324–345, (1940).CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Kendall M. G.: A new measure of rank correlation. Biometrika, 30, 81–93, (1938).CrossRefzbMATHMathSciNetGoogle Scholar
  20. 20.
    VQEG.: Final report from the video quality experts group on the validation of objective models of video quality assessment II. (2009). Available. http://www.its.bldrdoc.gov/vqeg/projects/frtv_phaseII/downloads/VQEGII_Final_Report.pdf.
  21. 21.
    VQEG. Final report from the video quality experts group on the validation of objective models of video quality assessment. (2000). Available. http://www.its.bldrdoc.gov/vqeg/projects/frtv_phaseI/.
  22. 22.
    VQEG. Final report from the video quality experts from group on the validation of objective models of multimedia quality assessment Phase 1. Available. ftp://vqeg.its.bldrdoc.gov/Documents/Projects/multimedia/MM_Final_Report/.
  23. 23.
    Sheikh H. R., Sabir M. F., and Bovik A. C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Transactions on Image Processing, 15(11), 3440–3451, Nov. (2006).Google Scholar
  24. 24.
    Soundararajan K., Soundararajan R., Bovik A. C., and Cormack L. K.: Study of subjective and objective quality assessment of video. IEEE Transaction on Image Processing, 19(6), 1427–1441, Jun. (2010). Available. http://live.ece.utexas.edu/research/quality/live_video.html.
  25. 25.
    Soundararajan K., Soundararajan R., Bovik A. C., and Cormack L. K.: A subjective study to evaluate video quality assessment algorithms. Proceedings of SPIE, Human Vision and Electronic Imaging, Jan. (2010).Google Scholar
  26. 26.
    Pinson M. H., and Wolf S.: Comparing subjective video quality testing methodologies. Proceedings of SPIE, 5150(3), 573–582, (2003).CrossRefGoogle Scholar
  27. 27.
    Van Dijk A. M., Martens J. B., and Watson A. B.: Quality assessment of coded images using numerical category scaling. Proceedings of SPIE Advanced Image and Video Communications and Storage Technologies, (1995).Google Scholar
  28. 28.
    VQEG: Final report from the video quality experts from group on the validation of objective models of multimedia quality assessment Phase 1. Available. ftp://vqeg.its.bldrdoc.gov/Documents/Projects/multimedia/MM_Final_Report/
  29. 29.
    ITU-R Recommendation BT.500–11.: Methodology for the subjective assessment of the quality of television pictures”, ITU, Geneva, Switzerland, (2002).Google Scholar
  30. 30.
    ITU-T Recommendation P.910.: Subjective video quality assessment methods for multimedia applications. ITU, Geneva, Switzerland, (2008).Google Scholar
  31. 31.
    Ma L., Lin W., Deng C., and Ngan K. N.: Image retargeting quality assessment: a study of subjective scores and objective metrics. IEEE Journal of Selected Topics in Signal Processing, 6(6), 626–639, Oct. (2012).Google Scholar
  32. 32.
    Ma L., Lin W., Deng C., and Ngan K. N.: Study of subjective and objective quality assessment of retargeted images. Proceedings of International Symposium on Circuits and Systems, (2012).Google Scholar
  33. 33.
    Pele O., and Werman M.: Fast and robust earth mover’s distances. Proceedings of International Conference on Computer Vision, (2009).Google Scholar
  34. 34.
    Rubner Y., Tomasi C., and Guibas l. J.: The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision, 40(2), 99–121, Nov. (2000).Google Scholar
  35. 35.
    Simakov Yaron Caspi D., Shechtman E., and Irani M.: Summarizing visual data using bidirectional similarity. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (2008).Google Scholar
  36. 36.
    Barnes C., Shechtman E., Finkelstein A., and Goldman D. B.: Patchmatch: a randomized correspondence algorithm for structural image editing. Proceedings of SIGGRAPH, (2009).Google Scholar
  37. 37.
    Liu C., Yuen J., Torralba A., Sivic J., and Freeman W. T.: SIFT flow: dense correspondence across different scenes. Proceedings of European Conference on Computer Vision, (2008).Google Scholar
  38. 38.
    Liu Y., Luo X., Xuan Y., Chen W., and Fu X.: Image retargeting quality assessment. Proceedings of EUROGRAPHICS, (2011).Google Scholar
  39. 39.
    Manjunath B. S., Ohm J. R., Vasudevan V. V., and Yamada A.: Color and texture descriptors. IEEE Transaction on Circuits and System for Video Technology, 11(6), 703–715, Jun. (2001).Google Scholar
  40. 40.
    Vedaldi A., and Fulkerson B.: VLFeat: An open and portable library of computer vision algorithms. Available. http://www.vlfeat.org/, (2008).
  41. 41.
    Kasutani E., and Yamada A.: The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segement retrieval. Proceedings of International Conference on Image Processing, 674–677, (2001).Google Scholar
  42. 42.
    Lowe D.:Object recognition from local scale-invariant features. Proceedings of International Conferene on Conmputer Vision, (1999).Google Scholar
  43. 43.
    Oliva A., and Torralba A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Internaltional Journal of Computer Vision, 42(3), 145–175, (2001).CrossRefzbMATHGoogle Scholar
  44. 44.
    Lu W., and Wu M.: Reduced-reference quality assessment for retargeted images. Proceedings of International Conference on Image Processing, 1497–1500, (2012).Google Scholar
  45. 45.
    Lowe D.: Dictinctive image features from scale invariant keypoints. International Journal of Conputer Vision, 60(2), 91–110,(2004).CrossRefGoogle Scholar
  46. 46.
    Wang Z., Bovik A., Sheikh H., Simoncelli E.: Image quality assessment: from error visibility to structureal similarity. IEEE Transactions on Image Processing, 13(4), 600–612, Apr. (2004).Google Scholar
  47. 47.
    D’Angelo A., Menegaz G., and Barni M.: Perceptual quality evaluation of geometric distortions in images. Proceedings of SPIE Human Vision and Electronic Imaging, 6492, (2007).Google Scholar
  48. 48.
    D’Angelo A., Zhao Z., and Barni M.: A full-reference quality metric for geometrically distorted images. IEEE Transactions on Image Processing, 19(4), 867–881, Apr. (2010).Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Lin Ma
    • 1
  • Chenwei Deng
    • 2
  • Weisi Lin
    • 3
  • King Ngi Ngan
    • 4
  • Long Xu
    • 5
  1. 1.Huawei Noah’s Ark LabHong KongChina
  2. 2.School of Information and ElectronicsBeijing Institute of TechnologyBeijingChina
  3. 3.School of Computer EngineeringNanyang Technological UniversitySingaporeSingapore
  4. 4.Department of Electronic EngineeringThe Chinese University of Hong KongHong KongChina
  5. 5.National Astronomical ObservatoriesChinese Academy of SciencesBeijingChina

Personalised recommendations