A structure-aware axis-aligned grid deformation approach for robust image retargeting

  • Yoonhyung Kim
  • Seungjun Jung
  • Chanho Jung
  • Changick Kim
Article
  • 80 Downloads

Abstract

Content-based image retargeting is a technique that resizes an input image to a given target resolution while minimizing distortions of important objects caused by aspect ratio variations. Conventional approaches have shared a similar methodology which aims to preserve salient regions as much as possible while allowing distortions of less important regions. Those methods have shown satisfactory results for input images whose objects are distinct and backgrounds are monotonous. However, their performance is not always guaranteed for images containing structural components such as straight lines, which are prone to be distorted after resizing and sensitive to human visual perception. In this paper, we propose a structure-aware axis-aligned grid deformation approach for robust image retargeting. Based on axis-aligned grid, our method finds the optimal grid for target image by quadratic optimization represented by two objective functions. The first one is the As-similar-as-possible (ASAP) energy function, which aims to preserve important regions while allowing distortions of trivial regions. The second one is the Adaptive Laplacian regularization (ALR) energy function, which aims to relieve structural distortions. Those two energy functions are combined into single quadratic optimization model ensuring the global convexity and solved by a quadratic programming solver for finding the optimal grid. Experimental results show that our method is robust to structural distortions while achieving the basic purpose of content-based image retargeting. For objective comparisons with other methods, we have provided objective evaluation scores by using a recent state-of-the-art image retargeting quality assessment scheme.

Keywords

Content-based image retargeting Quadratic programming Saliency detection Line segment detection 

Notes

Acknowledgments

This work was supported by Hancom Co., Ltd.

References

  1. 1.
    Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. ACM Trans Graph (TOG) 26(3):10CrossRefGoogle Scholar
  2. 2.
    Chang C, Chuang Y (2012) A line-structure-preserving approach to image resizing. In: IEEE computer vision and pattern recognition (CVPR), pp 1075–1082Google Scholar
  3. 3.
    Cheng M, Mitra N, Huang X, Torr P, Hu S (2015) Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 37(3):569–582CrossRefGoogle Scholar
  4. 4.
    Choi J, Kim C (2015) Sparse seam carving for structure preserving image retargeting. J Signal Process Systems 85(2):275–283CrossRefGoogle Scholar
  5. 5.
    Cui J, Liu Y, Xu Y, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low- and high-dimensional approaches. IEEE Trans Syst Man Cybern Syst 43(4):996–1002CrossRefGoogle Scholar
  6. 6.
    Das B, Gopalakrishnan V, Iyer K, Gaurav A (2016) Similarity and rigidity preserving image retargeting. In: IEEE international conference on image processing (ICIP), pp 1584–1588Google Scholar
  7. 7.
    Dong W, Wu F, Kong Y, Mei X, Lee T -Y, Zhang X (2016) Image retargeting by texture-aware synthesis. IEEE Trans Vis Comput Graph 22(2):1088–1101CrossRefGoogle Scholar
  8. 8.
    Fang Y, Fang Z, Yuan F, Yang Y, Yang S, Xiong N (2016) Optimized multioperator image retargeting based on perceptual similarity measure. IEEE Trans Syst Man Cybern SystGoogle Scholar
  9. 9.
    Gaspero L (2007) Quadratic programming problem solver by means of an active-set dual method. www.diegm.uniud.it/digaspero
  10. 10.
    Gioi R -G, Jakubowicz J, Morel J, Randall G (2008) LSD: A fast line segment detector with a false detection control. IEEE Transactions on Pattern Analysis & Machine Intelligence (TPAMI) 32(4):722– 732CrossRefGoogle Scholar
  11. 11.
    Guo Y, Liu F, Shi J, Zhou Z, Cleicher M (2009) Image retargeting using mesh parametrization. IEEE Trans Multimed 11(5):856–867CrossRefGoogle Scholar
  12. 12.
    Hsu C, Lin C, Fang Y, Lin W (2014) Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss. IEEE Journal of Selected Topics in Signal Processing 8(3):377–389CrossRefGoogle Scholar
  13. 13.
    Hu W, Zhongxuan L, Xin F (2014) Image retargeting via adaptive scaling with geometry preservation. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 4(1):70–81CrossRefGoogle Scholar
  14. 14.
    Kiess J, Kopf S, Guthier B, Effelsberg W (2010) Seam carving with improved edge preservation. IS&t/SPIE electronic imaging, international society for optics and photonics, pp 75420g– 75420gGoogle Scholar
  15. 15.
    Kim W, Kim C (2012) Saliency detection via textural contrast. Opt Lett 37 (9):1550–1552CrossRefGoogle Scholar
  16. 16.
    Liang Y, Su Z, Wang C, Wang D, Luo X (2013) Optimised image retargeting using aesthetic-based cropping and scaling. IET Image Process 7(1):61–69MathSciNetCrossRefGoogle Scholar
  17. 17.
    Liu C, Yuen J, Torralba A (2011) SIFT flow: dense correspondence across scenes and its applications. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 33(5):978–994CrossRefGoogle Scholar
  18. 18.
    Liu L, Cheng L, Liu Y, Jia Y, Rosenblum D (2016) Recognizing complex activities by a probabilistic interval-based model. In: Proceedings of the 13th AAAI conference on artificial intelligence pp 1266– 1272Google Scholar
  19. 19.
    Liu T, Sun J, Zheng N, Tang X, Shum H (2007) Learning to detect a saliency object. In: IEEE computer vision and pattern recognition (CVPR), IEEE, pp 353–367Google Scholar
  20. 20.
    Liu Y, Cui J, Zhao H, Zha H (2012) Fusion of low-and high-dimensional approaches by trackers sampling for generic human motion tracking. In: IEEE 21st international conference on pattern recognition (ICPR) pp 898–901Google Scholar
  21. 21.
    Liu Y, Liang Y, Liu S, Rosenblum D, Zheng Y (2016) Predicting urban water quality with ubiquitous data. arXiv:1610.09462
  22. 22.
    Liu Y, Nie L, Han L, Zhang L, Rosenblum D (2016) Action2activity: recognizing complex activities from sensor data. arXiv:1611.01872
  23. 23.
    Liu Y, Nie L, Liu L, Rosenblum D (2016) From action to activity: Sensor-based activity recognition. Neurocomputing 181:108–115CrossRefGoogle Scholar
  24. 24.
    Liu Y, Zhang L, Nie L, Yan Y, Rosenblum D (2016) Fortune teller: predicting your career path. In: Proceedings of the 13th AAAI conference on artificial intelligence, pp 201–207Google Scholar
  25. 25.
    Liu Y, Zhang X, Cui J, Wu C, Aghajan H, Zha H (2010) Visual analysis of child-adult interactive behaviors in video sequences. In: IEEE 16th international conference on virtual systems and multimedia (VSMM), pp 26–33Google Scholar
  26. 26.
    Liu Y, Zheng Y, Liang Y, Liu S, Rosenblum D (2016) Urban water quality prediction based on multi-task multi-view learning. In: Proceedings of the 25th international conference on artificial intelligence, pp 2576–2582Google Scholar
  27. 27.
    Lu Y, Wei Y, Liu L, Zhong J, Sun L, Liu Y (2016) Towards unsupervised physical activity recognition using smartphone accelerometers. Multimedia Tools and Applications, pp 1–19Google Scholar
  28. 28.
    Panozzo D, Weber O, Sorkine O (2012) Robust image retargeting via axis-aligned deformation. Computer Graphics Forum 31(2):229–236CrossRefGoogle Scholar
  29. 29.
    Rubinstein M, Gutierrez D, Sorkine O, Shamir A (2010) A comparative study of image retargeting. ACM Trans Graph (TOG) 29(6):160CrossRefGoogle Scholar
  30. 30.
    Rubinstein M, Shamir A, Avidan S (2008) Improved seam carving for video retargeting. ACM Trans Graph (TOG) 27(3):16CrossRefGoogle Scholar
  31. 31.
    Rubinstein M, Shamir A, Avidan S (2009) Multi-operator media retargeting. ACM Trans Graph (TOG) 28(3):23CrossRefGoogle Scholar
  32. 32.
    Wang S, Lai S (2009) Fast structure-preserving image retargeting. In: IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1049–1052Google Scholar
  33. 33.
    Wang Y, Tai C, Sorkine O, Lee T (2008) Optimized scale-and-stretch for image resizing. ACM Trans Graph (TOG) 27(5):118CrossRefGoogle Scholar
  34. 34.
    Wei J, Li C, Hu S, Martin R, Tai C (2012) Fisheye video correction. IEEE Trans Vis Comput Graph 18(10):1771–1783CrossRefGoogle Scholar
  35. 35.
    Wolf L, Guttmann M, Cohen D (2007) Non-homogeneous content-driven video-retargeting. In: Proceedings of the 11th IEEE international conference on computer vision (ICCV), pp 1–6Google Scholar
  36. 36.
    Zhang G, Cheng M, Hu S, Martin R (2009) A shape preserving approach to image resizing. Comput Graph Forum 28(7):1897–1906CrossRefGoogle Scholar
  37. 37.
    Zhang Y, Ngan K (2016) Fast patch-wise image retargeting. In: IEEE international conference on image processing (ICIP), pp 1813–1817Google Scholar
  38. 38.
    Zhang Y, Sun Z, Jiang P, Huang Y, Peng J (2016) Hybrid image retargeting using optimized seam carving and scaling. Multimedia Tools and Applications, pp 1–19Google Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Yoonhyung Kim
    • 1
  • Seungjun Jung
    • 1
  • Chanho Jung
    • 2
  • Changick Kim
    • 1
  1. 1.School of Electrical EngineeringKorea Advanced Institute of Science and Technology(KAIST)DaejeonRepublic of Korea
  2. 2.Department of Electrical EngineeringHanbat National UniversityDaejeonRepublic of Korea

Personalised recommendations