Skip to main content
SpringerLink
Log in

A structure-aware axis-aligned grid deformation approach for robust image retargeting

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Content-based image retargeting is a technique that resizes an input image to a given target resolution while minimizing distortions of important objects caused by aspect ratio variations. Conventional approaches have shared a similar methodology which aims to preserve salient regions as much as possible while allowing distortions of less important regions. Those methods have shown satisfactory results for input images whose objects are distinct and backgrounds are monotonous. However, their performance is not always guaranteed for images containing structural components such as straight lines, which are prone to be distorted after resizing and sensitive to human visual perception. In this paper, we propose a structure-aware axis-aligned grid deformation approach for robust image retargeting. Based on axis-aligned grid, our method finds the optimal grid for target image by quadratic optimization represented by two objective functions. The first one is the As-similar-as-possible (ASAP) energy function, which aims to preserve important regions while allowing distortions of trivial regions. The second one is the Adaptive Laplacian regularization (ALR) energy function, which aims to relieve structural distortions. Those two energy functions are combined into single quadratic optimization model ensuring the global convexity and solved by a quadratic programming solver for finding the optimal grid. Experimental results show that our method is robust to structural distortions while achieving the basic purpose of content-based image retargeting. For objective comparisons with other methods, we have provided objective evaluation scores by using a recent state-of-the-art image retargeting quality assessment scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. ACM Trans Graph (TOG) 26(3):10

    Article  Google Scholar 

  2. Chang C, Chuang Y (2012) A line-structure-preserving approach to image resizing. In: IEEE computer vision and pattern recognition (CVPR), pp 1075–1082

  3. Cheng M, Mitra N, Huang X, Torr P, Hu S (2015) Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 37(3):569–582

    Article  Google Scholar 

  4. Choi J, Kim C (2015) Sparse seam carving for structure preserving image retargeting. J Signal Process Systems 85(2):275–283

    Article  Google Scholar 

  5. Cui J, Liu Y, Xu Y, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low- and high-dimensional approaches. IEEE Trans Syst Man Cybern Syst 43(4):996–1002

    Article  Google Scholar 

  6. Das B, Gopalakrishnan V, Iyer K, Gaurav A (2016) Similarity and rigidity preserving image retargeting. In: IEEE international conference on image processing (ICIP), pp 1584–1588

  7. Dong W, Wu F, Kong Y, Mei X, Lee T -Y, Zhang X (2016) Image retargeting by texture-aware synthesis. IEEE Trans Vis Comput Graph 22(2):1088–1101

    Article  Google Scholar 

  8. Fang Y, Fang Z, Yuan F, Yang Y, Yang S, Xiong N (2016) Optimized multioperator image retargeting based on perceptual similarity measure. IEEE Trans Syst Man Cybern Syst

  9. Gaspero L (2007) Quadratic programming problem solver by means of an active-set dual method. www.diegm.uniud.it/digaspero

  10. Gioi R -G, Jakubowicz J, Morel J, Randall G (2008) LSD: A fast line segment detector with a false detection control. IEEE Transactions on Pattern Analysis & Machine Intelligence (TPAMI) 32(4):722– 732

    Article  Google Scholar 

  11. Guo Y, Liu F, Shi J, Zhou Z, Cleicher M (2009) Image retargeting using mesh parametrization. IEEE Trans Multimed 11(5):856–867

    Article  Google Scholar 

  12. Hsu C, Lin C, Fang Y, Lin W (2014) Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss. IEEE Journal of Selected Topics in Signal Processing 8(3):377–389

    Article  Google Scholar 

  13. Hu W, Zhongxuan L, Xin F (2014) Image retargeting via adaptive scaling with geometry preservation. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 4(1):70–81

    Article  Google Scholar 

  14. Kiess J, Kopf S, Guthier B, Effelsberg W (2010) Seam carving with improved edge preservation. IS&t/SPIE electronic imaging, international society for optics and photonics, pp 75420g– 75420g

  15. Kim W, Kim C (2012) Saliency detection via textural contrast. Opt Lett 37 (9):1550–1552

    Article  Google Scholar 

  16. Liang Y, Su Z, Wang C, Wang D, Luo X (2013) Optimised image retargeting using aesthetic-based cropping and scaling. IET Image Process 7(1):61–69

    Article  MathSciNet  Google Scholar 

  17. Liu C, Yuen J, Torralba A (2011) SIFT flow: dense correspondence across scenes and its applications. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 33(5):978–994

    Article  Google Scholar 

  18. Liu L, Cheng L, Liu Y, Jia Y, Rosenblum D (2016) Recognizing complex activities by a probabilistic interval-based model. In: Proceedings of the 13th AAAI conference on artificial intelligence pp 1266– 1272

  19. Liu T, Sun J, Zheng N, Tang X, Shum H (2007) Learning to detect a saliency object. In: IEEE computer vision and pattern recognition (CVPR), IEEE, pp 353–367

  20. Liu Y, Cui J, Zhao H, Zha H (2012) Fusion of low-and high-dimensional approaches by trackers sampling for generic human motion tracking. In: IEEE 21st international conference on pattern recognition (ICPR) pp 898–901

  21. Liu Y, Liang Y, Liu S, Rosenblum D, Zheng Y (2016) Predicting urban water quality with ubiquitous data. arXiv:1610.09462

  22. Liu Y, Nie L, Han L, Zhang L, Rosenblum D (2016) Action2activity: recognizing complex activities from sensor data. arXiv:1611.01872

  23. Liu Y, Nie L, Liu L, Rosenblum D (2016) From action to activity: Sensor-based activity recognition. Neurocomputing 181:108–115

    Article  Google Scholar 

  24. Liu Y, Zhang L, Nie L, Yan Y, Rosenblum D (2016) Fortune teller: predicting your career path. In: Proceedings of the 13th AAAI conference on artificial intelligence, pp 201–207

  25. Liu Y, Zhang X, Cui J, Wu C, Aghajan H, Zha H (2010) Visual analysis of child-adult interactive behaviors in video sequences. In: IEEE 16th international conference on virtual systems and multimedia (VSMM), pp 26–33

  26. Liu Y, Zheng Y, Liang Y, Liu S, Rosenblum D (2016) Urban water quality prediction based on multi-task multi-view learning. In: Proceedings of the 25th international conference on artificial intelligence, pp 2576–2582

  27. Lu Y, Wei Y, Liu L, Zhong J, Sun L, Liu Y (2016) Towards unsupervised physical activity recognition using smartphone accelerometers. Multimedia Tools and Applications, pp 1–19

  28. Panozzo D, Weber O, Sorkine O (2012) Robust image retargeting via axis-aligned deformation. Computer Graphics Forum 31(2):229–236

    Article  Google Scholar 

  29. Rubinstein M, Gutierrez D, Sorkine O, Shamir A (2010) A comparative study of image retargeting. ACM Trans Graph (TOG) 29(6):160

    Article  Google Scholar 

  30. Rubinstein M, Shamir A, Avidan S (2008) Improved seam carving for video retargeting. ACM Trans Graph (TOG) 27(3):16

    Article  Google Scholar 

  31. Rubinstein M, Shamir A, Avidan S (2009) Multi-operator media retargeting. ACM Trans Graph (TOG) 28(3):23

    Article  Google Scholar 

  32. Wang S, Lai S (2009) Fast structure-preserving image retargeting. In: IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1049–1052

  33. Wang Y, Tai C, Sorkine O, Lee T (2008) Optimized scale-and-stretch for image resizing. ACM Trans Graph (TOG) 27(5):118

    Article  Google Scholar 

  34. Wei J, Li C, Hu S, Martin R, Tai C (2012) Fisheye video correction. IEEE Trans Vis Comput Graph 18(10):1771–1783

    Article  Google Scholar 

  35. Wolf L, Guttmann M, Cohen D (2007) Non-homogeneous content-driven video-retargeting. In: Proceedings of the 11th IEEE international conference on computer vision (ICCV), pp 1–6

  36. Zhang G, Cheng M, Hu S, Martin R (2009) A shape preserving approach to image resizing. Comput Graph Forum 28(7):1897–1906

    Article  Google Scholar 

  37. Zhang Y, Ngan K (2016) Fast patch-wise image retargeting. In: IEEE international conference on image processing (ICIP), pp 1813–1817

  38. Zhang Y, Sun Z, Jiang P, Huang Y, Peng J (2016) Hybrid image retargeting using optimized seam carving and scaling. Multimedia Tools and Applications, pp 1–19

Download references

Acknowledgments

This work was supported by Hancom Co., Ltd.

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Korea Advanced Institute of Science and Technology(KAIST), Daejeon, 34141, Republic of Korea

    Yoonhyung Kim, Seungjun Jung & Changick Kim

  2. Department of Electrical Engineering, Hanbat National University, Daejeon, 34158, Republic of Korea

    Chanho Jung

Authors
  1. Yoonhyung Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seungjun Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chanho Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Changick Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Changick Kim.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, Y., Jung, S., Jung, C. et al. A structure-aware axis-aligned grid deformation approach for robust image retargeting. Multimed Tools Appl 77, 7717–7739 (2018). https://doi.org/10.1007/s11042-017-4674-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4674-1

Keywords

Search

Navigation