Skip to main content

Advertisement

SpringerLink
Log in

Analysis of seam carving technique: limitations, improvements and possible solutions

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Nowadays, many efficient content-aware image resizing techniques are being used to safeguard the prominent regions of the image so that aesthetically pleasing retargeting results can be generated. In this paper, firstly various energy map generation methods are analyzed based on the conventional seam carving technique. After objective image quality assessment of the retargeted image obtained from the conventional seam carving technique, it has been found that the percentage of similarity obtained from the gradient energy map generation method is 77.44% which is higher than the other methods. Further, to minimize the deformations on the parameters such as luminance, color, and structure the improved seam carving technique utilizes the gradient energy generation method to obtain the energy map of all kinds of input images during the retargeting operation. To check the efficiency of the improved retargeting technique the obtained results are compared with the conventional seam carving technique based on different properties of the objects present in the image. After objective image quality assessment based on SSIM, it is found that the improved seam carving technique produces 70% similarity between reference and retargeted images which is 10% higher than the conventional seam carving technique. Furthermore, the energy modification operation of the improved seam carving technique also plays its significant contribution to minimize the deformation on the defined parameters. After the subjective and objective image quality assessment, it is found that the high percentage similarity in the retargeted results justifies the efficiency of the improved seam carving technique.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

References

  1. Avidan, S. and Shamir, A.: Seam carving for content-aware image resizing. In: Proceedings of ACM SIGGRAPH 2007 papers, pp. 10-es, (2007) doi: https://doi.org/10.1145/1275808.1276390

  2. Shamir, A., Avidan, S.: Seam carving for media retargeting. Commun. ACM 52(1), 77–85 (2009). https://doi.org/10.1145/1435417.1435437

    Article  Google Scholar 

  3. Garg, A., Negi, A.: Structure preservation in content-aware image retargeting using multi-operator. IET Image Proc. 14(13), 2965–2975 (2020). https://doi.org/10.1049/iet-ipr.2019.1032

    Article  Google Scholar 

  4. Zhang, Y., Sun, Z., Jiang, P., Huang, Y., Peng, J.: Hybrid image retargeting using optimized seam carving and scaling. Multim. Tools Appl. 76(6), 8067–8085 (2017). https://doi.org/10.1007/s11042-016-3318-1

    Article  Google Scholar 

  5. Fang, Y., Fang, Z., Yuan, F., et al.: Optimized multioperator image retargeting based on perceptual similarity measure. IEEE Trans. Syst., Man, Cybern.: Syst. 47(11), 2956–2966 (2016). https://doi.org/10.1109/TSMC.2016.2557225

    Article  Google Scholar 

  6. Tang, Z., Yao, J., Zhang, Q.: Multi-operator image retargeting in compressed domain by preserving aspect ratio of important contents. Multim. Tools Appl. 3, 1–22 (2021). https://doi.org/10.1007/s11042-021-11376-z

    Article  Google Scholar 

  7. Abhayadev, M., Santha, T.: Multi-operator content aware image retargeting on natural images. J. Sci. Ind. Res. 78, 193–198 (2019)

    Google Scholar 

  8. Garg, A., Negi, A., Jindal, P.: Structure preservation of image using an efficient content-aware image retargeting technique. SIViP 15(1), 185–193 (2021). https://doi.org/10.1007/s11760-020-01736-x

    Article  Google Scholar 

  9. Garg, A., Negi, A.: A survey on content aware image resizing methods. KSII Trans. Internet Inf. Syst. (TIIS). 14(7), 2997–3017 (2020). https://doi.org/10.3837/tiis.2020.07.015

    Article  Google Scholar 

  10. Chang, C.H., Chuang, Y.Y.: A line-structure-preserving approach to image resizing. In: IEEE conference on computer vision and pattern recognition, pp. 1075–1082, 2012 doi: https://doi.org/10.1109/CVPR.2012.6247786

  11. Han, R., Ke, Y., Du, L., Qin, F., Guo, J.: Exploring the location of object deleted by seam-carving. Expert Syst. Appl. 95, 162–171 (2018). https://doi.org/10.1016/j.eswa.2017.11.023

    Article  Google Scholar 

  12. Patel, D., Raman, S.: Accelerated seam carving for image retargeting. IET Image Proc. 13(6), 885–895 (2019). https://doi.org/10.1049/iet-ipr.2018.5283

    Article  Google Scholar 

  13. Kiess, J., Kopf, S., Guthier, B., Effelsberg, W.: Seam carving with improved edge preservation. multimedia on mobile devices. Int. Soc. Opt. Photon. 7542, 75420 (2010)

    Google Scholar 

  14. Yang, Y., Cheng, Z., Yu, H., et al.: MSE-Net: generative image inpainting with multi-scale encoder. Vis. Computer. 65, 1–13 (2021). https://doi.org/10.1007/s00371-021-02143-0

    Article  Google Scholar 

  15. Simakov, D., Caspi, Y., Shechtman, E., Irani, M.: Summarizing visual data using bidirectional similarity. In: IEEE conference on computer vision and pattern recognition, pp. 1–8, 2008 doi: https://doi.org/10.1109/CVPR.2008.4587842

  16. Bolduc, F., Lejeune, A., Magnenat-Thalmann, N.: Image synthesis and 3-D computer animation: a new approach for strategic analysis. Vis. Comput. 3(1), 51–56 (1987). https://doi.org/10.1007/BF02153650

    Article  Google Scholar 

  17. Nie, Y., Zhang, Q., Wang, R., Xiao, C.: Video retargeting combining warping and summarizing optimization. Vis. Comput. 29(6), 785–794 (2013). https://doi.org/10.1007/s00371-013-0830-4

    Article  Google Scholar 

  18. Su, Z., Luo, X., Artusi, A.: A novel image decomposition approach and its applications. Vis. Comput. 29(10), 1011–1023 (2013). https://doi.org/10.1007/s00371-012-0753-5

    Article  Google Scholar 

  19. Banerjee, A., Das, N., Santosh, K.C.: Weber local descriptor for image analysis and recognition: a survey. Vis. Computer. 68, 1–23 (2020). https://doi.org/10.1007/s00371-020-02017-x

    Article  Google Scholar 

  20. Hashemzadeh, M., Asheghi, B., Farajzadeh, N.: Content-aware image resizing: an improved and shadow-preserving seam carving method. Signal Process. 155, 233–246 (2019). https://doi.org/10.1016/j.sigpro.2018.09.037

    Article  Google Scholar 

  21. Abhayadev, M., and T. Santha.: Efficient retargeting of shadow images using improved CRIST. In: International conference on intelligent computing and control (I2C2), pp. 1–5, 2017

  22. Senturk, Z.K. and Akgun, D.: Seam carving based image retargeting: A survey. In: 1st international informatics and software engineering conference (UBMYK), pp. 1–6, 2019

  23. Chen, L., Fu, G.: Structure-preserving image smoothing with semantic cues. Vis. Comput. 36(10), 2017–2027 (2020). https://doi.org/10.1007/s00371-020-01950-1

    Article  Google Scholar 

  24. Lin, W., Zhang, F., Lian, R., et al.: Seam Carving Algorithm Based on Saliency. In: International Conference on Smart Vehicular Technology, Transactions, Communication and Applications, pp. 282–291, 2017 doi: https://doi.org/10.1007/978-3-319-70730-3_34

  25. Patel, D., Shanmuganathan, S., Raman, S.: Adaptive multiple-pixel wide seam carving. National conference on communications (NCC), pp. 1–6, 2019 doi: https://doi.org/10.1109/NCC.2019.8732245

  26. Xu, J., Kang, H., Chen, F.: Content-aware image resizing using quasi-conformal mapping. Vis. Comput. 34(3), 431–442 (2018). https://doi.org/10.1007/s00371-017-1350-4

    Article  Google Scholar 

  27. Guo, Z. and Zhang, J.: Seam Carving Algorithm for Maintaining the Shape Structure of Significant Objects. In: 2nd International Conference on Artificial Intelligence and Engineering Application (AIEA), pp. 651–658, 2017 doi: https://doi.org/10.12783/dtcse/aiea2017/14995

  28. Zhang, L., Li, K., Ou, Z., Wang, F.: Seam warping: a new approach for image retargeting for small displays. Soft. Comput. 21(2), 447–457 (2017). https://doi.org/10.1007/s00500-015-1795-1

    Article  Google Scholar 

  29. Shafieyan, F., Karimi, N., Mirmahboub, B., et al.: Image retargeting using depth assisted saliency map. Signal Process.: Image Commun. 50, 34–43 (2017). https://doi.org/10.1016/j.image.2016.10.006

    Article  Google Scholar 

  30. Solanki, P., Bhatnagar, C., Jalal, A.S., et al.: Content Aware Image Size Reduction Using Low Energy Maps for Reduced Distortion. In: Proceeding of International Conference on Computer Visual and Image Proceeding, pp. 467–474, 2017 doi: https://doi.org/10.1007/978-981-10-2104-6_42

  31. Guo, Y., Liang, Y., Yu, M., et al.: An improved seam carving algorithm based on image blocking and optimized cumulative energy map. J. Elect. Info. Tech. 40(2), 331–337 (2018). https://doi.org/10.11999/JEIT170501

    Article  MathSciNet  Google Scholar 

  32. Alavi Gharahbagh, A., Yaghmaee, F.: Improved content aware image retargeting using strip partitioning. Int. J. Eng. 31(5), 684–692 (2018)

    Google Scholar 

  33. Patel, D., Nagar, R., Raman, S.: Reflection symmetry aware image retargeting. Pattern Recogn. Lett. 125, 179–186 (2019). https://doi.org/10.1016/j.patrec.2019.04.013

    Article  Google Scholar 

  34. Arai, K.: Modified seam carving by changing resizing depending on the object size in time and space domains. Int. J. Adv. Comput. Sci. Appl. 10(9), 143–150 (2019)

    Google Scholar 

  35. Choi, B., Lee, M., Jung, S.W. and Lu, Y.: Distortion-aware Panoramic Image Resizing Using Seam Carving. In: 2021 International Conference on Electrical, Information, and Communication (ICEIC), pp. 1–2, 2021 doi: https://doi.org/10.1109/ICEIC51217.2021.9369775

  36. Rubinstein, M., Shamir, A., Avidan, S.: Multi-operator media retargeting. ACM Trans. Gr. (TOG). 28(3), 1–11 (2009). https://doi.org/10.1145/1531326.1531329

    Article  Google Scholar 

  37. Dong, W.M., Bao, G.B., Zhang, X.P., et al.: Fast multi-operator image resizing and evaluation. J. Comput. Sci. Technol. 27(1), 121–134 (2012). https://doi.org/10.1007/s11390-012-1211-6

    Article  Google Scholar 

  38. Kiess, J., Guthier, B., Kopf, S., et al.: SeamCrop for image retargeting. Multimedia on Mobile Devices 2012; and Multimedia Content Access: Algorithms and Systems VI. 8304, 83040K (2012) doi: https://doi.org/10.1117/12.906386

  39. Zhou, Y., Chen, Z., Li, W.: Weakly supervised reinforced multi-operator image retargeting. IEEE Trans. Circuits Syst. Video Technol. 31(1), 126–139 (2020). https://doi.org/10.1109/TCSVT.2020.2977943

    Article  Google Scholar 

  40. Valdez-Balderas, D., Muraveynyk, O. and Smith, T.: Fast Hybrid Image Retargeting. In: 2021 IEEE International conference on image processing (ICIP), pp. 1849–1853, 2021 doi: https://doi.org/10.1109/ICIP42928.2021.9506584

  41. Mei, Y., Guo, X., Sun, D., Pan, G. and Zhang, J.: Deep Supervised Image Retargeting. In: 2021 IEEE international conference on multimedia and expo (ICME), pp. 1–6, 2021 doi: https://doi.org/10.1109/ICME51207.2021.9428129

  42. Patel, D., and Raman, S.: Object proposals-based significance map for image retargeting. in: proceedings of 2nd international conference on computer vision and image Processing, pp. 89–101, 2018 doi: https://doi.org/10.1007/978-981-10-7898-9_8

  43. Tsai, Y.J., Lin, H.J., Li, Y.S.: A straight line preserving seam carving technique. Appl. Mech. Mater. 385, 1453–1456 (2013). https://doi.org/10.4028/www.scientific.net/AMM.385-386.1453

    Article  Google Scholar 

  44. Conge, D.D., Kumar, M., Miller, R.L., et al.: Improved seam carving for image resizing. IEEE workshop on signal processing systems, pp. 345–349, 2010 doi: https://doi.org/10.1109/SIPS.2010.5624813

  45. Kumar, M., Conger, D.D., Miller, R.L., et al.: A distortion-sensitive seam carving algorithm for content-aware image resizing. J. Signal Process. Syst. 65(2), 159–169 (2011). https://doi.org/10.1007/s11265-011-0613-y

    Article  Google Scholar 

  46. Kim, H.K., Lee, K.W., Jung, J.Y., et al.: A content-aware image stitching algorithm for mobile multimedia devices. IEEE Trans. Cons. Elect. 57(4), 1875–1882 (2011). https://doi.org/10.1109/TCE.2011.6131166

    Article  Google Scholar 

  47. Zhang, D., Yin, T., Yang, G., Xia, M., Li, L., Sun, X.: Detecting image seam carving with low scaling ratio using multi-scale spatial and spectral entropies. J. Vis. Commun. Image Represent. 48, 281–291 (2017). https://doi.org/10.1016/j.jvcir.2017.07.006

    Article  Google Scholar 

  48. Song, E., Lee, M., Lee, S.: CarvingNet: content-guided seam carving using deep convolution neural network. IEEE Access. 7, 284–292 (2018). https://doi.org/10.1109/ACCESS.2018.2885347

    Article  Google Scholar 

  49. Guo, T., Xu, X.: Salient object detection from low contrast images based on local contrast enhancing and non-local feature learning. Vis. Computer. 45, 1–13 (2020). https://doi.org/10.1007/s00371-020-01964-9

    Article  Google Scholar 

  50. Koo, H.I., Kuk, J.G. and Cho, N.I.: Eliminating structure misalignments using robust matching and image editing based on seam carving. In: 2009 16th IEEE international conference on image processing (ICIP), pp. 209–212, 2009 doi: https://doi.org/10.1109/ICIP.2009.5414470

  51. Vaquero, D., Turk, M., Pulli, K., Tico, M., Gelfand, N.: A survey of image retargeting techniques. Appl. Digital Image Process XXXIII. 7798, 779814 (2010). https://doi.org/10.1117/12.862419

    Article  Google Scholar 

  52. Kiess, J., Kopf, S., Guthier, B., Effelsberg, W.: A survey on content-aware image and video retargeting. Acm Trans. Multim. Comput., Commun., Appl. (TOMM). 14(3), 1–28 (2018). https://doi.org/10.1145/3231598

    Article  Google Scholar 

  53. Chen, Y., Pan, Y., Song, M., Wang, M.: Improved seam carving combining with 3D saliency for image retargeting. Neurocomputing 151, 645–653 (2015). https://doi.org/10.1016/j.neucom.2014.05.089

    Article  Google Scholar 

  54. Frankovich, M., Wong, A.: Enhanced seam carving via integration of energy gradient functionals. IEEE Signal Process. Lett. 18(6), 375–378 (2011). https://doi.org/10.1109/LSP.2011.2140396

    Article  Google Scholar 

  55. Lin, H., Hosu, V., and Saupe, D.: KADID-10k: A large-scale artificially distorted IQA database. In: 2019 Eleventh International Conference on Quality of Multimedia Experiment (QoMEX), pp. 1–3, 2019. doi: https://doi.org/10.1109/QoMEX.2019.8743252

  56. Wang, Z., Zhang, W., Zhou, H.: Perception-guided multi-channel visual feature fusion for image retargeting. Signal Process.: Image Commun. 79, 63–70 (2019). https://doi.org/10.1016/j.image.2019.08.015

    Article  Google Scholar 

  57. Fang, Y., Zeng, K., Wang, Z., Lin, W., Fang, Z., Lin, C.W.: Objective quality assessment for image retargeting based on structural similarity. IEEE J. Emerg. Selec. Topics Circuits Syst. 4(1), 95–105 (2014). https://doi.org/10.1109/JETCAS.2014.2298919

    Article  Google Scholar 

  58. Chen, Y., Liu, L., Tao, J., et al.: The improved image inpainting algorithm via encoder and similarity constraint. Vis. Computer. 36, 1–15 (2020). https://doi.org/10.1007/s00371-020-01932-3

    Article  Google Scholar 

  59. Xin, Z., Fu, S.: User-centric QoE model of visual perception for mobile videos. Vis. Computer 35(9), 1245–1254 (2019). https://doi.org/10.1007/s00371-018-1590-y

    Article  Google Scholar 

  60. Senturk, Z.K., Akgun, D., Senturk, A.: A performance analysis for seam carving algorithm. Int. J. Adv. Stud. Computers, Sci. Eng. 3(12), 5–11 (2014)

    Google Scholar 

  61. Venkataramanan, A.K., Wu, C., Bovik, A.C., Katsavounidis, I., Shahid, Z.: A Hitchhiker’s guide to structural similarity. IEEE Access. 9, 28872–28896 (2021). https://doi.org/10.1109/ACCESS.2021.3056504

    Article  Google Scholar 

  62. Wei, Y., Xu, M.: Detection of lane line based on Robert operator. J. Measure. Eng 9(3), 156–166 (2021). https://doi.org/10.21595/jme.2021.22023

    Article  Google Scholar 

  63. Zhai, G., Min, X.: Perceptual image quality assessment: a survey. Sci. China Inf. Sci. 63(11), 211301 (2020). https://doi.org/10.1007/s11432-019-2757-1

    Article  Google Scholar 

  64. Zhang, Y., Lai, Y.K., Zhang, F.L.: Stereoscopic image stitching with rectangular boundaries. Vis. Computer 35(6), 823–835 (2019). https://doi.org/10.1007/s00371-019-01694-7

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Amity University, Gurugram, Haryana, 122413, India

    Ankit Garg & Anuj Kumar Singh

Authors
  1. Ankit Garg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anuj Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ankit Garg.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Garg, A., Singh, A.K. Analysis of seam carving technique: limitations, improvements and possible solutions. Vis Comput 39, 2683–2709 (2023). https://doi.org/10.1007/s00371-022-02486-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02486-2

Keywords

Search

Navigation