Skip to main content

Advertisement

SpringerLink
Log in

Performance analysis of seam diversion based image retargeting technique based on edge detection operators

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Nowadays various resizing algorithms are being used to resize the images in a content-aware fashion. The seam diversion-based image retargeting (SDIR) algorithm improves the process of the existing seam carving technique. In this paper, the performance of the SDIR algorithm is analyzed based on a number of search and seam diversion operations. Further, the performance of the algorithm is analyzed based on the type of edge detection operators. To analyze the performance two experimentations are conducted. In experimentation-1, different edge detection operators are used to produce an importance map of the image. Further, the computational time of the SDIR algorithm is tested based on the identified line structures and other image objects. In experimentation-2, the performance of the algorithm is analyzed based on the visual quality of retargeted results and the quality of importance maps. To achieve this objective an objective image quality assessment (IQA) is carried out based on the structural similarity index measure (SSIM). The obtained results from phase-2 experimentation show that the distortions on the prominent regions can easily be noticeable to the human eyes when the algorithm performs many seam diversion operations. After experiment-2, a comparative analysis is conducted to justify its performance among the existing state of the arts. To meet this objective a total of 5 types of importance map is supplied to the algorithms to obtain the objective and subjective scores. After both analyses, the SDIR algorithm outperforms the other state of arts and minimize the structural deformations on the prominent objects of the image.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data Availability

In this manuscript, a dataset used in the experimentation and analysis is freely available on https://people.csail.mit.edu/mrub/retargetme/download.html#analysis. The dataset contains all the source images having different attributes, retargeted results, and user data for the researchers for further improvement and investigation.

References

  1. Abhayadev M, Santha T (2019) Multi-operator content aware image retargeting on natural images. J Sci Ind Res 78(4):193–198. https://doi.org/10.1007/s11042-021-11376-z

    Article  Google Scholar 

  2. Aggarwal N, Karl WC (2006) Line detection in images through regularized Hough transform. IEEE Trans Image Process 15(3):582–591. https://doi.org/10.1109/TIP.2005.863021

    Article  Google Scholar 

  3. Ahmadi M, Karimi N, Samavi S (2021) Context-aware saliency detection for image retargeting using convolutional neural networks. Multimed Tools Appl 80(8):11917–11941. https://doi.org/10.1007/s11042-020-10185-0

    Article  Google Scholar 

  4. Arai K (2019) Modified seam carving by changing resizing depending on the object size in time and space domains. Int J Adv Comput Sci Appl 10(9):143–150. https://doi.org/10.14569/IJACSA.2019.0100919

    Article  Google Scholar 

  5. Asheghi B, Salehpour P, Khiavi AM, Hashemzadeh M (2022) A comprehensive review on content-aware image retargeting: from classical to state-of-the-art methods. Signal Process 195:108496. https://doi.org/10.1016/j.sigpro.2022.108496

    Article  Google Scholar 

  6. Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. In ACM SIGGRAPH 2007 papers 26(3):10-es. https://doi.org/10.1145/1276377.1276390

  7. Bakurov I, Buzzelli M, Schettini R, Castelli M, Vanneschi L (2022) Structural similarity index (SSIM) revisited: a data-driven approach. Expert Syst Appl 189:116087. https://doi.org/10.1016/j.eswa.2021.116087

    Article  Google Scholar 

  8. Chai X, Shao F, Jiang Q, Ho YS (2019) MSTGAR: multioperator-based stereoscopic thumbnail generation with arbitrary resolution. IEEE Trans Multimed 22(5):1208–1219. https://doi.org/10.1109/TMM.2019.2939707

    Article  Google Scholar 

  9. Chen Y, Pan Y, Song M, Wang M (2015) Improved seam carving combining with 3D saliency for image retargeting. Neurocomputing 151:645–653. https://doi.org/10.1016/j.neucom.2014.05.089

    Article  Google Scholar 

  10. Cui J, Cai Q, Lu H, Jia Z, Tang M (2020) Distortion-aware image retargeting based on continuous seam carving model. Signal Process 166:107242. https://doi.org/10.1016/j.sigpro.2019.107242

    Article  Google Scholar 

  11. Fang Y, Fang Z, Yuan F, Yang Y et al (2016) Optimized multioperator image retargeting based on perceptual similarity measure. IEEE Trans Syst Man, Cybern Syst 47(11):2956–2966. https://doi.org/10.1109/TSMC.2016.2557225

    Article  Google Scholar 

  12. Garg A, Negi A (2020) Structure preservation in content-aware image retargeting using multi-operator. IET Image Process 14(13):2965–2975. https://doi.org/10.1049/iet-ipr.2019.1032

    Article  Google Scholar 

  13. Garg A, Singh AK (2022) Analysis of seam carving technique: limitations, improvements and possible solutions. Vis Comput:1–27. https://doi.org/10.1007/s00371-022-02486-2

  14. Garg A, Negi A, Jindal P (2021) Structure preservation of image using an efficient content-aware image retargeting technique. SIViP 15(1):185–193. https://doi.org/10.1007/s11760-020-01736-x

    Article  Google Scholar 

  15. Garg A, Nayyar A, Singh AK (2022) Improved seam carving for structure preservation using efficient energy function. Multimed Tools Appl 81(9):12883–12924. https://doi.org/10.1007/s11042-022-12003-1

    Article  Google Scholar 

  16. Guo Z, Zhang J (2017) Seam Carving Algorithm for Maintaining the Shape Structure of Significant Objects. In: 2nd Int Conf Arti Inte Eng App (AIEA), pp. 651–658. https://doi.org/10.12783/dtcse/aiea2017/14995

  17. Guo Y, Liang Y, Yu M, Zhang T (2018) An improved seam carving algorithm based on image blocking and optimized cumulative energy map. J Electron Inf Technol 40(2):331–337. https://doi.org/10.11999/JEIT170501

    Article  Google Scholar 

  18. Hashemzadeh M, Asheghi B, Farajzadeh N (2019) Content-aware image resizing: an improved and shadow-preserving seam carving method. Signal Process 155:233–246. https://doi.org/10.1016/j.sigpro.2018.09.037

    Article  Google Scholar 

  19. Kajiura N, Kosugi S, Wang X, Yamasaki T (2020) Self-play reinforcement learning for fast image retargeting. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 1755-1763. https://doi.org/10.1145/3394171.3413857

  20. Kumar S, Upadhyay, A K, Dubey P et al (2017) Comparative analysis for Edge Detection Techniques. In: 2021 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS), pp. 675–681. https://doi.org/10.1109/ICCCIS51004.2021.9397225

  21. Lin W, Zhang F, Lian R et al (2017) Seam Carving Algorithm Based on Saliency. In: Int. Conf. on Smart Vehi Tech, Trans, Comm App, pp. 282–291. https://doi.org/10.1007/978-3-319-70730-3_34

  22. Liu C, Yuen J, Torralba A et al (2008) Sift flow: Dense correspondence across different scenes. In European conference on computer vision, pp. 28–42. https://doi.org/10.1007/978-3-540-88690-7_3

  23. Liu C, Yuen J, Torralba A (2010) Sift flow: dense correspondence across scenes and its applications. IEEE Trans Pattern Anal Mach Intell 33(5):978–994. https://doi.org/10.1109/TPAMI.2010.147

    Article  Google Scholar 

  24. Ma WD, Chang HH, Shih TK et al (2012) Seam carving based on improved line detection. In: 2012 Int Sym on Intell Sig Proc Comm Sys, pp. 498–503. https://doi.org/10.1109/ISPACS.2012.6473541

  25. Mei Y, Guo X, Sun D et al (2021) Deep Supervised Image Retargeting. In: 2021 IEEE international conference on multimedia and expo (ICME), pp. 1–6. https://doi.org/10.1109/ICME51207.2021.9428129

  26. Niu Y, Liu F, Li X, Gleicher M (2012) Image resizing via non-homogeneous warping. Multimed Tools Appl 56(3):485–508. https://doi.org/10.1007/s11042-010-0613-0

    Article  Google Scholar 

  27. Parsania P, Virparia DPV (2014) A review: image interpolation techniques for image scaling. Int J Innov Res Comput Commun Eng 2(12):7409–7414. https://doi.org/10.15680/ijircce.2014.0212024

    Article  Google Scholar 

  28. Patel D, Raman S (2019) Accelerated seam carving for image retargeting. IET Image Process 13(6):885–895. https://doi.org/10.1049/iet-ipr.2018.5283

    Article  Google Scholar 

  29. Patel D, Shanmuganathan S, Raman S (2019) Adaptive multiple-pixel wide seam carving. In: National Conference on Communications (NCC), pp. 1–6. https://doi.org/10.1109/NCC.2019.8732245

  30. Patel D, Nagar R, Raman S (2019) Reflection symmetry aware image retargeting. Pattern Recogn Lett 125:179–186. https://doi.org/10.1016/j.patrec.2019.04.013

    Article  Google Scholar 

  31. Qi S, Chi YTJ, Peter A et al (2016) CASAIR: content and shape-aware image retargeting and its applications. IEEE Trans Image Proc 25(5):2222–2232. https://doi.org/10.1109/TIP.2016.2528040

    Article  MathSciNet  MATH  Google Scholar 

  32. Qiu Z, Ren T, Liu Y, Bei J et al (2013) Multi-operator image retargeting based on automatic quality assessment. In: 2013 Seventh Int. Conf. on Image and Grap., pp. 428–433. https://doi.org/10.1109/ICIG.2013.92

  33. Rajput V, Ansari IA (2020) Image tamper detection and self-recovery using multiple median watermarking. Multimed Tools Appl 79(47):35519–35535. https://doi.org/10.1007/s11042-019-07971-w

    Article  Google Scholar 

  34. Setiadi DRIM (2021) PSNR vs SSIM: imperceptibility quality assessment for image steganography. Multimed Tools Appl 80(6):8423–8444. https://doi.org/10.1007/s11042-020-10035-z

    Article  Google Scholar 

  35. Shilpa M, Gopalakrishna MT, Naveena C (2020) Approach for shadow detection and removal using machine learning techniques. IET Image Process 14(13):2998–3005. https://doi.org/10.1049/iet-ipr.2020.0001

    Article  Google Scholar 

  36. Solanki P, Bhatnagar C, Jalal AS et al (2017) Content aware image size reduction using low energy maps for reduced distortion. In: Proceedings of Int Conf Comp Vis Image Proc, pp. 467–474. https://doi.org/10.1007/978-981-10-2104-6_42

  37. Suh B, Ling H, Bederson B B et al (2003) Automatic thumbnail cropping and its effectiveness. In: Proceedings of the 16th annual ACM symposium on User interface software and technology, pp. 95–104. https://doi.org/10.1145/964696.964707

  38. Tang Z, Yao J, Zhang Q (2022) Multi-operator image retargeting in compressed domain by preserving aspect ratio of important contents. Multimed Tools Appl 81(1):1501–1522. https://doi.org/10.1007/s11042-021-11376-z

    Article  Google Scholar 

  39. Valdez-Balderas D, Muraveynyk O, Smith T (2021) Fast Hybrid Image Retargeting. In: 2021 IEEE International conference on image processing (ICIP), pp. 1849-1853. https://doi.org/10.1109/ICIP42928.2021.9506584

  40. Wang Y S, Tai C L, Sorkine O et al, (2008) Optimized scale-and-stretch for image resizing. In: ACM SIGGRAPH Asia 2008 papers, pp. 1-8, 2008. https://doi.org/10.1145/1457515.1409071

  41. Wang W, Shen J, Yu Y, Ma KL (2016) Stereoscopic thumbnail creation via efficient stereo saliency detection. IEEE Trans Vis Comput Graph 23(8):2014–2027. https://doi.org/10.1109/TVCG.2016.2600594

    Article  Google Scholar 

  42. Wang S, Tang Z, Dong W et al (2020) Multi-operator video retargeting method based on improved seam carving. In: 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC), pp. 1609-1614. https://doi.org/10.1109/ITOEC49072.2020.9141774

  43. Wei D Y, Chou Y C, Su P C (2018) A multi-operator retargeting scheme for compressed videos. In: 2018 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), pp. 1–2. https://doi.org/10.1109/ICCE-China.2018.8448819

  44. Zhai G, Min X (2020) Perceptual image quality assessment: a survey. SCIENCE CHINA Inf Sci 63(11):1–52. https://doi.org/10.1007/s11432-019-2757-1

    Article  Google Scholar 

  45. Zhang L, Li K, Ou Z, Wang F (2017) Seam warping: a new approach for image retargeting for small displays. Soft Comput 21(2):447–457. https://doi.org/10.1007/s00500-015-1795-1

    Article  Google Scholar 

  46. Zhang Q, Tang Z, Jiang H, Chang K (2017) Multi-operator image retargeting with preserving aspect ratio of important contents. In: Pacific Rim Conference on Multimedia, pp. 306–315. https://doi.org/10.1007/978-3-319-77383-4_30

  47. Zhang Y, Sun Z, Jiang P, Huang Y, Peng J (2017) Hybrid image retargeting using optimized seam carving and scaling. Multimed Tools Appl 76(6):8067–8085. https://doi.org/10.1007/s11042-016-3318-1

    Article  Google Scholar 

  48. Zhang D, Yin T, Yang G, Xia M, Li L, Sun X (2017) Detecting image seam carving with low scaling ratio using multi-scale spatial and spectral entropies. J Vis Commun Image Represent 48:281–291. https://doi.org/10.1016/j.jvcir.2017.07.006

    Article  Google Scholar 

  49. Zhou B, Wang X, Cao S, Xiang K, Zhao S (2016) Optimal bi-directional seam carving for compressibility-aware image retargeting. J Vis Comm Image Rep 41:21–30. https://doi.org/10.1016/j.jvcir.2016.09.002

    Article  Google Scholar 

  50. Zhou Y, Chen Z, Li W (2020) Weakly supervised reinforced multi-operator image retargeting. IEEE Trans Circuits Syst Vid Technol 31(1):126–139. https://doi.org/10.1109/TCSVT.2020.2977943

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing, University of Engineering & Technology (UETR), Roorkee, Uttarakhand, 247667, India

    Ankit Garg & Anuj Kumar Singh

Authors
  1. Ankit Garg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anuj Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ankit Garg.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Garg, A., Singh, A.K. Performance analysis of seam diversion based image retargeting technique based on edge detection operators. Multimed Tools Appl 82, 23207–23250 (2023). https://doi.org/10.1007/s11042-022-14157-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14157-4

Keywords

Search

Navigation