Skip to main content

NIF-based seam carving for image resizing

Abstract

In this paper, improving the existing seam carving technique, we propose a novel image-resizing method which considers both texture and color information of the original image. Neighborhood inhomogeneity factor (NIF), which describes the image inhomogeneity, is used to compute the saliency map of an image in the Lab space. Although a saliency map well preserves the uniformity of important areas within an image, it lacks details of the image. Hence, we propose to combine a gradient map of an image with the saliency map to generate a final energy map for seam carving. Extensive experiments and comparisons are conducted to evaluate the performance of the proposed NIF-based seam carving method. Experimental results demonstrate that the proposed method preserves the original details and scale of important areas and achieves robust resizing results with reducing distortion, even when the background shows a high degree of resemblance in appearance to the contained objects .

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. 1.

    Shamir, A., Sorkine, O.: Visual media retargeting. In: ACM SIGGRAPH ASIA 2009 Courses. ACM (2009)

  2. 2.

    Vaquero, D., Turk, M., Pulli, K., Tico, M., Gelfand, N.: A survey of image retargeting techniques. In: SPIE Optical Engineering + Applications. International Society for Optics and Photonics, pp. 779814–779814 (2010)

  3. 3.

    Chen, L., Xie, X., Fan, X., Ma, W., Zhang, H., Zhou, H.: A visual attention model for adapting images on small displays. Multimed. Syst. 9(4), 353–364 (2003)

    Article  Google Scholar 

  4. 4.

    Liu, H., Xie, X., Ma, W., Zhang, H.: Automatic browsing of large pictures on mobile devices. In: Proceedings of the Eleventh ACM International Conference on Multimedia. ACM, pp. 148–155 (2003)

  5. 5.

    Suh, B., Ling, H., Bederson, B.B., Jacobs, D.W.: Automatic thumbnail cropping and its effectiveness. In: Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology, pp. 95–104. ACM (2003)

  6. 6.

    Zhang, M., Zhang, L., Sun, Y., Feng, L., Ma, W.: Auto cropping for digital photographs. In: IEEE International Conference on Multimedia and Expo, 2005. ICME 2005

  7. 7.

    Santella, A., Agrawala, M., DeCarlo, D. Salesin, D., Cohen, M.: Gaze-based interaction for semi-automatic photo cropping. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 771–780. ACM (2006)

  8. 8.

    Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2006, vol. 1, pp. 419–426 (2006)

  9. 9.

    Nishiyama, M., Okabe, T., Sato, Y., Sato, I.: Sensation-based photo cropping. In: Proceedings of the 17th ACM International Conference on Multimedia, pp. 669–672. ACM (2009)

  10. 10.

    Liu, F., Gleicher, M.: Automatic image retargeting with fisheye-view warping. In: Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, pp 153–162. ACM (2005)

  11. 11.

    Wang, Y., Tai, C., Sorkine, O., Lee, T.: Optimized scale-and-stretch for image resizing. In: ACM Transactions on Graphics (TOG), vol. 27, p. 118, ACM (2008)

  12. 12.

    Zhang, G., Cheng, M., Hu, S., Martin, R.R.: A shape-preserving approach to image resizing. In: Computer Graphics Forum, vol. 28, pp. 1897–1906. Wiley, New York (2009)

  13. 13.

    Setlur, V., Takagi, S., Raskar, R., Gleicher, M., Gooch, B.: Automatic image retargeting. In: Proceedings of the 4th International Conference on Mobile and Ubiquitous Multimedia, ACM. pp 59–68 (2005)

  14. 14.

    Avidan, S., Shamir, A.: Seam carving for content-aware image resizing. In: ACM Transactions on graphics (TOG), ACM, vol. 26 (2007)

  15. 15.

    Rubinstein, M., Shamir, A., Avidan S.: Improved seam carving for video retargeting. In: ACM Transactions on Graphics (TOG), vol. 27, ACM (2008)

  16. 16.

    Achanta, R., Susstrunk, S.: Saliency detection for content-aware image resizing. In: 16th IEEE International Conference on Image Processing (ICIP), pp. 1005–1008 (2009)

  17. 17.

    Guo, Y., Liu, F., Shi, J., Zhou, Z., Gleicher, M.: Image retargeting using mesh parametrization. IEEE Trans. Multimed. 11(5), 856–867 (2009)

    Article  Google Scholar 

  18. 18.

    Mansfield, A., Gehler, P., Van Gool, L. , Rother, C.: Scene carving: scene consistent image retargeting. In: Computer Vision-ECCV 2010, pp 143–156. Springer, Berlin (2010)

  19. 19.

    Li, B, Duan, L., Wang, J., Chen, J., Ji, R., Gao, W.: Grid-based retargeting with transformation consistency smoothing. In: Advances in Multimedia Modeling, pp. 12–24. Springer, Berlin (2011)

  20. 20.

    Li, B., Chen, Y., Wang, J., Duan, L., Gao, W.: Fast retargeting with adaptive grid optimization. In: IEEE International Conference on Multimedia and Expo (ICME), pp. 1–4 (2011)

  21. 21.

    Wolf, L., Guttmann, M., Cohen-Or, D.: Non-homogeneous content-driven video-retargeting. In: IEEE 11th International Conference on Computer Vision, 2007, ICCV 2007, pp. 1–6 (2007)

  22. 22.

    Krähenbühl, P., Lang, M., Hornung, A., Gross, M.: A system for retargeting of streaming video. ACM Trans. Graph. (TOG) 28(5), 126 (2009)

    Article  Google Scholar 

  23. 23.

    Dong, W., Zhou, N., Paul, J.-C., Zhang, X.: Optimized image resizing using seam carving and scaling. In: ACM Transactions on Graphics (TOG), vol. 28, ACM (2009)

  24. 24.

    Rubinstein, M., Shamir, A., Avidan, S.: Multi-operator media retargeting. In: ACM Transactions on Graphics (TOG), vol. 28, ACM (2009)

  25. 25.

    Laffont, P.-Y., Jun, J.Y., Wolf, C., Tai, Y., Idrissi, K., Drettakis, G., Yoon, S.-E.: Interactive content-aware zooming. In: Proceedings of Graphics Interface 2010, pp. 79–87. Canadian Information Processing Society (2010)

  26. 26.

    Chang, C., Chuang, Y.: A line-structure-preserving approach to image resizing. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1075–1082 (2012)

  27. 27.

    Panozzo, D., Weber, O., Sorkine, O.: Robust image retargeting via axis-aligned deformation. In: Computer Graphics Forum, vol. 31, pp. 229–236. Wiley, New York (2012)

  28. 28.

    Sun, J., Ling, H.: Scale and object aware image retargeting for thumbnail browsing. In: IEEE International Conference on Computer Vision (ICCV), pp. 1511–1518 (2011)

  29. 29.

    Fang, Y., Chen, Z., Lin, W.: Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans. Image Process. 21(9), 3888–3901 (2012)

    MathSciNet  Article  Google Scholar 

  30. 30.

    Greisen, P., Lang, M., Heinzle, S., Smolic, A.: Algorithm and vlsi architecture for real-time 1080p60 video retargeting. In: Proceedings of the Fourth ACM SIGGRAPH/Eurographics Conference on High-Performance Graphics, pp. 57–66. Eurographics Association (2012)

  31. 31.

    Li, X., Ling, H.: Learning based thumbnail cropping. In: IEEE International Conference on Multimedia and Expo, 2009. ICME 2009. pp. 558–561 (2009)

  32. 32.

    Luo, Y., Yuan, J., Xue, P., Tian, Q.: Saliency density maximization for efficient visual objects discovery. IEEE Trans. Circuits Syst. Video Technol. 21(12), 1822–1834 (2011)

    Article  Google Scholar 

  33. 33.

    Gal, R., Sorkine, O., Cohen-Or, D.: Feature-aware texturing. In: Proceedings of the 17th Eurographics Conference on Rendering Techniques, pp. 297–303. Eurographics Association (2006)

  34. 34.

    Shi, L., Wang, J., Duan, L., Lu, H.: Consumer video retargeting: context assisted spatial-temporal grid optimization. In: Proceedings of the 17th ACM International Conference on Multimedia, pp. 301–310. ACM (2009)

  35. 35.

    Li, B., Duan, L., Wang, J., Ji, R., Lin, C., Gao, W.: Spatiotemporal grid flow for video retargeting. IEEE Trans. Image Process. 23(4), 1615–1628 (2014)

  36. 36.

    Ding, J., Shen, J., Pang, H.H., Chen, S., Yang, J.: Exploiting intensity inhomogeneity to extract textured objects from natural scenes. In: Computer Vision-ACCV 2009, pp. 1–10. Springer, Berlin (2010)

  37. 37.

    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

Download references

Acknowledgments

We thank Yu-Shuen Wang for providing their image resizing software for the comparison presented in Fig. 8. This work is supported in part by 973 Program under Grants 2012CB316304, in part by the NSFC under Grant 61103058, 61103059, 61173104 and 61272220, in part by the NSF of Jiangsu Province under Grants BK2011700, BK2012399 and Open Project Program of the National Laboratory of Pattern Recognition (NLPR).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jinhui Tang.

Additional information

Communicated by T. Plagemann.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Guo, D., Ding, J., Tang, J. et al. NIF-based seam carving for image resizing. Multimedia Systems 21, 603–613 (2015). https://doi.org/10.1007/s00530-014-0425-6

Download citation

Keywords

  • Image resizing
  • Seam carving
  • Neighborhood inhomogeneity factor
  • Content-aware image manipulation