Skip to main content
SpringerLink
Log in

Change-Based Image Cropping with Exclusion and Compositional Features

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Image cropping is a common operation used to improve the visual quality of photographs. In this paper, we present an automatic cropping technique that accounts for the two primary considerations of people when they crop: removal of distracting content, and enhancement of overall composition. Our approach utilizes a large training set consisting of photos before and after cropping by expert photographers to learn how to evaluate these two factors in a crop. In contrast to the many methods that exist for general assessment of image quality, ours specifically examines differences between the original and cropped photo in solving for the crop parameters. To this end, several novel image features are proposed to model the changes in image content and composition when a crop is applied. The effectiveness of each feature is empirically analyzed in determining a final feature set for crop computation. Our experiments demonstrate improvements of our method over recent cropping algorithms on a broad range of images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. Parameters: http://research.microsoft.com/en-us/downloads/94289852-5e1a-4bc7-953d-f0ec0c6d821e/default.aspx Images: http://mmlab.ie.cuhk.edu.hk/Cropping_Dataset_Images.rar.

  2. Our previous work (Yan et al. (2013)) was implemented in Matlab, and the current work is in C++.

References

  • Avidan, S., & Shamir, A. (2007). Seam carving for content-aware image resizing. ACM Transactions on Graphics, 26(3), 10.

    Article  Google Scholar 

  • Cao, C., Liu, J., & Zuo, Y. (2013). Automatic image cropping via the novel saliency detection algorithm. In IEEE international conference on software engineering and service science (ICSESS) (pp. 955–958).

  • Chang, C.-C., & Lin, C.-J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2, 27:1–27:27.

    Article  Google Scholar 

  • Cheng, B., Ni, B., Yan, S., & Tian, Q. (2010). Learning to photograph. In ACM multimedia (pp. 291–300).

  • Cheng, M.-M. Zhang, G.-X., Mitra, N., Huang, X., & Hu, S.-M. (2011). Global contrast based salient region detection. In Proceedings IEEE conference on computer vision and pattern recognition (CVPR) (pp. 409–416).

  • Ciocca, G., Cusano, C., Gasparini, F., & Schettini, R. (2007). Self-adaptive image cropping for small displays. IEEE Transactions on Consumer Electronics, 53(4), 1622–1627.

    Article  Google Scholar 

  • Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In Proceedings IEEE conference on computer vision and pattern recognition (CVPR) (pp. I:886–893).

  • Endres, I., & Hoiem, D. (2014). Category-independent object proposals with diverse ranking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(2), 222–234.

    Article  Google Scholar 

  • Felzenszwalb, P. F., & Huttenlocher, D. P. (2004). Efficient graph-based image segmentation. International Journal of Computer Vision, 59(2), 167–181.

    Article  Google Scholar 

  • Itti, L., & Koch, C. (1998). A model of saliency based visual attention of rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254–1259.

    Article  Google Scholar 

  • Ke, Y., Tang, X., & Jing, F. (2006). The design of high-level features for photo quality assessment. In Proceedings IEEE conference on computer vision and pattern recognition (CVPR).

  • Kennedy, L., van Zwol, R., Torzec, N., & Tseng, B. (2011). Learning crop regions for content-aware generation of thumbnail images. In ACM international conference on multimedia retrieval (pp. 30:1–30:8).

  • Liu, L., Chen, R., Wolf, L., & Cohen-Or, D. (2010). Optimizing photo composition. Computer Graphics Forum, 29(2), 469–478.

    Article  Google Scholar 

  • Luo, J. (2007). Subject content-based intelligent cropping of digital photos. In IEEE international conference on multimedia and expo (pp. 2218–2221).

  • Luo, Y., & Tang, X. (2008). Photo and video quality evaluation: Focusing on the subject. In Proceedings European conference on computer vision (ECCV) (pp. III:386–399).

  • Luo, W., Wang, X., & Tang, X. (2011). Content-based photo quality assessment. In Proceedings international conference on computer vision (ICCV) (pp. 2206–2213).

  • Ma, M., & Guo, J. K. (2004). Automatic image cropping for mobile devices with built-in camera. In Consumer communication and networking (pp. 710–711).

  • Marchesotti, L., Cifarelli, C., & Csurka, G. (2009). A framework for visual saliency detection with applications to image thumbnailing. In Proceedings international conference on computer vision (ICCV) (pp. 2232–2239).

  • Nielsen, F., Owada, S., & Hasegawa, Y. (2006). Autoframing: A recommendation system for detecting undesirable elements and cropping automatically photos. In IEEE international conference on multimedia and expo (pp. 417–420).

  • Nishiyama, M., Okabe, T., Sato, Y., & Sato, I. (2009). Sensation-based photo cropping. In ACM multimedia.

  • Park, J., Lee, J.-Y., Tai, Y.-W., & Kweon, I. S. (2012). Modeling photo composition and its application to photo re-arrangement. In Proceeidngs international conference on image processing (ICIP) (pp. 2741–2744).

  • Pritch, Y., Kav-Venaki, E., & Peleg, S. (2009). Shift-map image editing. In Proceeidngs international conference on computer vision (ICCV), Kyoto (pp. 151–158).

  • Santella, A., Agrawala, M., DeCarlo, D., Salesin, D., & Cohen, M. (2006). Gaze-based interaction for semi-automatic photo cropping. In ACM SIGCHI (pp. 771–780).

  • Stentiford, F. (2007) Attention based auto image cropping. In ICVS workshop on computational attention & application.

  • Stricker, M., & Orengo, M. (1995). Similarity of color images. In Storage retrieval image video database (pp. 381–392).

  • Suh, B., Ling, H., Bederson, B. B., & Jacobs, D. W. (2003) Automatic thumbnail cropping and its effectiveness. In ACM symposium UIST (pp. 95–104).

  • Tang, X., Luo, W., & Wang, X. (2013). Content-based photo quality assessment. IEEE Transactions on Multimedia, 15(8), 1930–1943.

  • Wang, Y.-S., Tai, C.-L., Sorkine, O., & Lee, T.-Y. (2008). Optimized scale-and-stretch for image resizing. ACM Transactions on Graphics, 27(5).

  • Wei, Y., Wen, F., Zhu, W., & Sun, J. (2012). Geodesic saliency using background priors. In Proceedings European conference on computer vision (ECCV) (pp. 29–42).

  • Xiao, R., Zhu, H., Sun, H., & Tang, X. (2007). Dynamic cascades for face detection. In Proceedings international conference on computer vision (ICCV).

  • Yan, J., Lin, S., Kang, S. B., & Tang, X. (2013). Learning the change for automatic image cropping. In Proceedings IEEE conference on computer vision and pattern recognition (CVPR) (pp. 971–978).

  • Zahn, C. T., & Roskies, R. Z. (1972). Fourier descriptors for plane closed curves. IEEE Transactions on Computers, 21(3), 269–281.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang, L., Song, M., Zhao, Q., Liu, X., Bu, J., & Chen, C. (2012) Probabilistic graphlet transfer for photo cropping. IEEE Trans. on Image Processing, 22(2), 802–815.

  • Zhang, M., Zhang, L., Sun, Y., Feng, L., & Ma, W. (2005). Auto cropping for digital photographs. In IEEE international conference on multimedia and expo.

Download references

Author information

Authors and Affiliations

  1. The Chinese University of Hong Kong, Hong Kong, Hong Kong

    Jianzhou Yan & Xiaoou Tang

  2. Microsoft Research, Redmond, USA

    Stephen Lin & Sing Bing Kang

Authors
  1. Jianzhou Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephen Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sing Bing Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoou Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianzhou Yan.

Additional information

Communicated by Y. Sato.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, J., Lin, S., Kang, S.B. et al. Change-Based Image Cropping with Exclusion and Compositional Features. Int J Comput Vis 114, 74–87 (2015). https://doi.org/10.1007/s11263-015-0801-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-015-0801-5

Keywords

Search

Navigation