Skip to main content
SpringerLink
Log in

Human-Centric Image Cropping with Partition-Aware and Content-Preserving Features

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13667))

Included in the following conference series:

Abstract

Image cropping aims to find visually appealing crops in an image, which is an important yet challenging task. In this paper, we consider a specific and practical application: human-centric image cropping, which focuses on the depiction of a person. To this end, we propose a human-centric image cropping method with two novel feature designs for the candidate crop: partition-aware feature and content-preserving feature. For partition-aware feature, we divide the whole image into nine partitions based on the human bounding box and treat different partitions in a candidate crop differently conditioned on the human information. For content-preserving feature, we predict a heatmap indicating the important content to be included in a good crop, and extract the geometric relation between the heatmap and a candidate crop. Extensive experiments demonstrate that our method can perform favorably against state-of-the-art image cropping methods on human-centric image cropping task. Code is available at https://github.com/bcmi/Human-Centric-Image-Cropping.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ardizzone, E., et al.: Saliency based image cropping. In: ICIAP (2013)

    Google Scholar 

  2. Cavalcanti, C.S.V.C., et al.: Combining multiple image features to guide automatic portrait cropping for rendering different aspect ratios. In: SITIS (2010)

    Google Scholar 

  3. Chen, H., Wang, B., Pan, T., Zhou, L., Zeng, H.: CropNet: real-time thumbnailing. In: ACMMM (2018)

    Google Scholar 

  4. Chen, J., Bai, G., Liang, S., Li, Z.: Automatic image cropping: a computational complexity study. In: CVPR (2016)

    Google Scholar 

  5. Chen, Y.L., Huang, T.W., Chang, K.H., Tsai, Y.C., Chen, H.T., Chen, B.Y.: Quantitative analysis of automatic image cropping algorithms: a dataset and comparative study. In: WACV (2017)

    Google Scholar 

  6. Chen, Y.L., Klopp, J., Sun, M., Chien, S.Y., Ma, K.L.: Learning to compose with professional photographs on the web. In: ACMMM (2017)

    Google Scholar 

  7. Chen, Z., Xu, Q., Cong, R., Huang, Q.: Global context-aware progressive aggregation network for salient object detection. In: AAAI (2020)

    Google Scholar 

  8. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR (2009)

    Google Scholar 

  9. Deng, Y., Loy, C.C., Tang, X., et al.: Image aesthetic assessment: an experimental survey. IEEE Signal Process. Mag. 34(4), 80–106 (2017)

    Article  Google Scholar 

  10. Esmaeili, S.A., Singh, B., Davis, L.S.: Fast-AT: fast automatic thumbnail generation using deep neural networks. In: CVPR (2017)

    Google Scholar 

  11. Fang, C., Lin, Z., Mech, R., Shen, X.: Automatic image cropping using visual composition, boundary simplicity and content preservation models. In: ACMMM (2014)

    Google Scholar 

  12. Freeman, M.: The photographer’s eye: Composition and design for better digital photos. CRC Press (2007)

    Google Scholar 

  13. Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. PAMI 10(34), 1915–1926 (2012)

    Google Scholar 

  14. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: ICCV (2017)

    Google Scholar 

  15. Hong, C., Du, S., Xian, K., Lu, H., Cao, Z., Zhong, W.: Composing photos like a photographer. In: CVPR (2021)

    Google Scholar 

  16. Hou, Q., Cheng, M.M., Hu, X., Borji, A., Tu, Z., Torr, P.H.: Deeply supervised salient object detection with short connections. In: CVPR (2017)

    Google Scholar 

  17. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: CVPR (2007)

    Google Scholar 

  18. Kao, Y., et al.: Automatic image cropping with aesthetic map and gradient energy map. In: ICASSP (2017)

    Google Scholar 

  19. Krishna, R., et al.: Visual genome: connecting language and vision using crowdsourced dense image annotations. Int. J. Comput. Vision 123(1), 32–73 (2017)

    Article  MathSciNet  Google Scholar 

  20. Li, D., Wu, H., Zhang, J., Huang, K.: A2-RL: aesthetics aware reinforcement learning for image cropping. In: CVPR (2018)

    Google Scholar 

  21. Li, D., Wu, H., Zhang, J., Huang, K.: Fast A3RL: aesthetics-aware adversarial reinforcement learning for image cropping. TIP 28(10), 5105–5120 (2019)

    MathSciNet  MATH  Google Scholar 

  22. Li, D., Zhang, J., Huang, K.: Learning to learn cropping models for different aspect ratio requirements. In: CVPR (2020)

    Google Scholar 

  23. Li, D., Zhang, J., Huang, K., Yang, M.H.: Composing good shots by exploiting mutual relations. In: CVPR (2020)

    Google Scholar 

  24. Li, Q., et al.: Deeper insights into graph convolutional networks for semi-supervised learning. In: AAAI (2018)

    Google Scholar 

  25. Li, X., Li, X., Zhang, G., Zhang, X.: Image aesthetic assessment using a saliency symbiosis network. J. Electron. Imaging 28(2), 023008 (2019)

    Google Scholar 

  26. Li, Z., Zhang, X.: Collaborative deep reinforcement learning for image cropping. In: ICME (2019)

    Google Scholar 

  27. Lu, P., Zhang, H., Peng, X., Jin, X.: An end-to-end neural network for image cropping by learning composition from aesthetic photos (2019)

    Google Scholar 

  28. Lu, W., Xing, X., Cai, B., Xu, X.: Listwise view ranking for image cropping. IEEE Access 7, 91904–91911 (2019)

    Article  Google Scholar 

  29. Mai, L., et al.: Composition-preserving deep photo aesthetics assessment. In: CVPR (2016)

    Google Scholar 

  30. Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: NeurIPS (2019)

    Google Scholar 

  31. Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. PAMI 39, 1137–1149 (2015)

    Article  Google Scholar 

  32. Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Networks 20(1), 61–80 (2008)

    Article  Google Scholar 

  33. She, D., Lai, Y.K., Yi, G., Xu, K.: Hierarchical layout-aware graph convolutional network for unified aesthetics assessment. In: CVPR (2021)

    Google Scholar 

  34. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2015)

    Google Scholar 

  35. Su, Y.C., et al.: Camera view adjustment prediction for improving image composition. arXiv preprint arXiv:2104.07608 (2021)

  36. Sun, J., Ling, H.: Scale and object aware image thumbnailing. IJCV 104(2), 135–153 (2013)

    Article  MathSciNet  Google Scholar 

  37. Tu, Y., Niu, L., Zhao, W., Cheng, D., Zhang, L.: Image cropping with composition and saliency aware aesthetic score map. In: AAAI (2020)

    Google Scholar 

  38. Vig, E., Dorr, M., Cox, D.: Large-scale optimization of hierarchical features for saliency prediction in natural images. In: CVPR (2014)

    Google Scholar 

  39. Wang, W., Shen, J.: Deep cropping via attention box prediction and aesthetics assessment. In: ICCV (2017)

    Google Scholar 

  40. Wang, W., Shen, J., Ling, H.: A deep network solution for attention and aesthetics aware photo cropping. PAMI 41(7), 1531–1544 (2018)

    Article  Google Scholar 

  41. Wei, Z., et al.: Good view hunting: learning photo composition from dense view pairs. In: CVPR (2018)

    Google Scholar 

  42. Yan, J., Lin, S., Bing Kang, S., Tang, X.: Learning the change for automatic image cropping. In: CVPR (2013)

    Google Scholar 

  43. Zeng, H., Li, L., Cao, Z., Zhang, L.: Reliable and efficient image cropping: a grid anchor based approach. In: CVPR (2019)

    Google Scholar 

  44. Zeng, H., Li, L., Cao, Z., Zhang, L.: Grid anchor based image cropping: a new benchmark and an efficient model. PAMI PP(01) (2020)

    Google Scholar 

  45. Zhang, L., Song, M., Yang, Y., Zhao, Q., Zhao, C., Sebe, N.: Weakly supervised photo cropping. TMM 16(1), 94–107 (2013)

    Google Scholar 

  46. Zhang, L., Song, M., Zhao, Q., Liu, X., Bu, J., Chen, C.: Probabilistic graphlet transfer for photo cropping. TIP 22(2), 802–815 (2012)

    MathSciNet  MATH  Google Scholar 

  47. Zhang, M., Zhang, L., Sun, Y., Feng, L., Ma, W.: Auto cropping for digital photographs. In: ICME (2005)

    Google Scholar 

  48. Zhang, X., Li, Z., Constable, M., Chan, K.L., Tang, Z., Tang, G.: Pose-based composition improvement for portrait photographs. IEEE Trans. Circuits Syst. Video Technol. 29(3), 653–668 (2018)

    Article  Google Scholar 

  49. Zhang, Y., Sun, X., Yao, H., Qin, L., Huang, Q.: Aesthetic composition represetation for portrait photographing recommendation. In: ICIP (2012)

    Google Scholar 

  50. Zhao, T., Wu, X.: Pyramid feature attention network for saliency detection. In: CVPR (2019)

    Google Scholar 

Download references

Acknowledgement

The work is supported by Shanghai Municipal Science and Technology Key Project (Grant No. 20511100300), Shanghai Municipal Science and Technology Major Project, China (2021SHZDZX0102), and National Science Foundation of China (Grant No. 61902247).

Author information

Authors and Affiliations

  1. MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China

    Bo Zhang, Li Niu, Xing Zhao & Liqing Zhang

Authors
  1. Bo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xing Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liqing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Niu .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 12305 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, B., Niu, L., Zhao, X., Zhang, L. (2022). Human-Centric Image Cropping with Partition-Aware and Content-Preserving Features. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13667. Springer, Cham. https://doi.org/10.1007/978-3-031-20071-7_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20071-7_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20070-0

  • Online ISBN: 978-3-031-20071-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation