Skip to main content

ILMICA - Interactive Learning Model of Image Collage Assessment: A Transfer Learning Approach for Aesthetic Principles

  • 1861 Accesses

Part of the Lecture Notes in Computer Science book series (LNCS,volume 13142)


The beauty of moments can be expressed in many ways. One of them is the image collage which captures events and expresses emotions. Nowadays there is a large number of digital images. Aesthetic analyses of image collages are rarely performed due to their complexity and time-consuming nature. For this reason, this is an important issue that has to be addressed. In this paper, we propose an interactive learning model for image collage assessment. It consists of two components: A pre-trained convolutional neural network with built-in knowledge about aesthetics obtained from single image analysis, and an “Interactive Transfer Learning” component specialized in collage aesthetics which is adaptable via Active Learning. We present a mixed method study in which rules for software-based collage generation are identified and a dataset of automatically generated collages representative of the rules is created. ILMICA’s performance is analyzed by a user survey. It is found that the knowledge transfer from single image assessment to collage assessment works: ILMICA can assess collage aesthetics based on predefined rules, thereby demonstrating the system’s ability to learn. Thus, this process can alleviate the end user and simplify aesthetic collage evaluations.


  • Multimedia
  • Aesthetic collage evaluation
  • User-centered design
  • Convolutional neural networks
  • Transfer and active learning

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. Abdenebaoui, L., Meyer, B., Bruns, A., Boll, S.: UNNA: a unified neural network for aesthetic assessment. In: 2018 International Conference on Content-Based Multimedia Indexing (CBMI), La Rochelle, pp. 1–6. IEEE (2018).

  2. Atkins, C.B.: Blocked recursive image composition. In: Proceeding of the 16th ACM International Conference on Multimedia - MM 2008, Vancouver, British Columbia, Canada, pp. 821–824. ACM Press (2008).

  3. Bossard, L., Guillaumin, M., Van Gool, L.: Event recognition in photo collections with a stopwatch HMM. In: 2013 IEEE International Conference on Computer Vision, Sydney, Australia, pp. 1193–1200. IEEE (2013).

  4. Ceroni, A., Solachidis, V., Niederée, C., Papadopoulou, O., Kanhabua, N., Mezaris, V.: To keep or not to keep: an expectation-oriented photo selection method for personal photo collections. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, Shanghai, China, pp. 187–194. ACM Press (2015).

  5. Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006).

    CrossRef  Google Scholar 

  6. Deng, Y., Loy, C.C., Tang, X.: Image aesthetic assessment: an experimental survey. IEEE Signal Process. Mag. 34(4), 80–106 (2017).

    CrossRef  Google Scholar 

  7. Diakopoulos, N., Essa, I.: Mediating photo collage authoring. In: Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, Seattle, WA, USA, pp. 183–186. ACM Press (2005)

    Google Scholar 

  8. Fogarty, J., Forlizzi, J., Hudson, S.E.: Aesthetic information collages: generating decorative displays that contain information. In: Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology, Orlando, Florida, pp. 141–150. ACM Press (2001).

  9. Girgensohn, A., Chiu, P.: Stained glass photo collages. In: IEEE International Conference on Image Processing, vol. 2, pp. 871–874 (2004)

    Google Scholar 

  10. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  11. Inkpen, K., Chancellor, S., De Choudhury, M., Veale, M., Baumer, E.P.S.: Where is the human?: bridging the gap between AI and HCI. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, Scotland, UK, pp. 1–9, May 2019.

  12. ISO9241: Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems (ISO 9241-210:2019) (2019)

    Google Scholar 

  13. Jendryschik, M.: DIN EN ISO 9241-210 konkretisiert user experience - Gesamtbetrachtung. iX Magazin für Professionelle Informationstechnik 7, 108–111 (2020)

    Google Scholar 

  14. Kang, C., Valenzise, G., Dufaux, F.: EVA: an explainable visual aesthetics dataset. In: Joint Workshop on Aesthetic and Technical Quality Assessment of Multimedia and Media Analytics for Societal Trends, Seattle, WA, USA, pp. 5–13, October 2020.

  15. Kong, S., Shen, X., Lin, Z., Mech, R., Fowlkes, C.: Photo aesthetics ranking network with attributes and content adaptation. arXiv:1606.01621, pp. 662–679, July 2016

  16. Liu, T., Wang, J., Sun, J., Zheng, N., Tang, X., Shum, H.Y.: Picture collage. IEEE Trans. Multimedia 11(7), 1225–1239 (2009).

    CrossRef  Google Scholar 

  17. Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z.: RAPID: rating pictorial aesthetics using deep learning. In: Proceedings of the ACM International Conference on Multimedia - MM 2014, Orlando, Florida, USA, pp. 457–466. ACM Press (2014).

  18. Malu, G., Bapi, R.S., Indurkhya, B.: Learning photography aesthetics with deep CNNs. arXiv preprint arXiv:1707.03981, July 2017

  19. Ren, J., Shen, X., Lin, Z., Mech, R., Foran, D.J.: Personalized image aesthetics. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, pp. 638–647. IEEE, October 2017.

  20. Rother, C., Bordeaux, L., Hamadi, Y., Blake, A.: Autocollage. ACM Trans. Graph. 25(3), 847–852 (2006).

    CrossRef  Google Scholar 

  21. Sandhaus, P., Rabbath, M., Boll, S.: Employing aesthetic principles for automatic photo book layout. In: Lee, K.-T., Tsai, W.-H., Liao, H.-Y.M., Chen, T., Hsieh, J.-W., Tseng, C.-C. (eds.) MMM 2011. LNCS, vol. 6523, pp. 84–95. Springer, Heidelberg (2011).

    CrossRef  Google Scholar 

  22. Settles, B.: Active learning. Synth. Lect. Artif. Intell. Mach. Learn. 6(1), 1–114 (2012)

    MathSciNet  MATH  Google Scholar 

  23. Wang, W., Zhao, M., Wang, L., Huang, J., Cai, C., Xu, X.: A multi-scene deep learning model for image aesthetic evaluation. Signal Process. Image Commun. 47, 511–518 (2016).

    CrossRef  Google Scholar 

  24. Wu, Z., Aizawa, K.: PicWall: photo collage on-the-fly. In: Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, Kaohsiung, Taiwan, pp. 1–10. IEEE, October 2013.

  25. Wu, Z., Aizawa, K.: Very fast generation of content-preserved photo collage under canvas size constraint. Multimedia Tools Appl. 75(4), 1813–1841 (2014).

    CrossRef  Google Scholar 

  26. Xiao, J., Zhang, X., Cheatle, P., Gao, Y., Atkins, C.B.: Mixed-initiative photo collage authoring. In: Proceeding of the 16th ACM International Conference on Multimedia, Vancouver, British Columbia, Canada, pp. 509–518. ACM Press (2008).

  27. Yoo, D., Kweon, I.S.: Learning loss for active learning. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp. 93–102. IEEE, June 2019.

Download references


We thank the reviewers for their valuable suggestions and comments. We appreciate the time and effort they invested in the review.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Ani Withöft .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Withöft, A., Abdenebaoui, L., Boll, S. (2022). ILMICA - Interactive Learning Model of Image Collage Assessment: A Transfer Learning Approach for Aesthetic Principles. In: Þór Jónsson, B., et al. MultiMedia Modeling. MMM 2022. Lecture Notes in Computer Science, vol 13142. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-98354-3

  • Online ISBN: 978-3-030-98355-0

  • eBook Packages: Computer ScienceComputer Science (R0)