Skip to main content

Personalised Aesthetics with Residual Adapters

Part of the Lecture Notes in Computer Science book series (LNIP,volume 11867)


The use of computational methods to evaluate aesthetics in photography has gained interest in recent years due to the popularization of convolutional neural networks and the availability of new annotated datasets. Most studies in this area have focused on designing models that do not take into account individual preferences for the prediction of the aesthetic value of pictures. We propose a model based on residual learning that is capable of learning subjective, user-specific preferences over aesthetics in photography, while surpassing the state-of-the-art methods and keeping a limited number of user-specific parameters in the model. Our model can also be used for picture enhancement, and it is suitable for content-based or hybrid recommender systems in which the amount of computational resources is limited.

C. Rodríguez-Pardo—Work performed at the University of Edinburgh.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-31332-6_44
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-31332-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.


  1. 1.

    Please visit for our implementation in PyTorch.


  1. Alif, M.A.R., Ahmed, S., Hasan, M.A.: Isolated Bangla handwritten character recognition with convolutional neural network. In: 2017 20th International Conference of Computer and Information Technology (ICCIT), pp. 1–6. IEEE, December 2017.

  2. Shaji, A.: Understanding aesthetics with deep learning (2016).

  3. Bhattacharya, S., Sukthankar, R., Shah, M.: A framework for photo-quality assessment and enhancement based on visual aesthetics. In: Proceedings of the International Conference on Multimedia - MM 2010, p. 271. ACM Press, New York (2010).

  4. Bianco, S., Celona, L., Napoletano, P., Schettini, R.: Predicting image aesthetics with deep learning. In: Blanc-Talon, J., Distante, C., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2016. LNCS, vol. 10016, pp. 117–125. Springer, Cham (2016).

    CrossRef  Google Scholar 

  5. Browniee, J.: A gentle introduction to transfer learning for deep learning (2017).

  6. Chen, Y.L., Huang, T.W., Chang, K.H., Tsai, Y.C., Chen, H.T., Chen, B.Y.: Quantitative analysis of automatic image cropping algorithms: a dataset and comparative study. In: Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV, pp. 226–234 (2017).

  7. Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006).

    CrossRef  Google Scholar 

  8. Datta, R., Li, J., Wang, J.Z.: Algorithmic inferencing of aesthetics and emotion in natural images: an exposition (2008).

  9. Deng, Y., Loy, C.C., Tang, X.: Image aesthetic assessment: an experimental survey. IEEE Signal Process. Mag. 34(4), 80–106 (2017).

    CrossRef  Google Scholar 

  10. Denzler, J., Rodner, E., Simon, M.: Convolutional neural networks as a computational model for the underlying processes of aesthetics perception. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 871–887. Springer, Cham (2016).

    CrossRef  Google Scholar 

  11. Hayn-Leichsenring, G.U., Lehmann, T., Redies, C.: Subjective ratings of beauty and aesthetics: correlations with statistical image properties in Western oil paintings (2017)

    Google Scholar 

  12. Hong, L., Doumith, A.S., Davison, B.D.: Co-factorization machines. In: Proceedings of the Sixth ACM International Conference on Web Search and Data Mining - WSDM 2013, p. 557. ACM Press, New York (2013).

  13. Isinkaye, F., Folajimi, Y., Ojokoh, B.: Recommendation systems: principles, methods and evaluation. Egypt. Inform. J. 16(3), 261–273 (2015).

    CrossRef  Google Scholar 

  14. Jiang, W., Loui, A.C., Cerosaletti, C.D.: Automatic aesthetic value assessment in photographic images. In: 2010 International Conference on Multimedia and Expo, pp. 920–925. IEEE, July 2010.

  15. Jin, X., Zhao, M., Chen, X., Zhao, Q., Zhu, S.-C.: Learning artistic lighting template from portrait photographs. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 101–114. Springer, Heidelberg (2010).

    CrossRef  Google Scholar 

  16. Joshi, D., et al.: Aesthetics and emotions in images. IEEE Signal Process. Mag. 28(5), 94–115 (2011).

    CrossRef  Google Scholar 

  17. Kong, S., Shen, X., Lin, Z., Mech, R., Fowlkes, C.: Photo aesthetics ranking network with attributes and content adaptation (2016).

  18. Leder, H., Belke, B., Oeberst, A., Augustin, D.: A model of aesthetic appreciation and aesthetic judgments. Br. J. Psychol. 95(4), 489–508 (2010).

    CrossRef  Google Scholar 

  19. Liu, Y., Zhang, Y.M., Zhang, X.Y., Liu, C.L.: Adaptive spatial pooling for image classification. Pattern Recogn. 55(C), 58–67 (2016).

    Google Scholar 

  20. Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z.: Rapid: rating pictorial aesthetics using deep learning. In: Proceedings of the ACM International Conference on Multimedia - MM 2014, pp. 457–466 (2014).

  21. Luo, Y., Tang, X.: Photo and video quality evaluation: focusing on the subject. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 386–399. Springer, Heidelberg (2008).

    CrossRef  Google Scholar 

  22. Murray, N., Marchesotti, L., Perronnin, F.: AVA: a large-scale database for aesthetic visual analysis.

  23. Niu, Y., Liu, F.: What makes a professional video? A computational aesthetics approach. IEEE Trans. Circuits Syst. Video Technol. 22(7), 1037–1049 (2012).

    CrossRef  Google Scholar 

  24. O’Donovan, P., Agarwala, A., Hertzmann, A.: Collaborative filtering of color aesthetics. In: Proceedings of Workshop Computational Aesthetics - CAe 2014, pp. 33–40. ACM Press, New York (2014).

  25. Rebuffi, S.A., Bilen, H., Vedaldi, A.: Learning multiple visual domains with residual adapters (2017).

  26. Rebuffi, S.A., Bilen, H., Vedaldi, A.: Efficient parametrization of multi-domain deep neural networks (2018).

  27. Ren, J., Shen, X., Lin, Z., Mech, R., Foran, D.J.: Personalized image aesthetics. In: Proceedings of the IEEE International Conference on Computer Vision, October 2017, pp. 638–647 (2017)

    Google Scholar 

  28. Rothe, R., Timofte, R., Van Gool, L.: Some like it hot-visual guidance for preference prediction. Technical report (2016).

  29. Schafer, J.B., Frankowski, D., Herlocker, J., Sen, S.: Collaborative filtering recommender systems. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web. LNCS, vol. 4321, pp. 291–324. Springer, Heidelberg (2007).

    CrossRef  Google Scholar 

  30. de Stoutz, E., Ignatov, A., Kobyshev, N., Timofte, R., Van Gool, L.: Fast perceptual image enhancement, December 2018.

  31. Tang, X., Luo, W., Wang, X.: Content-based photo quality assessment. IEEE Trans. Multimed. 15(8), 1930–1943 (2013).

    CrossRef  Google Scholar 

  32. Vogel, D., Khan, S.S.: Evaluating visual aesthetics in photographic portraiture. In: Proceedings of Eighth Annual Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging, p. 128 (2012).

  33. Wang, W., Shen, J.: Deep cropping via attention box prediction and aesthetics assessment. In: Proceedings of the IEEE International Conference on Computer Vision, October 2017, pp. 2205–2213 (2017)

    Google Scholar 

  34. Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 1, pp. 419–426. IEEE (2006).

  35. Yu, W., Chen, X.: Aesthetic-based clothing recommendation 2 (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Carlos Rodríguez-Pardo .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Rodríguez-Pardo, C., Bilen, H. (2019). Personalised Aesthetics with Residual Adapters. In: Morales, A., Fierrez, J., Sánchez, J., Ribeiro, B. (eds) Pattern Recognition and Image Analysis. IbPRIA 2019. Lecture Notes in Computer Science(), vol 11867. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-31331-9

  • Online ISBN: 978-3-030-31332-6

  • eBook Packages: Computer ScienceComputer Science (R0)