In this paper we describe how to include high level semantic information, such as aesthetics and emotions, into Content Based Image Retrieval. We present a color-based emotion-related image descriptor that can be used for describing the emotional content of images. The color emotion metric used is derived from psychophysical experiments and based on three variables: activity, weight and heat. It was originally designed for single-colors, but recent research has shown that the same emotion estimates can be applied in the retrieval of multi-colored images. Here we describe a new approach, based on the assumption that perceived color emotions in images are mainly affected by homogenous regions, defined by the emotion metric, and transitions between regions. RGB coordinates are converted to emotion coordinates, and for each emotion channel, statistical measurements of gradient magnitudes within a stack of low-pass filtered images are used for finding interest points corresponding to homogeneous regions and transitions between regions. Emotion characteristics are derived for patches surrounding each interest point, and saved in a bag-of-emotions, that, for instance, can be used for retrieving images based on emotional content.


Emotion Region Image Retrieval Interest Point Emotional Content Content Base Image Retrieval 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Datta, R., Joshi, D., Li, J., Wang, J.: Image retrieval: Ideas, influences, and trends of the new age. ACM Comput. Surv. 40, 2 (2008)CrossRefGoogle Scholar
  2. 2.
    Fairchild, M.: Color Appearance Models. Wiley-IS&T, Chichester (2005)Google Scholar
  3. 3.
    Solli, M., Lenz, R.: Color emotions for image classification and retrieval. In: Proceedings CGIV 2008, pp. 367–371 (2008)Google Scholar
  4. 4.
    Solli, M.: Topics in content based image retrieval - fonts and color emotions. Licentiate thesis No. 1397, Linköping University (2009),
  5. 5.
    Csurka, G., Dance, C.R., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Proceedings ECCV 2004, pp. 1–22 (2004)Google Scholar
  6. 6.
    Ou, L.C., Luo, M., Woodcock, A., Wright, A.: A study of colour emotion and colour preference. Part I-III. Color Res. Appl. 29, 3, 4, 5 (2004)Google Scholar
  7. 7.
    Kobayashi, S.: Aim and method of the color image scale. Col. Res. Appl. 6, 93–107 (1981)CrossRefGoogle Scholar
  8. 8.
    Sato, T., Kajiwara, K., Hoshino, H., Nakamura, T.: Quantitative evaluation and categorising of human emotion induced by colour. Col. Sci. & Tech. 3, 53–59 (2000)Google Scholar
  9. 9.
    Wang, W.N., Yu, Y.L.: Image emotional semantic query based on color semantic description. In: Proceedings of ICMLC 2005, pp. 4571–4576 (2005)Google Scholar
  10. 10.
    Corridoni, J., Del Bimbo, A., Pala, P.: Image retrieval by color semantics. Multimedia Syst. 7, 175–183 (1999)CrossRefGoogle Scholar
  11. 11.
    Wang, W.N., Yu, Y.L., Jiang, S.M.: Image retrieval by emotional semantics: A study of emotional space and feature extraction. 2006 IEEE Int. Conf. on Systems, Man and Cybernetics 4, 3534–3539 (2007)Google Scholar
  12. 12.
    Cho, S.B., Lee, J.Y.: A human-oriented image retrieval system using interactive genetic algorithm. IEEE Trans. Syst. Man Cybern. 32, 452–458 (2002)CrossRefGoogle Scholar
  13. 13.
    Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., Xu, Y.-Q.: Color harmonization. In: ACM SIGGRAPH 2006, vol. 25, pp. 624–630 (2006)Google Scholar
  14. 14.
    Datta, R., Joshi, D., Li, J., Wang, J.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  15. 15.
    Datta, R., Li, J., Wang, J.: Learning the consensus on visual quality for next-generation image management. In: 15th ACM Int. Conf. on Multimedia, pp. 533–536 (2007)Google Scholar
  16. 16.
    Datta, R., Li, J., Wang, J.Z.: Algorithmic inferencing of aesthetics and emotion in natural images: An exposition. In: Proceedings of ICIP 2008, pp. 105–108 (2008)Google Scholar
  17. 17.
    Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A comparison of affine region detectors. IJCV 65, 43–72 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Martin Solli
    • 1
  • Reiner Lenz
    • 1
  1. 1.ITN, Linköping UniversityNorrköpingSweden

Personalised recommendations