ImSe: Exploratory Time-Efficient Image Retrieval System

  • Ksenia KonyushkovaEmail author
  • Dorota Głowacka
Part of the Communications in Computer and Information Science book series (CCIS, volume 505)


We consider the problem of Content-Based Image Retrieval (CBIR) with interactive user feedback when the user is unable to query the system with natural language text. We employ content-based techniques with Relevance Feedback mechanism to capture the precise need of the user and interactively refine the query. We apply the Exploration/Exploitation trade-off with Hierarchical Gaussian Process Bandits and pseudo feedback in order to tackle the problem of optimization in face of uncertainty and to improve the quality of multiple images selection. We tackle the scalability issue with Self-Organizing Map as a preprocessing techniques. A prototype system called ImSe was developed and tested in experiments with real users in different types of search tasks. The experiments show favorable results and indicate the benefits of proposed aprroach.


Relevance feedback Exploration/Exploitation Content-based image retrieval Gaussian process bandits Self-organizing maps 


  1. 1.
    Auer, P., Hussain, Z., Kaski, S., Klami, A., Kujala, J., Laaksonen, J., Leung, A.P., Pasupa, K., Shawe-Taylor, J.: Pinview: implicit feedback in content-based image retrieval. JMLR 11, 51–57 (2010)Google Scholar
  2. 2.
    Cox, I., Miller, M., Minka, T., Papathomas, T., Yianilos, P.: The Bayesian image retrieval system, pichunter: theory, implementation, and psychophysical experiments. Image Process. 9(1), 20–37 (2000)CrossRefGoogle Scholar
  3. 3.
    Datta, R., Li, J., Wang, J.: Content-based image retrieval: approaches and trends of the new age. In: Multimedia information retrieval, pp. 253–262. ACM (2005)Google Scholar
  4. 4.
    Hellinger, E.: Neue begründung der theorie quadratischer formen von unendlichvielen veränderlichen. Journal für die reine und angewandte Mathematik 136, 210–271 (1909)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Huiskes, M., Lew, M.: The MIR flickr retrieval evaluation. In: MIR 2008 (2008)Google Scholar
  6. 6.
    Hussain, Z., Leung, A.P., Pasupa, K., Hardoon, D.R., Auer, P., Shawe-Taylor, J.: Exploration-exploitation of eye movement enriched multiple feature spaces for content-based image retrieval. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010, Part I. LNCS, vol. 6321, pp. 554–569. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  7. 7.
    Kato, T., Kurita, T., Otsu, N., Hirata, K.: A sketch retrieval method for full color image database-query by visual example. In: Pattern Recognition. Computer Vision and Applications, IAPR, pp. 530–533. IEEE (1992)Google Scholar
  8. 8.
    Kohonen, T.: Self-organizing Maps, vol. 30. Springer Verlag, Heidelberg (2001) CrossRefGoogle Scholar
  9. 9.
    Konyushkova, K., Glowacka, D.: Content-based image retrieval with hierarchical gaussian process bandits with self-organizing maps. In: ESANN (2013)Google Scholar
  10. 10.
    Kosch, H., Maier, P.: Content-based image retrieval systems-reviewing and benchmarking. JDIM 8(1), 54–64 (2010)Google Scholar
  11. 11.
    Laaksonen, J., Koskela, M., Laakso, S., Oja, E.: Picsom-content-based image retrieval with self-organizing maps. Pattern Recognition Letters 21(13), 1199–1207 (2000)CrossRefGoogle Scholar
  12. 12.
    Manjunath, B., Ohm, J., Vasudevan, V., Yamada, A.: Color and texture descriptors. Circuits and Systems for Video Technology 11(6), 703–715 (2001)CrossRefGoogle Scholar
  13. 13.
    Hussain, Z., Auer, P., Leung, A., Shawe-Taylor, J.: Report on using side information for exploration-exploitation trade-offs, fp7-216529 pinview. Technical report, European Community’s Seventh Framework Programme (2009)Google Scholar
  14. 14.
    Pandey, S., Agarwal, D., Chakrabarti, D., Josifovski, V.: Bandits for taxonomies: a model-based approach. In: SIAM International Conference on Data Mining (SDM) (2007)Google Scholar
  15. 15.
    Smeulders, A., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)CrossRefGoogle Scholar
  16. 16.
    Srinivas, N., Krause, A., Kakade, S., Seeger, M.: Gaussian process bandits without regret: An experimental design approach. In: CoRR (2009).
  17. 17.
    Eickhoff, J.: Onboard Computers, Onboard Software and Satellite Operations. SAT, vol. 1. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  18. 18.
    Veltkamp, R.C., Tanase, M.: Content-Based Image Retrieval Systems: A Survey, pp. 1–62. Department of Computing Science, Utrecht University (2002). (preprint)Google Scholar
  19. 19.
    Zhou, X., Huang, T.: Relevance feedback in image retrieval: a comprehensive review. Multimedia Syst. 8(6), 536–544 (2003)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (, which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  1. 1.Helsinki Institute for Information TechnologyEspooFinland
  2. 2.Department of Computer ScienceUniversity of HelsinkiHelsinkiFinland

Personalised recommendations