A Reinforcement Learning Approach to Query-Less Image Retrieval

  • Sayantan Hore
  • Lasse Tyrvainen
  • Joel Pyykko
  • Dorota GlowackaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8820)


Search algorithms in image retrieval tend to focus exclusively on giving the user more and more similar images based on queries that the user has to explicitly formulate. Implicitly, such systems limit the users exploration of the image space and thus remove the potential for serendipity. Thus, in recent years there has been an increased interest in developing exploration–exploitation algorithms for image search. We present an interactive image retrieval system that combines Reinforcement Learning together with a user interface designed to allow users to actively engage in directing the search. Reinforcement Learning is used to model the user interests by allowing the system to trade off between exploration (unseen types of image) and exploitation (images the system thinks are relevant).


Image Retrieval Image Space Relevance Feedback User Feedback Image Search 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The project was supported by The Finnish Funding Agency for Innovation (under projects Re:Know and D2I) and by the Academy of Finland (under the Finnish Centre of Excellence in Computational Inference).


  1. 1.
    Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47, 235–256 (2002)CrossRefzbMATHGoogle Scholar
  2. 2.
    Auer, P., Hussain, Z., Kaski, S., Klami, A., Kujala, J., Laaksonen, J., Leung, A.P., Pasupa, K., Shawe-Taylor, J.: Pinview: implicit feedback in content-based image retrieval. JMLR Workshop Conf. Proc. 11, 51–57 (2010)Google Scholar
  3. 3.
    Cox, I., Miller, M., Minka, T., Papathomas, T., Yianilos, P.: The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments. Image Process. 9(1), 20–37 (2000)CrossRefGoogle Scholar
  4. 4.
    Datta, R., Li, J., Wang, J.: Content-based image retrieval: approaches and trends of the new age. In: Multimedia Information Retrieval, pp. 253–262. ACM (2005)Google Scholar
  5. 5.
    Głowacka, D., Hore, S.: Balancing exploration-exploitation in image retrieval. In: Proceedings of UMAP 2014 Posters, Demonstrations and Late-Breaking Results (2014)Google Scholar
  6. 6.
    Głowacka, D., Shawe-Taylor, J.: Content-based image retrieval with multinomial relevance feedback. In: Proceedings of ACML, pp. 111–125 (2010)Google Scholar
  7. 7.
    Guiver, J., Snelson, E.: Learning to rank with softrank and Gaussian processes. In: Proceedings of SIGIR, pp. 259–266 (2008)Google Scholar
  8. 8.
    Kato, T., Kurita, T., Otsu, N., Hirata, K.: A sketch retrieval method for full color image database-query by visual example. In: Pattern Recognition. Computer Vision and Applications, pp. 530–533 (1992)Google Scholar
  9. 9.
    Kelly, D., Fu, X.: Elicitation of term relevance feedback: an investigation of term source and context. In: Proceedings of SIGIR (2006)Google Scholar
  10. 10.
    Kosch, H., Maier, P.: Content-based image retrieval systems-reviewing and benchmarking. JDIM 8(1), 54–64 (2010)Google Scholar
  11. 11.
    Laaksonen, J., Koskela, M., Laakso, S., Oja, E.: Picsom-content-based image retrieval with self-organizing maps. Pattern Recogn. Lett. 21(13), 1199–1207 (2000)CrossRefzbMATHGoogle Scholar
  12. 12.
    Pham, T.-T., Maillot, N.E., Lim, J.-H., Chevallet, J.-P.: Latent semantic fusion model for image retrieval and annotation. In: Proceedings of CIKM (2007)Google Scholar
  13. 13.
    Piras, L., Giacinto, G., Paredes, R.: Enhancing image retrieval by an exploration-exploitation approach. In: Perner, P. (ed.) MLDM 2012. LNCS (LNAI), vol. 7376, pp. 355–365. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  14. 14.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  15. 15.
    Smeulders, A., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. Pattern Anal. Mach. Intell. 22(12), 1349–1380 (2000)CrossRefGoogle Scholar
  16. 16.
    Suditu, N., Fleuret, F.: Iterative relevance feedback with adaptive exploration/exploitation trade-off. In: Proceedings of CIKM (2012)Google Scholar
  17. 17.
    Veltkamp, R.C., Tanase, M.: Content-based image retrieval systems: a survey. Department of Computing Science, Utrecht University (2002)Google Scholar
  18. 18.
    Villegas, M., Leiva, L.A., Paredes, R.: Interactive image retrieval based on relevance feedback. In: Sappa, A.D., Vitrià, J., Multimodal Interaction in Image and Video Application, vol. 48, pp. 83–109. Springer, Heidelberg (2013)Google Scholar
  19. 19.
    Yee, K.-P., Swearingen, K., Li, K., Hearst, M.: Faceted metadata for image search and browsing. In: Proceedings of CHI, pp. 401–408 (2003)Google Scholar
  20. 20.
    Zhou, X., Huang, T.: Relevance feedback in image retrieval: a comprehensive review. Multimed. Syst. 8(6), 536–544 (2003)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Sayantan Hore
    • 1
  • Lasse Tyrvainen
    • 1
  • Joel Pyykko
    • 1
  • Dorota Glowacka
    • 1
    Email author
  1. 1.Helsinki Institute for Information Technology HIIT, Department of Computer ScienceUniversity of HelsinkiHelsinkiFinland

Personalised recommendations