The VLDB Journal

, Volume 18, Issue 1, pp 329–343 | Cite as

Speed up interactive image retrieval

  • Heng Tao ShenEmail author
  • Shouxu Jiang
  • Kian-Lee Tan
  • Zi Huang
  • Xiaofang Zhou
Regular Paper


In multimedia retrieval, a query is typically interactively refined towards the “optimal” answers by exploiting user feedback. However, in existing work, in each iteration, the refined query is re-evaluated. This is not only inefficient but fails to exploit the answers that may be common between iterations. Furthermore, it may also take too many iterations to get the “optimal” answers. In this paper, we introduce a new approach called OptRFS (optimizing relevance feedback search by query prediction) for iterative relevance feedback search. OptRFS aims to take users to view the “optimal” results as fast as possible. It optimizes relevance feedback search by both shortening the searching time during each iteration and reducing the number of iterations. OptRFS predicts the potential candidates for the next iteration and maintains this small set for efficient sequential scan. By doing so, repeated candidate accesses (i.e., random accesses) can be saved, hence reducing the searching time for the next iteration. In addition, efficient scan on the overlap before the next search starts also tightens the search space with smaller pruning radius. As a step forward, OptRFS also predicts the “optimal” query, which corresponds to “optimal” answers, based on the early executed iterations’ queries. By doing so, some intermediate iterations can be saved, hence reducing the total number of iterations. By taking the correlations among the early executed iterations into consideration, OptRFS investigates linear regression, exponential smoothing and linear exponential smoothing to predict the next refined query so as to decide the overlap of candidates between two consecutive iterations. Considering the special features of relevance feedback, OptRFS further introduces adaptive linear exponential smoothing to self-adjust the parameters for more accurate prediction. We implemented OptRFS and our experimental study on real life data sets show that it can reduce the total cost of relevance feedback search significantly. Some interesting features of relevance feedback search are also discovered and discussed.


Image retrieval Relevance feedback Query processing Indexing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    Amer-Yahia, S., Case, P., Roelleke, T., Shanmugasundaram, J., Weikum, G.: Report on the db/ir panel at sigmod 2005. SIGMOD Record. 34(4), 71–74 (2005)CrossRefGoogle Scholar
  3. 3.
    Andoni, A., Indyk, P.: Near-optimal hashing algorithms for near neighbor problem in high dimensions. In: FOCS, pp. 459–468 (2006)Google Scholar
  4. 4.
    Baeza-Yates, R., Consens, M.: The continued saga of DB-IR integration. In: VLDB Tutotial (2004)Google Scholar
  5. 5.
    Bang, H., Chen, T.: Feature space warping: an approach to relevance feedback. In: ICIP, pp. I: 968–971 (2002)Google Scholar
  6. 6.
    Bartolini, I., Ciaccia, P., Waas, F.: Feedbackbypass: a new approach to interactive similarity query processing. In: VLDB, pp. 201–210 (2001)Google Scholar
  7. 7.
    Benninga, S.: Financial modeling. In: MIT Press (2000)Google Scholar
  8. 8.
    Böhm, C., Berchtold, S., Keim, D.: Searching in high-dimensional spaces: Index structures for improving the performance of multimedia databases. ACM Comput. Surv. 33(3), 322–373 (2001)CrossRefGoogle Scholar
  9. 9.
    Chaudhuri, S., Ramakrishnan, R., Weikum, G.: Integrating DB and IR technologies: what is the sound of one hand clapping? In: CIDR, pp. 1–12 (2005)Google Scholar
  10. 10.
    Cheng, J., Wang, K.: Multi-view sampling for relevance feedback in image retrieval. In: ICPR, pp. II: 881–884 (2006)Google Scholar
  11. 11.
    Croft, W.B., Schek, H.J.: Introduction to the special issue on database and information retrieval integration. VLDB J. 17(1), 1–3 (2008)Google Scholar
  12. 12.
    El Naqa, I., Yang, Y., Galatsanos, N., Wernick, M.: Relevance feedback based on incremental learning for mammogram retrieval. In: ICIP, pp. I: 729–732 (2003)Google Scholar
  13. 13.
    Ferecatu, M., Boujemaa, N.: Interactive remote sensing image retrieval using active relevance feedback. IEEE Trans. Geosci. Remote Sensing 45(4), 818–826 (2007)CrossRefGoogle Scholar
  14. 14.
    Franco, A., Lumini, A., Maio, D.: A new approach for relevance feedback through positive and negative samples. In: ICPR, pp. IV: 905–908 (2004)Google Scholar
  15. 15.
    Hua, K., Yu, N., Liu, D.: A multiple neighborhood approach to relevance feedback in content-based image retrieval. In: ICDE (2006)Google Scholar
  16. 16.
    Ishikawa, Y., Subramanya, R., Faloutsos, C.: Mindreader: querying databases through multiple examples. In: VLDB, pp. 218–227 (1998)Google Scholar
  17. 17.
    K. A. Hua, N.Y., Liu, D.Z.: Query decomposition: a multiple neighborhood approach to relevance feedback in content-based image retrieval. In: ICDE, p. 84 (2006)Google Scholar
  18. 18.
    Kherfi, M., Ziou, D.: Relevance feedback for cbir: a new approach based on probabilistic feature weighting with positive and negative examples. IP 15(4), 1017–1030 (2006)Google Scholar
  19. 19.
    Kim, D.H., Chung, C.W.: Qcluster: relevance feedback using adaptive clustering for content based image retrieval. In: SIGMOD, pp. 599–610 (2003)Google Scholar
  20. 20.
    Lejsek, H., Asmundsson, F.H., Jonsson, B.T., Amsaleg, L.: Scalability of local image descriptors: a comparative study. In: ACM Multimedia, pp. 589–598 (2006)Google Scholar
  21. 21.
    Lu, H., Ooi, B.C., Shen, H.T., Xue, X.: Hierarchical indexing structure for efficient similarity search in video retrieval. IEEE Trans. Knowl. Data Eng. 18(11), 1544–1559 (2006)CrossRefGoogle Scholar
  22. 22.
    Lu, Y., Hu, C., Zhu, X., Zhang, H., Yang, Q.: A unified framework for semantics and feature based relevance feedback in image retrieval systems. In: ACM Multimedia, pp. 31–37 (2000)Google Scholar
  23. 23.
    Lu, Y., Zhang, H., Liu, W., Hu, C.: Joint semantics and feature based image retrieval using relevance feedback. IEEE Trans. Multimedia 5(3), 339–347 (2003)CrossRefGoogle Scholar
  24. 24.
    Muneesawang, P., Guan, L.: An interactive approach for cbir using a network of radial basis functions. IEEE Trans. Multimedia 6(5), 703–716 (2004)CrossRefGoogle Scholar
  25. 25.
    Oh, S., Chung, M., Sull, S.: Relevance feedback reinforced with semantics accumulation. In: CIVR, pp. 448–454 (2004)Google Scholar
  26. 26.
    Rao, Y., Mundur, P., Yesha, Y.: Fuzzy svm ensembles for relevance feedback in image retrieval. In: CIVR, pp. 350–359 (2006)Google Scholar
  27. 27.
    Rui, Y., Huang, T.: Optimizing learning in image retrieval. In: ICCV, pp. 236–243 (2000)Google Scholar
  28. 28.
    Rui, Y., Huang, T.S., Mehrotra, S.: Content-based image retrieval with relevance feedback in mars. In: ICIP, pp. 815–818 (1997)Google Scholar
  29. 29.
    Rui, Y., Huang, T.S., Ortega, M., Mehrotra, S.: Relevance feedback: a power tool in interactive content-based image retrieval. IEEE Trans. Circuits Systems Video Technol. 8(5), 644–655 (1998)CrossRefGoogle Scholar
  30. 30.
    Saha, S., Das, A., Chanda, B.: Image retrieval based on indexing and relevance feedback. PRL 28(3), 357–366 (2007)Google Scholar
  31. 31.
    Shao, J., Huang, Z., Shen, H.T., Zhou, X., Li, Y.: Dynamic batch nearest neighbour search in video retrieval. In: ICDE, pp. 1395–1399 (2007)Google Scholar
  32. 32.
    Shen, H.T., Ooi, B.C., Zhou, X., Huang, Z.: Towards effective indexing for very large video sequence database. In: SIGMOD, pp. 730–741 (2005)Google Scholar
  33. 33.
    Tong, S., Chang, E.: Support vector machine active learning for image retrieval. In: ACM Multimedia, pp. 107–118 (2001)Google Scholar
  34. 34.
    de Ves, E., Domingo, J., Ayala, G., Zuccarello, P.: A novel bayesian framework for relevance feedback in image content-based retrieval systems. PR 39(9), 1622–1632 (2006)zbMATHGoogle Scholar
  35. 35.
    Wang, L., Gao, Y., Chan, K., Xue, P., Yau, W.: Retrieval with knowledge-driven kernel design: an approach to improving svm-based cbir with relevance feedback. In: ICCV, pp. II: 1355–1362 (2005)Google Scholar
  36. 36.
    Weber, R., Schek, H., Blott, S.: A quantitative analysis and performance study for similarity search methods in high dimensional spaces. In: VLDB, pp. 194–205 (1998)Google Scholar
  37. 37.
    Wu, L., Faloutsos, C., Sycara, K., Payne, T.R.: Falcon: feedback adaptive loop for content-based retrieval. In: VLDB, pp. 297–306 (2000)Google Scholar
  38. 38.
    Wu, P., Manjunath, B.: Adaptive nearest neighbor search for relevance feedback in large image datasets. In: ACM Multimedia, pp. 87–98 (2001)Google Scholar
  39. 39.
    Yin, P., Bhanu, B., Chang, K., Dong, A.: Integrating relevance feedback techniques for image retrieval using reinforcement learning. PAMI 27(10), 1536–1551 (2005)Google Scholar
  40. 40.
    Zhang, H., Chen, Z., Li, M., Su, Z.: Relevance feedback and learning in content-based image search. World Wide Web 6(2), 131–155 (2003)CrossRefGoogle Scholar
  41. 41.
    Zhou, X.S., Huang, T.S.: Relevance feedback in image retrieval: a comprehensive review. Multimedia Systems 8(6), 536–544 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2008

Authors and Affiliations

  • Heng Tao Shen
    • 1
    Email author
  • Shouxu Jiang
    • 2
  • Kian-Lee Tan
    • 3
  • Zi Huang
    • 1
  • Xiaofang Zhou
    • 1
  1. 1.School of Information Technology and Electrical EngineeringThe University of QueenslandBrisbaneAustralia
  2. 2.Department of Computer ScienceHarbin Institute of TechnologyHarbinChina
  3. 3.Department of Computer ScienceNational University of SingaporeSingaporeSingapore

Personalised recommendations