Boosting Image Retrieval

Abstract

We present an approach for image retrieval using a very large number of highly selective features and efficient learning of queries. Our approach is predicated on the assumption that each image is generated by a sparse set of visual “causes” and that images which are visually similar share causes. We propose a mechanism for computing a very large number of highly selective features which capture some aspects of this causal structure (in our implementation there are over 46,000 highly selective features). At query time a user selects a few example images, and the AdaBoost algorithm is used to learn a classification function which depends on a small number of the most appropriate features. This yields a highly efficient classification function. In addition we show that the AdaBoost framework provides a natural mechanism for the incorporation of relevance feedback. Finally we show results on a wide variety of image queries.

This is a preview of subscription content, access via your institution.

References

  1. Abdel-Mottaleb, M., Dimitrova, N., Desai, R., and Martino, J. 1996. CONIVAS: Content-based image and video access system. In ACM Multimedia, pp. 427–428.

  2. Amit, Y. and Geman, D. 1997. Shape quantization and recognition with randomized trees. Neural Computation, 9(7):1545–1588.

    Google Scholar 

  3. Aslandogan, Y.A., Thier, C., and Yu, C. 1996. A system for effective content based image retrieval. In ACM Multimedia, pp. 429–430.

  4. Bach, J.R., Fuller, C., Gupta, A., Hampapur, A., Horowitz, B., Jain, R., and Shu, C.F. 1996. The Virage image search engine: An open framework for image management. In International Society for Optical Engineering, vol. 4, pp. 76–87.

    Google Scholar 

  5. Belongie, S., Carson, C., Greenspan, H., and Malik, J. 1998. Colorand texture-based image segmentation using EM and its application to content-based image retrieval. In International Conference on Computer Vision, pp. 675–682.

  6. Burt, P.J. and Adelson, E. 1983. The laplacian pyramid as a compact image code. IEEE Transactions on Communications, 31(4):532–540.

    Google Scholar 

  7. Cohn, D., Atlas, L., and Ladner, R. 1994. Improving generalization with active learning. Machine Learning, 15(2):201–221.

    Google Scholar 

  8. Cortes, C. and Vapnik, V. 1995. Support vector networks. Machine Learning, 20(3):273–297.

    Google Scholar 

  9. Cox, I., Miller, M., Minka, T., and Yianilos, P.N. 1998. An optimized interaction strategy for bayesian relevance feedback. In Computer Vision and Pattern Recognition, pp. 553–558.

  10. DeBonet, J.S. and Viola, P. 1998. Structure driven image database retrieval. In Neural Information Processing Systems, vol. 10, pp. 866–872.

    Google Scholar 

  11. Diaconis, P. and Freedman, D. 1984. Asymptotics of graphical projection pursuit. Annals of Statistics, 12:793–815.

    Google Scholar 

  12. Flickner, M., Sawhney, H., Niblack W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., and Yanker, P. 1995. Query by image and video content: The QBIC system. IEEE Computer, 28(9):23–32.

    Google Scholar 

  13. Freund, Y. and Schapire, R.E. 1997. A decision-theoretic generalization of online learning and an application to boosting. Journal of Computer and Systems Science, 55(1):119–139.

    Google Scholar 

  14. Guo, G., Jain, A.K., Ma W., and Zhang, H. 2002. Learning similarity measure for natural image retrieval with relevance feedback. IEEE Transactions on Neural Networks, 13(4):811–820.

    Google Scholar 

  15. Huang, J., Kumar, S.R., Mitra, M., Zhu, W.J., and Zabih, R. 1997. Image indexing using color correlograms. In Computer Vision and Pattern Recognition, pp. 762–768.

  16. Jacobs, C.E., Finkelstein, A., and Salesin, D.H. 1995. Fast multiresolution image querying. In ACM SIGGRAPH, pp. 277–286.

  17. Kelly, M. and Cannon, T.M. 1995. Query by image example: The CANDID approach. In International Society for Optical Engineering, vol. 2420, pp. 238–248.

    Google Scholar 

  18. La Cascia, M. and Ardizzone, E. 1996. JACOB: Just a content-based query system for video databases. In International Conference on Acoustics Speech and Signal Processing, vol. 2, pp. 1216–1219.

    Google Scholar 

  19. LeCun, Y., Boser, B., Denker, J.S., Henderson, B., Howard, R.E., Hubbard, W., and Jackel, L.D. 1989. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4):541–551.

    Google Scholar 

  20. Lipson, P., Grimson, E., and Sinha, P. 1997. Configuration based scene classification and image indexing. In Computer Vision and Pattern Recognition, pp. 1007–1013.

  21. Nastar, C., Mitschke, M., and Meilhac, C. 1998. Efficient query refinement for image retrieval. In Computer Vision and Pattern Recognition, pp. 547–552.

  22. Ogle, V.E. and Stonebraker, M. 1995. Chabot: Retrieval from a relational database of images. IEEE Computer, 28(9):40–48.

    Google Scholar 

  23. Papageorgiou, C., Oren, M., and Poggio, T. 1998. A general framework for object detection. In International Conference on Computer Vision, pp. 555–562.

  24. Pentland, A., Picard, R.W., and Sclaroff, S. 1996. Photobook: Content-based manipulation of image databases. International Journal of Computer Vision, 18(3):233–254.

    Google Scholar 

  25. Ratan, A.L. and Maron, O. 1998. Multiple instance learning for natural scene classification. In International Conference on Machine Learning, pp. 341–349.

  26. Rowley, H.A., Baluja, S., and Kanade, T. 1998. Neural networkbased face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1):23–38.

    Google Scholar 

  27. Rui, Y. and Huang, T. 2000. Optimizing learning in image retrieval. In Computer Vision and Pattern Recognition, vol. 1, pp. 236–243.

    Google Scholar 

  28. Rui, Y., Huang, T., and Chang, S. 1999. Image retrieval: Current techniques, promising directions and open issues. Journal of Visual Communication and Image Representation, 10:39–62.

    Google Scholar 

  29. Rui, Y., Huang, T., and Mehrotra, S. 1997. Content-based image retrieval with relevance feedback in mars. In IEEE International Conference on Image Processing, vol. 2, pp. 815–815.

    Google Scholar 

  30. Schapire, R.E., Freund, Y., Bartlett, P., and Lee, W.S. 1998. Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5):1651–1686.

    Google Scholar 

  31. Simoncelli, E.P. and Adelson, E.H. 1996. Noise removal via bayesian wavelet coring. In IEEE International Conference on Image Processing, pp. 379–382.

  32. Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12):1349–1380.

    Google Scholar 

  33. Smith, J.R. and Chang, S.-F. 1996. VisualSEEk: A fully automated content-based image query system. In ACM Multimedia, pp. 87–98.

  34. Swain, M.J. and Ballard, D.H. 1991. Color indexing. International Journal of Computer Vision, 7(1).

  35. Vasconcelos, N. and Lippman, A. 2000. A probabilistic architecture for content-based image retrieval. In Computer Vision and Pattern Recognition, vol. 1, pp. 216–221.

    Google Scholar 

  36. Viola, P. and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition, vol. 1, pp. 511–518.

    Google Scholar 

  37. Wu, Y., Tian, Q., and Huang, T.S. 2000. Discriminant-EM algorithm with application to image retrieval. In ComputerVision andPattern Recognition, vol. 1, pp. 222–227.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Tieu, K., Viola, P. Boosting Image Retrieval. International Journal of Computer Vision 56, 17–36 (2004). https://doi.org/10.1023/B:VISI.0000004830.93820.78

Download citation

  • image database
  • sparse representation
  • feature selection
  • relevance feedback