International Journal of Computer Vision

, Volume 56, Issue 1–2, pp 17–36 | Cite as

Boosting Image Retrieval

  • Kinh Tieu
  • Paul Viola

Abstract

We present an approach for image retrieval using a very large number of highly selective features and efficient learning of queries. Our approach is predicated on the assumption that each image is generated by a sparse set of visual “causes” and that images which are visually similar share causes. We propose a mechanism for computing a very large number of highly selective features which capture some aspects of this causal structure (in our implementation there are over 46,000 highly selective features). At query time a user selects a few example images, and the AdaBoost algorithm is used to learn a classification function which depends on a small number of the most appropriate features. This yields a highly efficient classification function. In addition we show that the AdaBoost framework provides a natural mechanism for the incorporation of relevance feedback. Finally we show results on a wide variety of image queries.

image database sparse representation feature selection relevance feedback 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abdel-Mottaleb, M., Dimitrova, N., Desai, R., and Martino, J. 1996. CONIVAS: Content-based image and video access system. In ACM Multimedia, pp. 427–428.Google Scholar
  2. Amit, Y. and Geman, D. 1997. Shape quantization and recognition with randomized trees. Neural Computation, 9(7):1545–1588.Google Scholar
  3. Aslandogan, Y.A., Thier, C., and Yu, C. 1996. A system for effective content based image retrieval. In ACM Multimedia, pp. 429–430.Google Scholar
  4. Bach, J.R., Fuller, C., Gupta, A., Hampapur, A., Horowitz, B., Jain, R., and Shu, C.F. 1996. The Virage image search engine: An open framework for image management. In International Society for Optical Engineering, vol. 4, pp. 76–87.Google Scholar
  5. Belongie, S., Carson, C., Greenspan, H., and Malik, J. 1998. Colorand texture-based image segmentation using EM and its application to content-based image retrieval. In International Conference on Computer Vision, pp. 675–682.Google Scholar
  6. Burt, P.J. and Adelson, E. 1983. The laplacian pyramid as a compact image code. IEEE Transactions on Communications, 31(4):532–540.Google Scholar
  7. Cohn, D., Atlas, L., and Ladner, R. 1994. Improving generalization with active learning. Machine Learning, 15(2):201–221.Google Scholar
  8. Cortes, C. and Vapnik, V. 1995. Support vector networks. Machine Learning, 20(3):273–297.Google Scholar
  9. Cox, I., Miller, M., Minka, T., and Yianilos, P.N. 1998. An optimized interaction strategy for bayesian relevance feedback. In Computer Vision and Pattern Recognition, pp. 553–558.Google Scholar
  10. DeBonet, J.S. and Viola, P. 1998. Structure driven image database retrieval. In Neural Information Processing Systems, vol. 10, pp. 866–872.Google Scholar
  11. Diaconis, P. and Freedman, D. 1984. Asymptotics of graphical projection pursuit. Annals of Statistics, 12:793–815.Google Scholar
  12. Flickner, M., Sawhney, H., Niblack W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., and Yanker, P. 1995. Query by image and video content: The QBIC system. IEEE Computer, 28(9):23–32.Google Scholar
  13. Freund, Y. and Schapire, R.E. 1997. A decision-theoretic generalization of online learning and an application to boosting. Journal of Computer and Systems Science, 55(1):119–139.Google Scholar
  14. Guo, G., Jain, A.K., Ma W., and Zhang, H. 2002. Learning similarity measure for natural image retrieval with relevance feedback. IEEE Transactions on Neural Networks, 13(4):811–820.Google Scholar
  15. Huang, J., Kumar, S.R., Mitra, M., Zhu, W.J., and Zabih, R. 1997. Image indexing using color correlograms. In Computer Vision and Pattern Recognition, pp. 762–768.Google Scholar
  16. Jacobs, C.E., Finkelstein, A., and Salesin, D.H. 1995. Fast multiresolution image querying. In ACM SIGGRAPH, pp. 277–286.Google Scholar
  17. Kelly, M. and Cannon, T.M. 1995. Query by image example: The CANDID approach. In International Society for Optical Engineering, vol. 2420, pp. 238–248.Google Scholar
  18. La Cascia, M. and Ardizzone, E. 1996. JACOB: Just a content-based query system for video databases. In International Conference on Acoustics Speech and Signal Processing, vol. 2, pp. 1216–1219.Google Scholar
  19. LeCun, Y., Boser, B., Denker, J.S., Henderson, B., Howard, R.E., Hubbard, W., and Jackel, L.D. 1989. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4):541–551.Google Scholar
  20. Lipson, P., Grimson, E., and Sinha, P. 1997. Configuration based scene classification and image indexing. In Computer Vision and Pattern Recognition, pp. 1007–1013.Google Scholar
  21. Nastar, C., Mitschke, M., and Meilhac, C. 1998. Efficient query refinement for image retrieval. In Computer Vision and Pattern Recognition, pp. 547–552.Google Scholar
  22. Ogle, V.E. and Stonebraker, M. 1995. Chabot: Retrieval from a relational database of images. IEEE Computer, 28(9):40–48.Google Scholar
  23. Papageorgiou, C., Oren, M., and Poggio, T. 1998. A general framework for object detection. In International Conference on Computer Vision, pp. 555–562.Google Scholar
  24. Pentland, A., Picard, R.W., and Sclaroff, S. 1996. Photobook: Content-based manipulation of image databases. International Journal of Computer Vision, 18(3):233–254.Google Scholar
  25. Ratan, A.L. and Maron, O. 1998. Multiple instance learning for natural scene classification. In International Conference on Machine Learning, pp. 341–349.Google Scholar
  26. Rowley, H.A., Baluja, S., and Kanade, T. 1998. Neural networkbased face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1):23–38.Google Scholar
  27. Rui, Y. and Huang, T. 2000. Optimizing learning in image retrieval. In Computer Vision and Pattern Recognition, vol. 1, pp. 236–243.Google Scholar
  28. Rui, Y., Huang, T., and Chang, S. 1999. Image retrieval: Current techniques, promising directions and open issues. Journal of Visual Communication and Image Representation, 10:39–62.Google Scholar
  29. Rui, Y., Huang, T., and Mehrotra, S. 1997. Content-based image retrieval with relevance feedback in mars. In IEEE International Conference on Image Processing, vol. 2, pp. 815–815.Google Scholar
  30. Schapire, R.E., Freund, Y., Bartlett, P., and Lee, W.S. 1998. Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5):1651–1686.Google Scholar
  31. Simoncelli, E.P. and Adelson, E.H. 1996. Noise removal via bayesian wavelet coring. In IEEE International Conference on Image Processing, pp. 379–382.Google Scholar
  32. Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12):1349–1380.Google Scholar
  33. Smith, J.R. and Chang, S.-F. 1996. VisualSEEk: A fully automated content-based image query system. In ACM Multimedia, pp. 87–98.Google Scholar
  34. Swain, M.J. and Ballard, D.H. 1991. Color indexing. International Journal of Computer Vision, 7(1).Google Scholar
  35. Vasconcelos, N. and Lippman, A. 2000. A probabilistic architecture for content-based image retrieval. In Computer Vision and Pattern Recognition, vol. 1, pp. 216–221.Google Scholar
  36. Viola, P. and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition, vol. 1, pp. 511–518.Google Scholar
  37. Wu, Y., Tian, Q., and Huang, T.S. 2000. Discriminant-EM algorithm with application to image retrieval. In ComputerVision andPattern Recognition, vol. 1, pp. 222–227.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Kinh Tieu
    • 1
  • Paul Viola
    • 2
  1. 1.Artificial Intelligence LaboratoryMassachusetts Institute of TechnologyCambridgeUSA
  2. 2.Mitsubishi Electric Research LabsCambridgeUSA

Personalised recommendations