Mining High-Level User Concepts with Multiple Instance Learning and Relevance Feedback for Content-Based Image Retrieval

  • Xin Huang
  • Shu-Ching Chen
  • Mei-Ling Shyu
  • Chengcui Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2797)


Understanding and learning the subjective aspect of humans in Content-Based Image Retrieval has been an active research field during the past few years. However, how to effectively discover users’ concept patterns when there are multiple visual features existing in the retrieval system still remains a big issue. In this book chapter, we propose a multimedia data mining framework that incorporates Multiple Instance Learning into the user relevance feedback in a seamless way to discover the concept patterns of users, especially where the user’s most interested region and how to map the local feature vector of that region to the high-level concept pattern of users. This underlying mapping can be progressively discovered through the feedback and learning procedure. The role the user plays in the retrieval system is to guide the system mining process to his/her own focus of attention. The retrieval performance is tested to show the feasibility and effectiveness of the proposed multimedia data mining framework.


Image Retrieval Relevance Feedback CBIR System Multiple Instance Learn Natural Scene Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aksoy, S., Haralick, R.M.: A Weighted Distance Approach to Relevance Feedback. In: Proceedings of the International Conference on Pattern Recognition, pp. 812–815 (2000)Google Scholar
  2. 2.
    Auer, P.: On Learning From Multi-instance Examples: Empirical Evaluation of a Theoretical Approach. In: Proceedings of 14th International Conference on Machine Learning, San Francisco, CA, pp. 21–29 (1997)Google Scholar
  3. 3.
    Buckley, C., Singhal, A., Miltra, M.: New Retrieval Approaches Using SMART: TREC4. In: Text Retrieval Conference, Sponsored by National Institute of Standard and Technology and Advanced Research Projects Agency, pp. 25–48 (1995)Google Scholar
  4. 4.
    Carson, C., Belongie, S., Greenspan, H., Malik, J.: Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(8), 1026–1038 (2002)CrossRefGoogle Scholar
  5. 5.
    Chang, C.-H., Hsu, C.-C.: Enabling Concept-Based Relevance Feedback for Information Retrieval on the WWW. IEEE Transactions on Knowledge and Data Engineering 11, 595–609 (1999)CrossRefGoogle Scholar
  6. 6.
    Cox, I.J., Minka, T.P., Papathomas, T.V., Yianilos, P.N.: The Bayesian Image Retrieval System, PicHunter: Theory, Implementation, and Psychophysical Experiments. IEEE Transactions on Image Processing–special issue on digital libraries 9(1), 20–37 (2000)Google Scholar
  7. 7.
    Dietterich, T.G., Lathrop, R.H., Lozano-Perez, T.: Solving the Multiple- Instance Problem with Axis-Parallel Rectangles. Artificial Intelligence Journal 89, 31–71 (1997)zbMATHCrossRefGoogle Scholar
  8. 8.
    Ishikawa, Y., Subramanya R., Faloutsos, C.: Mindreader: Query Databases Through Multiple Examples. In: Proceedings of the 24th International Conference on Very Large Databases, (1998) Google Scholar
  9. 9.
    Marks II, R.J., Oh, S., Arabshahi, P., Caudell, T.P., Choi, J.J., Song, B.G.: Steepest Descent Adaptation of Min-Max Fuzzy If-Then Rules. In: Proceedings of the IEEE/INNS International Conference on Neural Networks, Beijing, China, vol. 3, pp. 471–477 (1992)Google Scholar
  10. 10.
    Maron, O., Lozano-Perez, T.: Multiple-Instance A Framework for Multiple- Instance Learning. In: Advances in Neural Information Processing System 10, MIT Press. Cambridge (1998)Google Scholar
  11. 11.
    Nagasska, A., Tanaka, Y.: Automatic Video Indexing and Full Video Search for Object Appearance. IFIP Trans. Visual Database Systems II, 113–127 (1992)Google Scholar
  12. 12.
    Ramon, J., De Raedt, L.: Multi-Instance Neural Networks. In: Proceedings of the ICML 2000 Workshop on Attribute-value and Relational Learning (2000) Google Scholar
  13. 13.
    Ray, S., Page, D.: Multiple-Instance Regression. In: Proceedings of the 18th International Conference on Machine Learning, San Francisco, CA, pp. 425–432 (2001)Google Scholar
  14. 14.
    Rocchio, J.J.: Relevance Feedback in Information Retrieval. In: The Smart System experiments in automatic document processing, pp. 313–323. Prentice Hall Inc., Englewood Cliffs (1971)Google Scholar
  15. 15.
    Rui, Y., Huang, T.S., Mehrotra, S.: Content-based image retrieval with relevance feedback in MARS. In: Proceedings of the, International Conference on Image Processing, pp. 815–818 (1997)Google Scholar
  16. 16.
    Rui, Y., Huang, T.S., Ortega, M., Mehrotra, S.: Relevance Feedback: A Power Tool in Interactive Content-Based Image Retrieval. IEEE Transaction on Circuits and Systems for Video Technology, Special Issue on Segmentation, Description, and Retrieval of Video Content 18(5), 644–655 (1998)Google Scholar
  17. 17.
    Rui, Y., Huang, T.S.: Optimizing Learnin In Image Retrieval. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp. 236–243 (2000)Google Scholar
  18. 18.
    Salton, G., McGill, M.J.: Introduction to Modern Information Retrieval. McGraw-Hill Book Company, New York (1983)zbMATHGoogle Scholar
  19. 19.
    Wang, J., Zucker, J.-D.: Solving the Multiple-Instance Learning Problem: A Lazy Learning Approach. In: Proceedings of the 17th International Conference on Machine Learning, pp. 1119–1125 (2000)Google Scholar
  20. 20.
    Yang, C., Lozano-Prez, T.: Image Database Retrieval with Multiple-Instance Learning Techniques. In: Proceedings of the 16th International Conference on Data Engineering, pp. 233–243 (2000)Google Scholar
  21. 21.
    Zhang, Q., Goldman, S.A.: EM-DD: An Improved Multiple-Instance Learning Technique. In: Advances in Neural Information Processing Systems, NIPS 2002 (2002) (to be published)Google Scholar
  22. 22.
    Zhang, Q., Goldman, S.A., Yu, W., Fritts, J.: Content-Based Image Retrieval Using Multiple-Instance Learning. In: Proceedings of the 9th International Conference on Machine Learning (2002)Google Scholar
  23. 23.
    Zucker, J.-D., Chevaleyre, Y.: Solving Multiple-instance and Multiple-part Learning Problems with Decision Trees and Decision Rules. Application to the Mutagenesis Problem. In: Proceedings of the 14th Biennial Conference of the Canadian Society for Computational Studies of Intelligence, AI 2001, pp. 204–214 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Xin Huang
    • 1
  • Shu-Ching Chen
    • 1
  • Mei-Ling Shyu
    • 2
  • Chengcui Zhang
    • 1
  1. 1.Distributed Multimedia Information System Laboratory, School of Computer ScienceFlorida International UniversityMiamiUSA
  2. 2.Department of Electrical and Computer EngineeringUniversity of MiamiCoral GablesUSA

Personalised recommendations