Abstract.
Relevance feedback is a mechanism to interactively learn a user’s query concept online. It has been extensively used to improve the performance of multimedia information retrieval. In this paper, we present a novel interactive pattern analysis method that reduces relevance feedback to a two-class classification problem and classifies multimedia objects as relevant or irrelevant. To perform interactive pattern analysis, we propose two online pattern classification methods, called interactive random forests (IRF) and adaptive random forests (ARF), that adapt a composite classifier known as random forests for relevance feedback. IRF improves the efficiency of regular random forests (RRF) with a novel two-level resampling technique called biased random sample reduction, while ARF boosts the performance of RRF with two adaptive learning techniques called dynamic feature extraction and adaptive sample selection. During interactive multimedia retrieval, both ARF and IRF run two to three times faster than RRF while achieving comparable precision and recall against the latter. Extensive experiments on a COREL image set (with 31,438 images) demonstrate that our methods (i.e., IRF and RRF) achieve at least a \(20\%\) improvement on average precision and recall over the state-of-the-art approaches.
Similar content being viewed by others
References
Baeza-Yates R, Ribiero-Neto B (1999) Modern information retrieval. Addison-Wesley, Reading, MA
Brandt S, Laaksonen J, Oja E (2000) Statistical shape features in content-based image retrieval. In: Proceedings of the international conference on pattern recognition, September 2000
Breiman L (1997) Bagging predictors. Mach Learn 24:123-140
Breiman L (1999) Random forests-random features. Technical Report 567, Department of Statistics, University of California, Berkeley
Breiman L, Friedman J, Olshen R, Stone C (1984) Classification and regression trees. Wadsworth, Belmont, CA
Charikar M, Chekuri C, Feder T, Motwani R (1997) Incremental clustering and dynamic information retrieval. In: Proceedings of the ACM symposium on theory of computing
Cox IJ, Miller ML, Minka T, Papathomas TV, Yianilos PN (2000) The beyesian image retrieval system, pichunter: theory, implementation and psychophysical experiments. In: Proceedings of the IEEE conference on computer vision and pattern recognition, XX(Y)
Elomaa T, Rousu J (1999) General and efficient multisplitting of numerical attributes. Mach Learn 36(3):1-49
Enfron B, Tibshirani R (1993) An introduction to the bootstrap. Chapman & Hall, London
Freund Y, Shapire R (1997) A decision-theoretic generalization of online learning and an application to boosting. J Comput Sys Sci 55(1):119-139
Fukunaga K (1990) Introduction to statistical pattern recognition. Academic, San Diego
Guo GD, Jain AK, Ma WY, Zhang HJ (2001) Learning similarity measure for natural image retrieval with relevance feedback. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 8-14 December 2001, Kauai, HI, 1:731-735
Bartolini I, Ciaccia P, Waas F (2001) Feedbackbypass: a new approach to interactive similarity query processing. In: Proceedings of the 27th international conference on very large databases, 31 October-2 November 2001
Ishikawa Y, Subramanya R, Faloutsos C (1998) Mindreader: query databases through multiple examples. In: Proceedings of the 24th international conference on very large databases
Joachims T (1998) Making large-scale SVM learning practical. In: Advances in kernel methods - support vector learning. MIT Press, Cambridge, MA, pp 41-56
Kononenko I (1995) On biases in estimating multi-valued attributes. In: Proceedings of the 14th international joint conference on artificial intelligence
Ma WY, Zhang H (1998) Content-based image indexing and retrieval. In: Handbook of multimedia computing. Routledge, London, pp 19-20
MacArthur SD, Brodley C, Shyu C (2000) Relevance feedback decision trees in content-based image retrieval. In: Proceedings of the IEEE workshop on CAIVL
Mitchell TM, Mitchell TM (1997) Machine learning. McGraw-Hill, New York
Niblack W(1993) The qbic project: querying images by content using color, texture, and shape. In: Proceedings of SPIE Storage and Retrieval for Image and Video Databases
Pass G, Zabih R, Miller J (1996) Comparing images using color coherence vectors. In: Proceedings of ACM Multimedia ‘96, Boston, pp 65-73
Porkaew K, Chakrabarti K, Mehrotra S (1999) Query refinement for multimedia similarity retrieval in mars. In: Proceedings of ACM Multimedia
Quinlan J (1986) Introduction of decision tree. Mach Learn 1:81-106
Rui Y, Huang T (2000) Optimizing learning in image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition, June 2000
Rui Y, Huang T, Mehrotra S (1997) Content-based image retrieval with relevance feedback in mars. In: Proceedings of the IEEE international conference on image processing
Smith JR, Chang SF (1994) Transform features for classification and discrimination in large image databases. In: Proceedings of the IEEE international conference on image processing
Su Z, Li S, Zhang H (2001) Extraction of feature subspaces for content-based image retrieval using relevance feedback. In: Proceedings of ACM Multimedia
Tieu K, Viola P (2000) Boosting image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition, June 2000
Tong S, Chang E (2001) Support vector machine active learning for image retrieval. In: Proceedings of ACM Multimedia
Vasconcelos N, Lippman A (2000) Bayesian relevance feedback for content-based image retrieval. In: Proceedings of the IEEE workshop on CAIVL
Wu Y, Zhang A (2002) Category-based search using metadatabase in image retrieval. In: Proceedings of the IEEE international conference on multimedia and expo
Wu Y, Zhang A (2002) A feature re-weighting approach for relevance feedback in image retrieval. In: Proceedings of the IEEE international conference on image processing
Wu Y, Zhang A (2002) Interactive pattern analysis for searching multimedia databases. In: Proceedings of the 8th international workshop on multimedia information systems, 31 October-2 November 2002
Yang Y, Pedersen JO (1997) A comparative study on feature selection in text categorization. In: Proceedings of ICML-97
Zhou XS, Huang TS (2001) Comparing discriminating transformations and svm for learning during multimedia retrieval. In: Proceedings of ACM Multimedia, September 2001
Zhou XS, Huang TS (2003) Relevance feedback for image retrieval: a comprehensive review. ACM Multimedia Sys 8(6):536-544
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wu, Y., Zhang, A. Interactive pattern analysis for relevance feedback in multimedia information retrieval. Multimedia Systems 10, 41–55 (2004). https://doi.org/10.1007/s00530-004-0136-5
Issue Date:
DOI: https://doi.org/10.1007/s00530-004-0136-5