Multimedia Tools and Applications

, Volume 32, Issue 1, pp 73–92 | Cite as

Capturing high-level image concepts via affinity relationships in image database retrieval

  • Mei-Ling ShyuEmail author
  • Shu-Ching Chen
  • Min Chen
  • Chengcui Zhang
  • Kanoksri Sarinnapakorn


In this paper, we present a mechanism called Markov Model Mediator (MMM) to facilitate the efficient and effective capturing of high-level image concepts in content-based image retrieval (CBIR). MMM serves as the retrieval engine of the CBIR system and uses affinity-based similarity measures. This mechanism is effective in capturing subjective user concepts in that it not only takes into consideration the global image features, but also learns the high-level concepts of the images from the history of user access patterns and access frequencies on the images in the image database, which differentiates it from the common methods in CBIR. The advantage of our proposed mechanism is that it exploits the richness in the structured description of visual contents as well as the relative affinity relationships among the images. Consequently, it provides the capability to bridge the gap between the low-level features and the high-level concepts. This mechanism is also efficient in that it integrates Principal Component Analysis (PCA) to significantly reduce the image search space at a low cost before performing exact similarity matching. An off-line training subsystem for this framework was implemented and integrated into our system. The experimental results demonstrate that MMM can effectively capture user’s high-level concept more quickly.


Content-Based Image Retrieval (CBIR) Markov Model Mediator (MMM) Principal Component Analysis (PCA) 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aksoy S, Haralick RM (2000) A weighted distance approach to relevance feedback. In: Proceedings of the international conference on pattern recognition. Elsevier, New York, pp 812–815Google Scholar
  2. 2.
    Carson, C et al (2002) Blobworld: image segmentation using expectation–maximization and its application to image querying. IEEE Trans Pattern Anal Mach Intell 24(8):1026–1038CrossRefGoogle Scholar
  3. 3.
    Chen S-C, Sista S, Shyu M-L, Kashyap RL (2000) An indexing and searching structure for multimedia database systems. In: Proceedings of IS&T/SPIE conference on storage and retrieval for media databases, San Jose, California, pp 262–270Google Scholar
  4. 4.
    Chen S-C, Shyu M-L, Zhao N, Zhang C (2003) An affinity-based image retrieval system for multimedia authoring and presentation. In: Proceedings of 11th ACM international conference on multimedia (MM’03), Berkeley, California, pp 446–447, 2–8 November 2003Google Scholar
  5. 5.
    Chen Y, Wang JZ (2002) A region-based fuzzy feature matching approach to content-based image retrieval. IEEE Trans Pattern Anal Mach Intell 24:1252–1267CrossRefGoogle Scholar
  6. 6.
    Cheng HD, Sun Y (2001 December) A hierarchical approach to color image segmentation using homogeneity. IEEE Trans Image Process 9(12):2071–2082Google Scholar
  7. 7.
    Cox IJ, Minka TP, Papathomas TV, Yianilos PN (2000) The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments. IEEE Trans Image Process 9(1):20–27CrossRefGoogle Scholar
  8. 8.
    Deerwester S, Dumais ST, Landauer TK, Furnas GW, Harshman RA (1990) Indexing by latent semantic analysis. J. Soc. Inf. Sci. 41(6):391–407CrossRefGoogle Scholar
  9. 9.
    Fauqueur J, Boujemaa N (2002) Image retrieval by regions: Coarse segmentation and fine color description. Visual Information and Information Systems, pp 24–35Google Scholar
  10. 10.
    Guttman A (1984 June) R-tree: A dynamic index structure for spatial search. In: Proceedings of ACM SIGMOD, Boston, Massachusetts, pp 47–57Google Scholar
  11. 11.
    Huang X, Chen S-C, Shyu M-L (2003) Incorporating real-valued multiple instance learning into relevance feedback for image retrieval. In: Proceedings of the IEEE international conference on multimedia & expo (ICME), vol. 1, Baltimore, Maryland, pp 321–324, 6–9 July 2003Google Scholar
  12. 12.
    Jing F, Li M, Zhang H-J, Zhang B (2002) An effective region-based image retrieval framework. In: Proceedings of the 2002 ACM workshops on multimedia, Juan-les-Pins, France, pp 456–465Google Scholar
  13. 13.
    Jobson JD (1992) Applied multivariate data analysis volume II: Categorical and multivariate methods. Springer, Berlin Heidelberg New YorkzbMATHGoogle Scholar
  14. 14.
    Johnson RA, Wichern DW (1998) Applied multivariate statistical analysis, 4th edn. Prentice-Hall, New JerseyGoogle Scholar
  15. 15.
    Jolliffe IT (2002) Principal component analysis, 2nd edn. Springer, Berlin Heidelberg New YorkzbMATHGoogle Scholar
  16. 16.
    Lin HC, Wang LL, Yang SN (1997) Color image retrieval based on hidden Markov models. IEEE Trans Image Process 6(2):332–339CrossRefGoogle Scholar
  17. 17.
    Lu Y, Hu CH, Zhu XQ, Zhang HJ, Yang Q (2000) A unified framework for semantics and feature based relevance feedback in image retrieval systems. In: Proceedings of the 8th ACM international conference on multimedia, Los Angeles, California, pp 31–37Google Scholar
  18. 18.
    Ma W-Y, Zhang HJ (1999) Content-based Image Indexing and Retrieval. Handbook of multimedia computing, Chapter 13, CRC Press, Boca Raton, FloridaGoogle Scholar
  19. 19.
    Natsev A, Rastogi R, Shim K (2004) WALRUS: A similarity retrieval algorithm for image databases. IEEE Trans Knowl Data Eng 16(3):301–316CrossRefGoogle Scholar
  20. 20.
    Ortega M, Rui Y et al (1998) Supporting ranked boolean similarity queries in MARS. IEEE Trans Knowl Data Eng 10(6):905–925CrossRefGoogle Scholar
  21. 21.
    Rabiner LR, Huang BH (1986 January) An introduction to hidden Markov models. IEEE ASSP Mag 3(1):4–16CrossRefGoogle Scholar
  22. 22.
    Rui Y, Huang TS (1999) A novel relevance feedback technique in image retrieval. In: Proceedings of the 7th ACM international conference on multimedia, Part 2, Orlando, Florida, pp 67–70Google Scholar
  23. 23.
    Shyu M-L, Chen S-C, Kashyap RL (2000) A probabilistic-based mechanism for video database management systems. In: Proceedings of IEEE international conference on multimedia and expo (ICME’00), New York, USA, pp 467–470Google Scholar
  24. 24.
    Shyu M-L, Chen S-C, Haruechaiyasak C, Shu C-M, Li S-T (2001) Disjoint web document clustering and management in electronic commerce. In: Proceedings of the 7th international conference on distributed multimedia systems (DMS’01), Tamkang University, Taipei, Taiwan, pp 494–497Google Scholar
  25. 25.
    Shyu M-L, Chen S-C, Rubin, SH (2004) Stochastic clustering for organizing distributed information sources. IEEE Trans Syst Man Cybern, Part B, Cybern 34(5):2035–2047CrossRefGoogle Scholar
  26. 26.
    Su Z, Li S, Zhang H (2001) Extraction of feature subspaces for content-based retrieval using relevance feedback. In: Proceedings of the 9th ACM international conference on multimedia (MM’01), Ottawa, Canada, pp 98–106Google Scholar
  27. 27.
    Wiederhold G (March 1992) Mediators in the architecture of future information systems. IEEE Computers 25:38–49Google Scholar
  28. 28.
    Zhang DS, Lu G (2002 August) Generic fourier descriptors for shape-based image retrieval. In: Proceedings of IEEE international conference on multimedia and expo (ICME’02), vol. 1. 562 Lausanne, Switzerland, pp 425–428Google Scholar
  29. 29.
    Zhang Q, Goldman SA, Yu W, Fritts J (2002 July) Content-based image retrieval using multiple-instance learning. In: Proceedings of the 19th international conference on machine learning, Sydney, Australia, pp 682–689Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2006

Authors and Affiliations

  • Mei-Ling Shyu
    • 1
    Email author
  • Shu-Ching Chen
    • 2
  • Min Chen
    • 2
  • Chengcui Zhang
    • 3
  • Kanoksri Sarinnapakorn
    • 1
  1. 1.Department of Electrical and Computer EngineeringUniversity of MiamiCoral GablesUSA
  2. 2.Distributed Multimedia Information System Laboratory, School of Computer ScienceFlorida International UniversityMiamiUSA
  3. 3.Department of Computer and Information ScienceUniversity of Alabama at BirminghamBirminghamUSA

Personalised recommendations