Multimedia Tools and Applications

, Volume 30, Issue 2, pp 131–147 | Cite as

Probabilistic semantic network-based image retrieval using MMM and relevance feedback

  • Mei-Ling Shyu
  • Shu-Ching Chen
  • Min Chen
  • Chengcui Zhang
  • Chi-Min Shu


The performance of content-based image retrieval (CBIR) systems is largely limited by the gap between the low-level features and high-level semantic concepts. In this paper, a probabilistic semantic network-based image retrieval framework using relevance feedback is proposed to bridge this gap, which not only takes into consideration the low-level image content features, but also learns high-level concepts from a set of training data, such as access frequencies and access patterns of the images. One of the distinct properties of our framework is that it exploits the structured description of visual contents as well as the relative affinity measurements among the images. Consequently, it provides the capability to bridge the gap between the low-level features and high-level concepts. Moreover, such high-level concepts can be learned off-line, and can be utilized and refined based on the user’s specific interest during the on-line retrieval process. Our experimental results demonstrate that the proposed framework can effectively assist in retrieving more accurate results for user queries.


Content-based image retrieval Probabilistic semantic network MMM mechanism Relevance feedback 



For Mei-Ling Shyu, this research was supported in part by NSF ITR (Medium) IIS-0325260. For Shu-Ching Chen, this research was supported in part by NSF HRD-0317692 and NSF EIA-0220562. For Chengcui Zhang, this research was supported in part by UAB Faculty Development Award, SBE-0245090 and the UAB ADVANCE program of the Office for the Advancement of Women in Science and Engineering.


  1. 1.
    Beigi M, Benitez A, Chang S-F (1998) Metaseek: a content-based meta search engine for images. In: Proceedings of IS&T/SPIE Storage and Retrieval for Image and Video Databases. San Jose, California, pp 118–128Google Scholar
  2. 2.
    Cheng HD, Sun Y (2001, December) A hierarchical approach to color image segmentation using homogeneity. IEEE Trans Image Process 9(12):2071–2082Google Scholar
  3. 3.
    Cox IJ, Miller ML, Minka TP, Papathornas TV, Yianilos PN (2000, January) The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments. IEEE Trans Image Process 9(1):20–37CrossRefGoogle Scholar
  4. 4.
    Flickner M et al (1995, September) Query by image and video content: the QBIC system. IEEE Computer 28(9):23–31Google Scholar
  5. 5.
    Frank O, Strauss D (1986) Markov graphs. J Am Stat Assoc 81:832–842zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Ishikawa Y, Subramanya R, Faloutsos C (1998) Mindreader: query databases through multiple examples. In: Proceedings of the 24th VLDB Conference (VLDB’98). New York, USA, pp 218–227Google Scholar
  7. 7.
    Kaplan LM et al (1998) Fast texture database retrieval using extended fractal features. In: Proceedings of IS&T/SPIE Conference on Storage and Retrieval for Media Databases, pp 162–173Google Scholar
  8. 8.
    Lin HC et al (1997) Color image retrieval based on hidden Markov models. IEEE Trans Image Process 6(2):332–339CrossRefGoogle Scholar
  9. 9.
    Lu Y, Zhang HJ, Liu W, Hu C (2003) Joint semantics and feature based image retrieval using relevance feedback. IEEE Trans Multimedia 5(3):339–347CrossRefGoogle Scholar
  10. 10.
    Ma W-Y, Zhang HJ (1999) Content-based image indexing and retrieval. Handbook of multimedia computing, Chapter 13 CRCGoogle Scholar
  11. 11.
    Naphade MR, Huang TS (2001, March) A probabilistic framework for semantic video indexing, filtering and retrieval. IEEE Trans Multimedia 3(1):141–151CrossRefGoogle Scholar
  12. 12.
    Pentland A, Picard RW, Sclaroff S (1994) Photobook: tools for content-based manipulation of image databases. In: Proceedings of IS&T/SPIE Storage and Retrieval for Image and Video Databases II, Vol. 2185, pp. 34–47Google Scholar
  13. 13.
    Rabiner LR, Huang BH (1986, January) An introduction to hidden Markov models. IEEE ASSP Mag 3(1):4–16CrossRefGoogle Scholar
  14. 14.
    Rui Y, Huang TS, Mehrotra S (1997) Content-based image retrieval with relevance feedback in MARS. In: Proceedings of the 1997 International Conference on Image Processing (ICIP’97) (3-Volumn Set), pp 815–818Google Scholar
  15. 15.
    Rui Y, Huang TS, Mehrotra S (1998) Relevance feedback: a power tool for interactive content-based image retrieval. IEEE Trans on Circuit and Video Technology, Special Issue on Segmentation, Description, and Retrieval of Video Content, 18(5):644–655Google Scholar
  16. 16.
    Shyu M-L, Chen S-C, Shu C-M (2000, October) Affinity-based probabilistic reasoning and document clustering on the WWW. In: Proceedings of the \(24^{th}\) IEEE Computer Society International Computer Software and Applications Conference (COMPSAC), Taipei, Taiwan, pp 149–154Google Scholar
  17. 17.
    Shyu M-L, Chen S-C, Kashyap RL (2000) A probabilistic-based mechanism for video database management systems. In: Proceedings of IEEE International Conference on Multimedia and Expo (ICME’00), New York, USA, pp 467–470Google Scholar
  18. 18.
    Shyu M-L, Chen S-C, Chen M, Zhang C, Sarinnapakorn K (2003) Image database retrieval utilizing affinity relationship. In: Proceedings of the first ACM International Workshop on Multimedia Databases (ACM MMDB’03), New Orleans, Louisiana, pp 78–85Google Scholar
  19. 19.
    Smith JR, Chang SF (1996, November) VisualSEEK: a fully automated content-based image query system. In: Proceedings of ACM International Conference on Multimedia, Boston, USA, pp 87–98Google Scholar
  20. 20.
    Stehling RO, Nascimento MA, Falcao AX (2000) On shapes of colors for content-based image retrieval. In: ACM International Workshop on Multimedia Information Retrieval (ACM MIR’00), Los Angeles, California, USA pp 171–174Google Scholar
  21. 21.
  22. 22.
    Wiederhold G (1992, March) Mediators in the architecture of future information systems. IEEE Computers 38–49Google Scholar
  23. 23.
    Wolf W (1997, April) Hidden Markov model parsing of video programs. In: Proceedings of the International Conference of Acoustics, Speech and Signal Processing. Munich, Germany, pp 2609–2611Google Scholar
  24. 24.
    Zhang DS, Lu G (2002, August) Generic Fourier descriptors for shape-based image retrieval. In: Proceedings of IEEE International Conference on Multimedia and Expo (ICME’02). Lausanne, Switzerland, pp 425–428Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  • Mei-Ling Shyu
    • 1
  • Shu-Ching Chen
    • 2
  • Min Chen
    • 2
  • Chengcui Zhang
    • 3
  • Chi-Min Shu
    • 4
  1. 1.Department of Electrical and Computer EngineeringUniversity of MiamiCoral GablesUSA
  2. 2.Distributed Multimedia Information System Laboratory, School of Computing and Information SciencesFlorida International UniversityMiamiUSA
  3. 3.Department of Computer and Information SciencesUniversity of Alabama at BirminghamBirminghamUSA
  4. 4.Department of Environmental and Safety EngineeringNational Yunlin University of Science and TechnologyYunlinRepublic of China

Personalised recommendations