Skip to main content

Advertisement

SpringerLink
Log in

DRM: dynamic region matching for image retrieval using probabilistic fuzzy matching and boosting feature selection

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

This paper considers the semantic gap in content-based image retrieval from two aspects: (1) irrelevant visual contents (e.g. background) scatter the mapping from image to human perception; (2) unsupervised feature extraction and similarity ranking method can not accurately reveal users’ image perception. This paper proposes a novel region-based retrieval framework—dynamic region matching (DRM) to bridge the semantic gap. (1) To address the first issue, a probabilistic fuzzy region matching algorithm is adopted to retrieve and match images precisely at object level, which copes with the problem of inaccurate segmentation. (2) To address the second issue, a “FeatureBoost” algorithm is proposed to construct an effective “eigen” feature set in relevance feedback (RF) process. And the significance of each region is dynamically updated in RF learning to automatically capture users’ region of interest (ROI). (3) User’s retrieval purpose is predicted using a novel log-learning algorithm, which predicts users’ retrieval target in the feature space using the accumulated user operations. Extensive experiments have been conducted on Corel image database with over 10,000 images. The promising experimental results reveal the effectiveness of our scheme in bridging the semantic gap.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

F Query j :

the jth feature of query image

F Image j :

the jth feature of the image in current segmentation procedure

N i :

the ith iteration (from 2 to 6)

α:

the geometrical circularity of a region

S :

the size of a region

P :

the geometric center pixel in a region

μ p,q :

the center scatter of a region

x c :

the x coordinate of the geometric center pixel in a region

y c :

the y coordinate of the geometric center pixel in a region

R i :

the ith block in a region.

I 1 and I 2 :

two matching images for similarity calculation

R i and R j :

the ith and jth region of these two images, respectively

w ij :

the weight between two corresponding regions R i and R j from different images

S(R i ,R j ):

the visual similarity between these two regions using the feature set

\(w^{0}_{i}\) :

the weights of regions in query image

\(S^{k}_{MN}\) :

the region similarity matrix

n max :

the maximum matching pair to which the DRM matching stops

\((w^{0}_{i}, w^{k}_{j})\) :

matching region pairs

\(w^{0}_{1} w^{0}_{m}\) :

rewarding vector of DRM in RF learning

Feature i :

ith feature in the basis feature set

Pos i :

ith positive relevance feedback example

Neg j :

jth negative relevance feedback example

W i :

ith classifying accuracy weight of the selected eigen feature

wp i :

ith positive feedback example

wn j :

jth negative feedback example

Featureselected :

the “eigen” feature selected in the current FeatureBoost loop

T i :

the ith retrieval central matrix

\(F_{ij}^{\rm Current}\) :

the ith feature in the jth central retrieval matrix of current retrieval

\(F_{ij}^{{\rm Log}_p}\) :

the ith feature in the jth of the pth retrieval log

n related :

the number of related images returned to users

n return :

the number of images returned to users

n 0 :

the number of images in each image class

Precision:

retrieval precision

Recall:

retrieval recall

(I query, I target):

the target and query pair for retrieval performance test

References

  1. Smeuldes A., Worring M., Santini S., Gupta A. and Jain R. (2000). Content-based image retrieval at the end of early years. IEEE Trans. Pattern Anal. Mach. Intell. 22(12): 1349–1380

    Article  Google Scholar 

  2. Carson, C., Belongie, S., Greenspan, H., Malik, J.: Region-based image querying. In: Proceeding of IEEE Workshop on Content-Based Access of Image and Video Libraries, pp. 42–49 (1997)

  3. Niblack W., Barber R., Equitz W., Flickner M.D., Glasman E.H., Petkovic D., Yanker P., Faloutsos C. and Taubin G. (1993). The QBIC project: Querying images by content using color, texture and shape. Proceeding of Storage and Retr. Image Video Database 1908: 1723–1787

    Google Scholar 

  4. Pentland A. and Picard R. (1996). Sclaroff: Photobook: Content-based manipulation of image databases. Int. J. Comput. Vis. 18(3): 233–254

    Article  Google Scholar 

  5. Smithet, J.R., Chang, S.F.: SaFe: A general framework for imtegrated spatial and feature image search. In: IEEE First Workshop on Multimedia Signal Processing, pp. 301–306 (1997)

  6. Rui Y. and Huang T.S. (1997). Mehortra: Content-based image retrieval with relevance feedback in MARS. Proc. IEEE Int. Conf. Image Process. 2(26–29): 815–818

    Article  Google Scholar 

  7. Ma, W.Y., Manjunath, B.S.: Netra: A Toolbox for navigation Large Image Database. Proc. Int. Conf. Image Process. pp. 568–571 (1997)

  8. Yamada, A., Pickerings, M., Jeannin, S., Cieplinski, L.: Multimedia Content Desiption Interface—Part 3: Visual, ISO/IEC 15938–3:2001, ver. 1, 2001

  9. Manjunath B.S., Salembier P. and Sikora T. (2002). Introduction to MPEG-7: Multimedia Content Desiption Interface. Wiley, New York

    Google Scholar 

  10. Kam, A.H., Ng, T.T., Kingsbury, N.G., Fitzgerald, W.J.: Content based image retrieval through object extraction and querying. In: Proceeding of IEEE Workshop on Content-based Access of Image and Video Libraries, pp. 91–95 (2000)

  11. Moghaddam B., Biermann H. and Margaritis D. (2000). Image retrieval with local and spatial queries. Proc. IEEE Int. Conf. Image Process. 2: 542–545

    Google Scholar 

  12. Ko, B.C., Lee, H.S., Byun, H.: Region-based image retrieval system using efficient feature desiption. In: Proceeding of 15th International Conference on Pattern Recognition vol. 4, pp. 283–286 (2000)

  13. Wang, W., Song, Y., Zhang, A.: Semantics retrieval by content and context of image regions. Proceeding of the 15th International Conference on Vision Interface, Calgary, Canada (2002)

  14. Chen, Y., Wang, J.Z.: A region-based fuzzy feature matching approach to content-based image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 24(9), Sept 2002

  15. Kushki A., Androutsos P., Plataniotis K.N. and Venetsanopoulos A.N. (2004). Retrieval from artistic repositories using a decision fusion framework. IEEE Trans. Image Process. 13(2): 277–292

    Article  Google Scholar 

  16. Zhang, R., Zhang, Z.: Hidden semantic discovery in region based image retrieval. Proc. Int. Conf. Comput. Vis. Pattern Recognit. 2, II-996–II-1001 July 2004

  17. Li, J., Wang, J.Z., Widerhold, G.: IRM: Integrated region matching for image retrieval. In: Proceeding of ACM Multimedia, Los Angeles, CA, pp. 147–156 (2000)

  18. Wang T., Rui Y. and Sun J.-G. (2004). Constraint based region matching for image retrieval. Int. J. Comput. Vis. 56(1/2/3): 37–45

    Article  Google Scholar 

  19. Gong, Y., Zhang H.J., Chua, T.C.: An image database system with content capturing and fast image indexing abilities. Proc. IEEE International Conference on Multimedia Computing and Systems, Boston, pp.121–130, 14–19 May 1994

  20. Buckley, C., Salton, G.:Optimization of relevance feedback weights. Proceedings of SIGIR’95

  21. Rocchio J.J. Jr. (1971). Relevance feedback in information retrieval. In: Salton, G. (eds) The SMART Retrieval System: Experiments in Automatic Document Processing, pp 313–323. Prentice-Hall, New Jessey

    Google Scholar 

  22. Kushki A., Androutsos P., Plataniotis K.N. and Venetsanopoulos A.N. (2004). Query feedback for interactive image retrieval. IEEE Trans. Circuits Syst. Video Technol. 14(5): 644–655

    Article  Google Scholar 

  23. Burges C.J.C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining Knowl. Discov. 2(2): 121–167

    Article  Google Scholar 

  24. Su, Z., Zhang, H., Li, S.: Relevance feedback in content-based image retrieval: Bayesian framework, feature subspaces and progressive learning. IEEE Trans. Image Process. 12(3), (2003)

  25. He, X., King, O., Ma, W.Y., Li, M., Zhang, H.J.: Learning asemantic space from user’s relevance feedback for image retrieval. IEEE Trans. Circuits and Syst. Video Technol. 13(1), 39–48 Jan 2003

    Google Scholar 

  26. Cai, D., He, X., Li, Z., Ma, W.-Y., Wen, J.: Hierarchical clustering of WWW image search results using visual, textual and link information. ACM Multimedia 04, New York, USA, pp. 952–959, October 10–16, 2004.

  27. Serra J. (1988). Image Analysis and Mathematical Morphology, vol 2. Academic Press, New York

    Google Scholar 

  28. Fukunaga, K., Hostetler, L.D.: The estimation of gradient of a density function, with applications in pattern recognition. IEEE Trans. Inf. Theory IT-21(1), (1975)

  29. Comaniciu D. and Meer P. (2002). Mean Shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach.Intell. 24(5): 603–619

    Article  Google Scholar 

  30. Tao, D., Tang, X., Li, X., Wu, X.: Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 28(7) July, 2006

  31. Tieu K. and Viola P. (2004). Boosting image retrieval. Int. J. Comput. Vis. 56(1/2/3): 17–36

    Article  Google Scholar 

  32. Hertz T., Bar-Hillel A. and Weinshall D. (2004). Learning distance functions for image retrieval. Compu. Vis. Pattern Recognit. 2: 570–577

    Google Scholar 

  33. Hoi, C.-H., Lyu, M.R.: A novel log-based relevance feedback technique in content-based image retrieval. ACM Multimedia 04, pp. 24–31, October 10–16, 2004, New York, USA

  34. He J., Li M., Zhang H., Tong H. and Zhang C (2006). Generalized manifold-ranking based image retrieval. IEEE Trans. Image Proces. 15(10): 3170–3177

    Article  Google Scholar 

  35. Stricker M. and Orengo M. (1995). Similarity of color images. SPIE Storage Retr. Image Video Databases 2185: 381–392

    Google Scholar 

  36. Haralick R.M., Shanmugam K. and Dinstein I. (1973). Texture features for image classification. IEEE Trans. Syst. Man Cybern 3: 610–621

    Article  Google Scholar 

  37. Yu, H., Li, M., Zhang, H., Feng, J.: Color texture moments for content-based image retrieval. In: Proc. of IEEE 2002 Inter. Conf. Image Process. (2002)

  38. Rui Y., Huang T.S., Mehrotra S. and Ortega M. (1998). Relevance Feedback: A power tool for interactive content-based image retrieval. IEEE Trans Circuits Syst. Video Technol. 8(5): 644–655

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Visual Intelligence Lab, School of Computer Science and Technology, Harbin Institute of Technology, West Dazhi Street, Harbin, 150001, China

    Rongrong Ji, Hongxun Yao & Dawei Liang

Authors
  1. Rongrong Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongxun Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dawei Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rongrong Ji.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ji, R., Yao, H. & Liang, D. DRM: dynamic region matching for image retrieval using probabilistic fuzzy matching and boosting feature selection. SIViP 2, 59–71 (2008). https://doi.org/10.1007/s11760-007-0037-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-007-0037-0

Keywords

Search

Navigation