Skip to main content
SpringerLink
Log in

Graph matching based on local and global information of the graph nodes

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Graph matching is an essential NP-problem in computer vision and pattern recognition. In this paper, we propose an approximate graph matching method. This method formulates the problem of computing the correspondences between two graphs as a problem of selecting nodes on an association graph. The nodes of the association graph represent candidate correspondences between the two original graphs. Our method first constructs an affinity matrix based on both the global and local information of the original graphs’ nodes. Each element of this matrix is used to measure the mutual consistency of a pair of nodes within the association graph. Our method then applies the reweighted random walks technique that preserves the one-to-one matching constraint to simulate random walks on the association graph and to iteratively compute a quasi-stationary distribution. To discretize this distribution, our method finally applies the Hungarian algorithm and obtains an approximate matching between the original two graphs. Experimental results demonstrate the effectiveness of our method for graph matching and the ability of our method for being robust to outlier and deformation noise.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. We did not show the experiments about the edge density because the experiment about the edge density cannot obtain the information of the nodes (x-y node coordinates) when computing the affinity matrix.

  2. For each frame pair, we randomly selected ns points of the first image by 10 times to generate 10 graph pairs. We used the average matching accuracy of these 10 pairs to represent the matching accuracy corresponding to this frame pair.

  3. http://www.vision.caltech.edu/Image_Datasets/Caltech101/

  4. http://research.microsoft.com/en-us/projects/objectclassrecognition/

References

  1. Albarelli A, Bulo SR, Torsello A, Pelillo M (2009) Matching as a non-cooperative game. In: IEEE conference on computer vision, pp 1319–1326

  2. Berg AC, Berg TL, Malik J (2005) Shape matching and object recognition using low distortion correspondences. In: IEEE Conference on computer vision and pattern recognition, vol 1, pp 26–33

  3. Carletti V, Foggia P, Greco A, Vento M, Vigilante V (2019) VF3-Light: a lightweight subgraph isomorphism algorithm and its experimental evaluation. Pattern Recognit Lett 125:591–596

    Article  Google Scholar 

  4. Chen HT, Lin HH, Liu TL (2001) Multi-object tracking using dynamical graph matching. In: IEEE Conference on computer vision and pattern recognition, vol 2, pp 210–217

  5. Cho M, Lee J, Lee KM (2010) Reweighted random walks for graph matching. In: European conference on computer vision. Springer, pp 492–505

  6. Cho M, Sun J, Duchenne O, Ponce J (2014) Finding matches in a haystack: a max-pooling strategy for graph matching in the presence of outliers. In: IEEE Conference on computer vision and pattern recognition, pp 2083–2090

  7. Cordella LP, Foggia P, Sansone C, Vento M (2004) A (sub)graph isomorphism algorithm for matching large graphs. IEEE Trans Pattern Anal Mach Intell 26(10):1367–1372

    Article  Google Scholar 

  8. Cour T, Shi J (2007) Solving Markov random fields with spectral relaxation. In: Artificial intelligence and statistics, pp 75–82

  9. Cour T, Srinivasan P, Shi J (2007) Balanced graph matching. In: Conference on advances in neural information processing systems, pp 313–320

  10. Duchenne O, Joulin A, Ponce J (2011) A graph-matching kernel for object categorization. In: International conference on computer vision. IEEE, pp 1792–1799

  11. Foggia P, Percannella G, Vento M (2014) Graph matching and learning in pattern recognition in the last 10 year. Int J Pattern Recognit Artifici Intelligenc 28 (1):1–40

    MathSciNet  Google Scholar 

  12. Gold S, Rangarajan A (1996) A graduated assignment algorithm for graph matching. IEEE Trans Pattern Anal Mach Intell 18(4):377–388

    Article  Google Scholar 

  13. Gori M, Maggini M, Sarti L (2004) Graph matching using random walks. In: International conference on pattern recognition, vol 3, pp 394–397

  14. Gori M, Maggini M, Sarti L (2005) The RW2 algorithm for exact graph matching. Lect Notes Comput Sci 3686:81–88

    Article  Google Scholar 

  15. Haveliwala TH (2002) Topic-sensitive pagerank. In: ACM International conference on World Wide Web, pp 517–526

  16. Hu YT, Lin YY (2016) Progressive feature matching with alternate descriptor selection and correspondence enrichment. In: IEEE Conference on computer vision and pattern recognition, pp 346–354

  17. Hu N, Rustamov RM, Guibas L (2014) Stable and informative spectral signatures for graph matching. In: IEEE Conference on computer vision and pattern recognition, pp 2305–2312

  18. Jiang B, Zhao H, Tang J, Luo B (2014) A sparse nonnegative matrix factorization technique for graph matching problems. Pattern Recognit 47(2):736–747

    Article  Google Scholar 

  19. Jiang B, Tang J, Ding C, Luo B (2015) A local sparse model for matching problem. In: Twenty-ninth AAAI conference on artificial intelligence, pp 3790–3796

  20. Jiang B, Tang J, Cao X, Luo B (2017) Lagrangian relaxation graph matching. Pattern Recognit 61:255–265

    Article  Google Scholar 

  21. Jiang B, Tang J, Ding CH, Luo B (2017) Nonnegative orthogonal graph matching. In: Association for the advance of artificial intelligence, pp 4089–4095

  22. Jiang B, Tang J, Luo B (2019) Efficient Feature Matching via Nonnegative Orthogonal Relaxation. Int J Comput Vis 127(9):1345–1360

    Article  MathSciNet  Google Scholar 

  23. Khue Le-Huu D, Paragios N (2017) Alternating direction graph matching. In: IEEE Conference on computer vision and pattern recognition, pp 6253–6261

  24. Lawler EL (1963) The quadratic assignment problem. Manag Sci 9(4):586–599

    Article  MathSciNet  Google Scholar 

  25. Leordeanu M, Hebert M (2005) A spectral technique for correspondence problems using pairwise constraints. In: IEEE International conference on computer vision, vol 2, pp 1482–1489

  26. Leordeanu M, Hebert M, Sukthankar R (2009) An integer projected fixed point method for graph matching and map inference. In: International conference on neural information processing systems, pp 1114–1122

  27. Liu ZY, Qiao H (2014) GNCCP-graduated nonconvexity and concavity procedure. IEEE Trans Pattern Anal Mach Intell 36(6):1258–1267

    Article  Google Scholar 

  28. Liu M, Wang L, Nie L, Dai J, Ji D (2016) Event graph based contradiction recognition from big data collection. Neurocomputing 181:64–75

    Article  Google Scholar 

  29. Liu M, Wei Y, Qian W, Zhang H (2017) Robust plant cell tracking in noisy image sequences using optimal crf graph matching. IEEE Signal Proc Lett 24(8):1168–1172

    Article  Google Scholar 

  30. Liu X, Li F, Na Z (2017) Optimal resource allocation in simultaneous cooperative spectrum sensing and energy harvesting for multichannel cognitive radio. IEEE Access 5:3801–3812

    Article  Google Scholar 

  31. Liu X, Jia M, Na Z, Lu W, Li F (2018) Multi-modal cooperative spectrum sensing based on Dempster-Shafer fusion in 5g-based cognitive radio. IEEE Access 6(99):199–208

    Article  Google Scholar 

  32. Liu ZY, Qiao H, Yang X, Hoi SC (2014) Graph matching by simplified convex-concave relaxation procedure. Int J Comput Vis 109(3):169–186

    Article  MathSciNet  Google Scholar 

  33. Mills-Tettey GA, Stentz A, Dias MB (2007) The dynamic hungarian algorithm for the assignment problem with changing costs. Carnegie Mellon University

  34. Nie W, Ding H, Liu A, Deng Z, Su Y (2018) Subgraph learning for graph matching. Pattern Recognit Lett, in press

  35. Nie WZ, Liu AA, Gao Y, Su YT (2018) Hyper-clique graph matching and applications. Trans on Circuit and Syst for Video Technol 99:1–12

    Google Scholar 

  36. Riesen K, Jiang X, Bunke H (2010) Exact and inexact graph matching: methodology and applications. Manag Min Graph Data, 217–247

  37. Sahni S, Gonzalez T (1976) P-complete approximation problems. J ACM 23 (3):555–565

    Article  MathSciNet  Google Scholar 

  38. Solnon C (2010) Alldifferent-based filtering for subgraph isomorphism. Artif Intell 174:850–864

    Article  MathSciNet  Google Scholar 

  39. Ullmann JR (1976) An algorithm for subgraph isomorphism. J ACM 23(1):31–42

    Article  MathSciNet  Google Scholar 

  40. Ullmann JR (2010) Bit-vector algorithms for binary constraint satisfaction and subgraph isomorphism. J Exp Algorithmics (JEA) 15:1–6

    MathSciNet  MATH  Google Scholar 

  41. Wang T, Ling H, Lang C, Feng S (2018) Graph matching with adaptive and branching path following. IEEE Trans Pattern Anal Mach Intell 40(12):2853–2867

    Article  Google Scholar 

  42. Wu J, Shen H, Li YD, Xiao ZB, Lu MY, Wang CL (2013) Learning a hybrid similarity measure for image retrieval. Pattern Recognit 46(11):2927–2939

    Article  Google Scholar 

  43. Wu Y, Gong M, Ma W, Wang S (2019) High-order graph matching based on ant colony optimization. Neurocomputing 328:97–104

    Article  Google Scholar 

  44. Yang X, Qiao H, Liu ZY (2015) Outlier robust point correspondence based on gnccp. Pattern Recognit Lett 55:8–14

    Article  Google Scholar 

  45. Yan J, Yin XC, Lin W, Deng C, Zha H, Yang X (2016) A short survey of recent advances in graph matching. In: ACM International conference on multimedia retrieval, pp 167–174

  46. Yu T, Yan J, Wang Y, Liu W (2018) Generalizing graph matching beyond quadratic assignment model. In: Advances in neural information processing systems, pp 853–863

  47. Zhang J, Ma S, Sclaroff S (2014) Meem: robust tracking via multiple experts using entropy minimization. In: European conference on computer vision. Springer, pp 188–203

  48. Zhang Z, Shi Q, McAuley J, Wei W, Zhang Y, Van Den Hengel A (2016) Pairwise matching through max-weight bipartite belief propagation. In: IEEE Conference on computer vision and pattern recognition, pp 1202–1210

  49. Zhang L, Liu M, Chen L, Qiu L, Zhang C, Hu Y, Zimmermann R (2017) Online modeling of esthetic communities using deep perception graph analytics. IEEE Trans Multimed 20(6):1462–1474

    Article  Google Scholar 

  50. Zhou F, Torre FDL (2012) Factorized graph matching. In: IEEE Conference on computer vision and pattern recognition, pp 127–134

Download references

Acknowledgements

This research was supported by Natural Science Foundation of Shandong province (No. ZR2019BF026, ZR2019MF013, ZR2017BF031), Project of Jinan Scientific Research Leader’s Laboratory (No. 2018GXRC023) and Doctoral Program of University of Jinan (No. 160100313). In addition, we thank Prof. Caiming Zhang for putting forward some good ideas and suggestions when revising the manuscript.

Author information

Author notes

    Authors and Affiliations

    1. Shandong Provincial Key Laboratory of Network based Intelligent Computing, University of Jinan, Jinan, 250022, China

      Yaru Zhan, Xiuyang Zhao, Xue Lin, Junkai Liu, Mingjun Liu & Dongmei Niu

    Authors
    1. Yaru Zhan
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Xiuyang Zhao
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Xue Lin
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Junkai Liu
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Mingjun Liu
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Dongmei Niu
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Dongmei Niu.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Yaru Zhan and Xiuyang Zhao contributed equally to this work and should be considered co-first authors.

    Rights and permissions

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Zhan, Y., Zhao, X., Lin, X. et al. Graph matching based on local and global information of the graph nodes. Multimed Tools Appl 79, 11567–11590 (2020). https://doi.org/10.1007/s11042-019-08516-x

    Download citation

    • Received:

    • Revised:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11042-019-08516-x

    Keywords

    Search

    Navigation