Layered Neighborhood Expansion for Incremental Multiple Graph Matching

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12355)


Graph matching has been a fundamental problem in computer vision and pattern recognition, for its practical flexibility as well as NP hardness challenge. Though the matching between two graphs and among multiple graphs have been intensively studied in literature, the online setting for incremental matching of a stream of graphs has been rarely considered. In this paper, we treat the graphs as graphs on a super-graph, and propose a novel breadth first search based method for expanding the neighborhood on the super-graph for a new coming graph, such that the matching with the new graph can be efficiently performed within the constructed neighborhood. Then depth first search is performed to update the overall pairwise matchings. Moreover, we show our approach can also be readily used in the batch mode setting, by adaptively determining the order of coming graph batch for matching, still under the neighborhood expansion based incremental matching framework. Experiments on both online and offline matching of graph collections show our approach’s state-of-the-art accuracy and efficiency.


Multi-graph matching Clustering Self-supervised learning 


  1. 1.
    Bunke, H.: Graph matching: theoretical foundations, algorithms, and applications. In: Vision Interface (2000)Google Scholar
  2. 2.
    Caetano, T., McAuley, J., Cheng, L., Le, Q., Smola, A.J.: Learning graph matching. TPAMI 31(6), 1048–1058 (2009)CrossRefGoogle Scholar
  3. 3.
    Chen, Y., Guibas, L., Huang, Q.: Near-optimal joint object matching via convex relaxation. In: ICML (2014)Google Scholar
  4. 4.
    Chertok, M., Keller, Y.: Efficient high order matching. TPAMI 32, 2205–2215 (2010)CrossRefGoogle Scholar
  5. 5.
    Cho, M., Alahari, K., Ponce, J.: Learning graphs to match. In: ICCV (2013)Google Scholar
  6. 6.
    Cho, M., Lee, J., Lee, K.M.: Reweighted random walks for graph matching. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 492–505. Springer, Heidelberg (2010). Scholar
  7. 7.
    Conte, D., Foggia, P., Sansone, C., Vento, M.: Thirty years of graph matching in pattern recognition. IJPRAI 18, 265–298 (2004)Google Scholar
  8. 8.
    Duchenne, O., Bach, F., Kweon, I., Ponce, J.: A tensor-based algorithm for high-order graph matching. TPAMI 33, 2383–2395 (2011)CrossRefGoogle Scholar
  9. 9.
    Egozi, A., Keller, Y., Guterman, H.: A probabilistic approach to spectral graph matching. TPAMI 35, 18–27 (2013)CrossRefGoogle Scholar
  10. 10.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Foggia, P., Percannella, G., Vento, M.: Graph matching and learning in pattern recognition in the last 10 years. IJPRAI 33(1), 1450001 (2014)MathSciNetGoogle Scholar
  12. 12.
    Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness. W. H. Freeman and Company, New York (1990)zbMATHGoogle Scholar
  13. 13.
    Gold, S., Rangarajan, A.: A graduated assignment algorithm for graph matching. TPAMI 18, 377–388 (1996)CrossRefGoogle Scholar
  14. 14.
    Guibas, L.J., Huang, Q., Liang, Z.: A condition number for joint optimization of cycle-consistent networks. In: NeurIPS (2019)Google Scholar
  15. 15.
    Hu, N., Huang, Q., Thibert, B., Guibas, L.J.: Distributable consistent multi-object matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2463–2471 (2018)Google Scholar
  16. 16.
    Hu, N., Rustamov, R.M., Guibas, L.: Graph matching with anchor nodes: a learning approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2906–2913 (2013)Google Scholar
  17. 17.
    Hu, N., Rustamov, R.M., Guibas, L.: Stable and informative spectral signatures for graph matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2305–2312 (2014)Google Scholar
  18. 18.
    Huang, Q., Zhang, G., Gao, L., Hu, S., Butscher, A., Guibas, L.: An optimization approach for extracting and encoding consistent maps in a shape collection. ACM Trans. Graph. (TOG) 31, 1–11 (2012)Google Scholar
  19. 19.
    Kulesza, A., Taskar, B., Liu, L.: Determinantal point processes for machine learning. Found. Trends Mach. Learn. 5, 123–286 (2012)CrossRefGoogle Scholar
  20. 20.
    Leordeanu, M., Sukthankar, R., Hebert, M.: Unsupervised learning for graph matching. IJCV 96, 28–45 (2012)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Leordeanu, M., Zanfir, A., Sminchisescu, C.: Semi-supervised learning and optimization for hypergraph matching. In: ICCV (2011)Google Scholar
  22. 22.
    Loiola, E.M., de Abreu, N.M., Boaventura-Netto, P.O., Hahn, P., Querido, T.: A survey for the quadratic assignment problem. EJOR 176, 657–690 (2007)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Ngoc, Q., Gautier, A., Hein, M.: A flexible tensor block coordinate ascent scheme for hypergraph matching. In: CVPR (2015)Google Scholar
  24. 24.
    Pachauri, D., Kondor, R., Vikas, S.: Solving the multi-way matching problem by permutation synchronization. In: NIPS (2013)Google Scholar
  25. 25.
    Shi, X., Ling, H., Hu, W., Xing, J., Zhang, Y.: Tensor power iteration for multi-graph matching. In: CVPR (2016)Google Scholar
  26. 26.
    Solé-Ribalta, A., Serratosa, F.: Models and algorithms for computing the common labelling of a set of attributed graphs. CVIU 115, 929–945 (2011)zbMATHGoogle Scholar
  27. 27.
    Solé-Ribalta, A., Serratosa, F.: Graduated assignment algorithm for multiple graph matching based on a common labeling. IJPRAI 27, 1350001 (2013)MathSciNetGoogle Scholar
  28. 28.
    Swoboda, P., Mokarian, A., Theobalt, C., Bernard, F., et al.: A convex relaxation for multi-graph matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11156–11165 (2019)Google Scholar
  29. 29.
    Vento, M.: A long trip in the charming world of graphs for pattern recognition. Pattern Recogn. 48(2), 291–301 (2015)CrossRefGoogle Scholar
  30. 30.
    Wang, R., Yan, J., Yang, X.: Learning combinatorial embedding networks for deep graph matching. In: ICCV (2019)Google Scholar
  31. 31.
    Yan, J., Cho, M., Zha, H., Yang, X., Chu, S.: Multi-graph matching via affinity optimization with graduated consistency regularization. TPAMI 38, 1228–1242 (2016)CrossRefGoogle Scholar
  32. 32.
    Yan, J., Li, Y., Liu, W., Zha, H., Yang, X., Chu, S.M.: Graduated consistency-regularized optimization for multi-graph matching. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 407–422. Springer, Cham (2014). Scholar
  33. 33.
    Yan, J., Tian, Y., Zha, H., Yang, X., Zhang, Y., Chu, S.: Joint optimization for consistent multiple graph matching. In: ICCV (2013)Google Scholar
  34. 34.
    Yan, J., Wang, J., Zha, H., Yang, X., Chu, S.: Consistency-driven alternating optimization for multigraph matching: a unified approach. IEEE Trans. Image Process. 24(3), 994–1009 (2015)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Yan, J., Xu, H., Zha, H., Yang, X., Liu, H., Chu, S.: A matrix decomposition perspective to multiple graph matching. In: ICCV (2015)Google Scholar
  36. 36.
    Yan, J., Yin, X., Lin, W., Deng, C., Zha, H., Yang, X.: A short survey of recent advances in graph matching. In: ICMR (2016)Google Scholar
  37. 37.
    Yan, J., Zhang, C., Zha, H., Liu, W., Yang, X., Chu, S.: Discrete hyper-graph matching. In: CVPR (2015)Google Scholar
  38. 38.
    Yu, T., Yan, J., Liu, W., Li, B.: Incremental multi-graph matching via diversity and randomness based graph clustering. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 142–158. Springer, Cham (2018). Scholar
  39. 39.
    Zass, R., Shashua, A.: Probabilistic graph and hypergraph matching. In: CVPR (2008)Google Scholar
  40. 40.
    Zhang, Z.: Iterative point matching for registration of free-form curves and surfaces. IJCV 13, 119–152 (1994)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringShanghai Jiao Tong UniversityShanghaiChina
  2. 2.MoE Key Laboratory of Artificial Intelligence, AI InstituteShanghai Jiao Tong UniversityShanghaiChina
  3. 3.National Institute of InformaticsChiyodaJapan

Personalised recommendations