Graduated Consistency-Regularized Optimization for Multi-graph Matching

  • Junchi Yan
  • Yin Li
  • Wei Liu
  • Hongyuan Zha
  • Xiaokang Yang
  • Stephen Mingyu Chu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8689)


Graph matching has a wide spectrum of computer vision applications such as finding feature point correspondences across images. The problem of graph matching is generally NP-hard, so most existing work pursues suboptimal solutions between two graphs. This paper investigates a more general problem of matching N attributed graphs to each other, i.e. labeling their common node correspondences such that a certain compatibility/affinity objective is optimized. This multi-graph matching problem involves two key ingredients affecting the overall accuracy: a) the pairwise affinity matching score between two local graphs, and b) global matching consistency that measures the uniqueness and consistency of the pairwise matching results by different sequential matching orders. Previous work typically either enforces the matching consistency constraints in the beginning of iterative optimization, which may propagate matching error both over iterations and across different graph pairs; or separates score optimizing and consistency synchronization in two steps. This paper is motivated by the observation that affinity score and consistency are mutually affected and shall be tackled jointly to capture their correlation behavior. As such, we propose a novel multi-graph matching algorithm to incorporate the two aspects by iteratively approximating the global-optimal affinity score, meanwhile gradually infusing the consistency as a regularizer, which improves the performance of the initial solutions obtained by existing pairwise graph matching solvers. The proposed algorithm with a theoretically proven convergence shows notable efficacy on both synthetic and public image datasets.


Permutation Matrix Graph Match Quadratic Assignment Problem Consistency Constraint Assignment Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
  2. 2.
  3. 3.
  4. 4.
    Besl, P., McKay, N.: A method for registration of 3-d shapes. PAMI (1992)Google Scholar
  5. 5.
    Brendel, W., Todorovic, S.: Learning spatiotemporal graphs of human activities. In: ICCV (2011)Google Scholar
  6. 6.
    Caetano, T., McAuley, J., Cheng, L., Le, Q., Smola, A.J.: Learning graph matching. IEEE Transaction on PAMI 31(6), 1048–1058 (2009)CrossRefGoogle Scholar
  7. 7.
    Chertok, M., Keller, Y.: Efficient high order matching. PAMI (2010)Google Scholar
  8. 8.
    Cho, M., Alahari, K., Ponce, J.: Learning graphs to match. In: ICCV (2013)Google Scholar
  9. 9.
    Cho, M., Lee, J., Lee, K.M.: Reweighted random walks for graph matching. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 492–505. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
    Cho, M., Lee, K.M.: Progressive graph matching: Making a move of graphs via probabilistic voting. In: CVPR (2012)Google Scholar
  11. 11.
    Cho, M., Sun, J., Duchenne, O., Ponce, J.: Finding matches in a haystack: A max-pooling strategy for graph matching in the presence of outliers. In: CVPR (2014)Google Scholar
  12. 12.
    Conte, D., Foggia, P., Sansone, C., Vento, M.: Thirty years of graph matching in pattern recognition. IJPRAI (2004)Google Scholar
  13. 13.
    Duchenne, O., Bach, F., Kweon, I., Ponce, J.: A tensor-based algorithm for high-order graph matching. In: CVPR (2009)Google Scholar
  14. 14.
    Eshera, M.A., Fu, K.S.: An image understanding system using attributed symbolic representation and inexact graph-matching. PAMI (1986)Google Scholar
  15. 15.
    Fischler, M., Bolles, R.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. of the ACM, 381–395 (1981)Google Scholar
  16. 16.
    Foggia, P., Percannella, G., Vento, M.: Graph matching and learning in pattern recognition in the last 10 years. IJPRAI (2014)Google Scholar
  17. 17.
    Gallagher, B.: Matching structure and semantics: A survey on graph-based pattern matching. In: AAAI, pp. 45–53 (2006)Google Scholar
  18. 18.
    Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness. W. H. Freeman and Co., New York (1990)Google Scholar
  19. 19.
    Gavril, F.: Generating the maximum spanning trees of a weighted graph. Journal of Algorithms, 592–597 (1987)Google Scholar
  20. 20.
    Goesele, M., Snavely, N., Curless, B., Hoppe, H., Seitz, S.: Multi-view stereo for community photo collections. In: ICCV (2007)Google Scholar
  21. 21.
    Gold, S., Rangarajan, A.: A graduated assignment algorithm for graph matching. IEEE Transaction on PAMI (1996)Google Scholar
  22. 22.
    Hancock, E.R., Wilson, R.C.: Pattern analysis with graphs: Parallel work at bern and york. Pattern Recognition Letters, 833–841 (2012)Google Scholar
  23. 23.
    Hu, N., Rustamov, R.M., Guibas, L.: Graph mmatching with anchor nodes: a learning approach. In: CVPR (2013)Google Scholar
  24. 24.
    Huang, Q., Zhang, G., Gao, L., Hu, S., Butscher, A., Guibas, L.: An optimization approach for extracting and encoding consistent maps in a shape collection. ACM Transactions on Graphics, TOG (2012)Google Scholar
  25. 25.
    Huang, Q.X., Flory, S., Gelfand, N., Hofer, M., Pottmann, H.: Reassembling fractured objects by geometric matching. ACM Trans. Graph., 569–578 (2006)Google Scholar
  26. 26.
    Kim, V.G., Li, W., Mitra, N.J., DiVerdi, S., Funkhouser, T.: Exploring collections of 3D models using fuzzy correspondences. In: SIGGRAPH (2012)Google Scholar
  27. 27.
    Kuhn, H.W.: The hungarian method for the assignment problem. Export. Naval Research Logistics Quarterly, 83–97 (1955)Google Scholar
  28. 28.
    Lee, J., Cho, M., Lee, K.M.: Hyper-graph matching via reweighted random walks. In: CVPR (2011)Google Scholar
  29. 29.
    Leordeanu, M., Hebert, M.: A spectral technique for correspondence problems using pairwise constraints. In: ICCV (2005)Google Scholar
  30. 30.
    Leordeanu, M., Hebert, M., Sukthankar, R.: Beyond local appearance: Category recognition from pairwise interactions of simple features. In: CVPR (2007)Google Scholar
  31. 31.
    Leordeanu, M., Herbert, M.: An integer projected fixed point method for graph matching and map inference. In: NIPS (2009)Google Scholar
  32. 32.
    Leordeanu, M., Sukthankar, R., Hebert, M.: Unsupervised learning for graph matching. Int. J. Comput. Vis., 28–45 (2012)Google Scholar
  33. 33.
    Leordeanu, M., Zanfir, A., Sminchisescu, C.: Semi-supervised learning and optimization for hypergraph matching. In: ICCV (2011)Google Scholar
  34. 34.
    Livi, L., Rizzi, A.: The graph matching problem. Pattern Anal. Applic., 253–283 (2013)Google Scholar
  35. 35.
    Loiola, E.M., de Abreu, N.M., Boaventura-Netto, P.O., Hahn, P., Querido, T.: A survey for the quadratic assignment problem. EJOR, 657–690 (2007)Google Scholar
  36. 36.
    Pachauri, D., Kondor, R., Vikas, S.: Solving the multi-way matching problem by permutation synchronization. In: NIPS (2013)Google Scholar
  37. 37.
    Pevzner, P.A.: Multiple alignment, communication cost, and graph matching. SIAM JAM (1992)Google Scholar
  38. 38.
    Qiu, H., Hancock, E.R.: Spectral simplification of graphs. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 114–126. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  39. 39.
    Shen, D., Hammer, C.D.: Hierarchical attribute matching mechanism for elastic registration. TMI (2002)Google Scholar
  40. 40.
    Sole-Ribalta, A., Serratosa, F.: Graduated assignment algorithm for multiple graph matching based on a common labeling. IJPRAI (2013)Google Scholar
  41. 41.
    Suh, Y., Cho, M., Lee, K.M.: Graph matching via sequential monte carlo. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 624–637. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  42. 42.
    Cour, T., Srinivasan, P., Shi, J.: Balanced graph matching. In: NIPS (2006)Google Scholar
  43. 43.
    Tian, Y., Yan, J., Zhang, H., Zhang, Y., Yang, X., Zha, H.: On the convergence of graph matching: Graduated assignment revisited. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 821–835. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  44. 44.
    Williams, M.L., Wilson, R.C., Hancock, E.: Multiple graph matching with bayesian inference. Pattern Recognition Letters, 1275–1281 (1997)Google Scholar
  45. 45.
    Wong, A., You, M.: Entropy and distance of random graphs with application to structural pattern recognition. IEEE Transactions on PAMI (1985)Google Scholar
  46. 46.
    Yan, J., Li, Y., Zheng, E., Liu, Y.: An accelerated human motion tracking system based on voxel reconstruction under complex environments. In: Zha, H., Taniguchi, R.-I., Maybank, S. (eds.) ACCV 2009, Part II. LNCS, vol. 5995, pp. 313–324. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  47. 47.
    Yan, J., Tian, Y., Zha, H., Yang, X., Zhang, Y.: Joint optimization for consistent multiple graph matching. In: ICCV (2013)Google Scholar
  48. 48.
    Zach, C., Klopschitz, M., Pollefeys, M.: Disambiguating visual relations using loop constraints, pp. 1246–1433 (2010)Google Scholar
  49. 49.
    Zaslavskiy, M., Bach, F.R., Vert, J.P.: A path following algorithm for the graph matching problem. PAMI (2009)Google Scholar
  50. 50.
    Zass, R., Shashua, A.: Probabilistic graph and hypergraph matching. In: CVPR (2008)Google Scholar
  51. 51.
    Zeng, Z., Chan, T.-H., Jia, K., Xu, D.: Finding correspondence from multiple images via sparse and low-rank decomposition. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 325–339. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  52. 52.
    Zhou, F., Torre, F.D.: Factorized graph matching. In: CVPR (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Junchi Yan
    • 1
    • 2
  • Yin Li
    • 3
  • Wei Liu
    • 4
  • Hongyuan Zha
    • 3
    • 5
  • Xiaokang Yang
    • 1
  • Stephen Mingyu Chu
    • 2
  1. 1.Shanghai Jiao Tong UniversityShanghaiChina
  2. 2.IBM Research - ChinaShanghaiChina
  3. 3.Georgia Institute of TechnologyAtlantaUSA
  4. 4.IBM T.J. Watson Research CenterNew YorkUSA
  5. 5.East China Normal UniversityShanghaiChina

Personalised recommendations