# Graph matching using conformal module

**Part of the following topical collections:**

## Abstract

Graph matching and classification play fundamental roles in computer vision. The computational complexity of the conventional method based on a spectrum method is high, which prevents it from handling large graphs in practice. This work proposes a novel framework for tackling the challenge by using conformal module. We apply the classical Hodge theory from differential manifold to the graph setting and compute the combinatorial conformal invariant of the graph, called as conformal module, which can be used as the fingerprint for the graph. The method is applicable for viewpoint classification and posture detection. The experimental results demonstrate the efficiency and efficacy of the proposed method.

## Keywords

Graph matching Graph classification Conformal module## Abbreviations

- GM
Graph matching

## 1 Introduction

Graph matching has been widely used to solve a large variety of problems in computer vision. Fundamentally, graph matching aims at finding correspondences between two sets of features extracted from images. When matching or recognizing structural objects, graph matching does well on considering the pairwise node interactions [1]. Furthermore, when there are correspondences between two images, graph matching can finish many vision tasks such as object categorization [2], feature tracking [3], symmetry analysis [4], and posture recognition [5, 6].

In the past thirty years, extensive research has been carried out on graphing matching [7], however, there still exists several challenges in current methods. For example, almost all matching algorithms require high time complexity, so it is difficult to deal with large graphs. No methods have been provided to deal with graphs in 3D vision field, which means that graphs embedded in high genus surfaces cannot be matched using current approaches. Many techniques are proposed to solve graph matching in terms of rigid transformation, which can hardly handle the circumstance where two graphs to be matched have large non-rigid transformations. Although the third issue is considered in [8], the other two issues are still open. Our main contributions conclude as follows: (1) the proposed method can discover the combinatorial invariant of a given graph, (2) the proposed method can tolerate non-rigid transformation on a given graph, (3) the proposed method can solve the matching problem in linear time, (4) the proposed method can be applied on not only planar graphs, but also those embedded in high genus surfaces.

The main mathematical theory in our method is based on combinatorial Hodge theory. There is some work on surface registration [9, 10] and shape analysis [11] by the combinatorial Hodge theory. In the textbook [12], Gu et al. introduced the detailed theoretic treatment on combinatorial Hodge theory. These works focus on computing the conformal invariants and conformal mappings induced by the Riemannian structure of the input surfaces. By contrast, current work focuses on computing the conformal modules induced solely by the combinatorial structure of the graphs. The process is that we apply combinatorial Hodge theory to compute the combinatorial harmonic functions on graphs. The solution of harmonic functions is equivalent to solving combinatorial Laplace matrices. The combinatorial harmonic 1-forms can give the combinatorial conformal invariant of a given graph, called as conformal module, which can be used as the fingerprint of a given graph. So the conformal module can be used to deal with some matching problems. In this paper, we compute the conformal modules by the combinatorial structure of the given graphs for viewpoint classification and posture detection.

## 2 Previous works

Graph matching problem has been an active field in computer science and mathematics. There is a large literature for this topic in the past thirty years, so a thorough survey is beyond the scope of the current work. In the following, we only review the most related work. For more detailed treatment, we refer readers to [7]. There usually exists two categories of graph matching methods: exact matching and inexact matching.

Exact matching aims at graph isomorphism, where the interest is in an exact matching between nodes and edges. Most of approaches to find the graph isomorphism are based on Tree Search. Due to the brute Tree Search algorithm, some improvements are developed. Ullmann proposes one of the most important and popular tree search algorithms [13]. He prunes the unfruitful matches which are not consistent with current partial matching. Cordella suggests another interesting approach based on Tree Search [14, 15]. He provides a heuristic based on analysis of the sets of nodes adjacent to the ones already considered in current partial matching. Except for Tree Search, some other techniques are proposed. According to group theory, an approach was proposed by McKay [16]. He defines equivalence classes for graphs in terms of isomorphism and does the equality verification in *O*(*n*^{2}). There are other methods based on machine learning techniques to speed up the matching against a large library of graphs [17, 18]. But they are used for filtering out the unmatched candidates not for matching.

Inexact matching focuses on mapping between graphs with weighted attributes on nodes and edges. The solution is a quadratic assignment problem and the optimization is NP-hard [19]. Looking for better optimization strategies is important for major research in graph matching. According to the relaxations of the constraints on a one-to-one mapping *F* between the nodes in both graphs [8], methods can be generally classified into several categories. The first is to approximate the constraints as an orthogonal one *F*^{T}*F* = *I*, which is called as spectral method. Under the orthogonal constraint, optimizing graph matching can be solved in closed form as an eigen value problem [20, 21, 22]. Furthermore, Leordeanu and Hebert relax *F* to be of unit length \( {\left\Vert F\right\Vert}_2^2=1 \) for handling more complex problems in computer vision [23]. The work in [24, 25] proposes a spectral method to decompose the graph into sub-graphs, where the process can be cast into a hierarchical framework and suitable for parallel computation. The second group of methods relaxes *F* to be a doubly stochastic matrix, the convex hull of *F*. Under this constraint, optimizing graph matching can be solved as a non-convex quadratic programming problem and a local optimization could be done within several proposed strategies. For instance, Gold and Rangarajan propose the graduated assignment algorithm for iteratively solving a series of linear approximations of the cost function in the form of Taylor expansion. The work of [26] proposes a path-following algorithm. Besides the optimization-based work, probabilistic frameworks are also useful for interpreting and solving graph matching [27]. Several work aims at higher-order tensor factorization [8, 28, 29], where higher order geometrical relations make graph matching invariant to rigid transformation. Due to a small increment in the order of relations induces a combination explosion of data, it is only suitable for very sparse graphs. Furthermore, this method cannot deal with non-rigid transformations.

## 3 Method

This section briefly introduces the theoretical background for combinatorial conformal module. We apply algebraic topology method to consider our matching problem. Algebraic topology focuses on studying topological problems by algebraic methods and uses simple linear algebraic methods to solve these problems. The detailed description of algebraic topology methodology can be seen in [12], where the authors consider these methods on surfaces with Riemann structure. In this paper, we apply algebraic topology method to the graph setting and consider combinatorial Hodge theory on graphs. Combinatorial Hodge theory can deduce a unique combinatorial conformal module for each graph, so it can be viewed as the fingerprint of a given graph.

We consider all objects on a three-connected planar graph *G* which is embedded on a topological surface *S*. A planar graph means that there are no edge crossings on *G* and different edges of the graph can only intersect at the endpoints. So each face of *G* can be viewed as a topological disk. We usually use a triple *G* = (*V*, *E*, *F*) to represent a graph, where *V* is the vertex set, *E* is the edge set and *F* is the face set. At the first step, we need to introduce some tools about computational topology. These similar concepts are defined on surfaces with Riemann structure in [12]. Here we use them on graphs.

*e*is denoted as

where *v*_{i}, *v*_{j} are its two endpoints.

*f*is represented as

where \( {v}_{i_k} \) is its vertex and (*i*_{1}, *i*_{2}, ⋯, *i*_{k}) is any cyclic permutation of (1, 2, ⋯, *k*).

We call the vertices of *G* as 0-dimensional cells, the oriented edges of *G* as 1-dimensional cells and the faces of *G* as 2-dimensional cells. Their linear combinations can form a linear chain space. We will give some definitions on the chain space of a graph.

*k*-dimensional chain group of a graph

*G*= (

*V*,

*E*,

*F*) is given by

where \( {\varepsilon}_k^i \) denotes the *i*th *k*-dimensional cell of *G*_{.} We call each element of *C*_{k}(*G*, *Z*) as a *k*-chain of *G*.

**Definition 1**: The

*k*-dimensional boundary operator

*∂*

_{k}:

*C*

_{k}(

*G*, ℤ) →

*C*

_{k − 1}(

*G*, ℤ) on graph

*G*is defined as

*k*-chain in the kernel of

*∂*

_{k}satisfies

We call each element in *Ker ∂*_{k} as a closed chain. The closed chains are closed loops and the boundaries of any 2-dimensional patches are called as boundary loops.

*k*-chain in the image of

*∂*

_{k + 1}satisfies

Each element of *Img ∂*_{k + 1} is called as an exact chain.

**Property 1**: The boundary of a boundary is empty:

The proof can be seen in [12].

From the above property, we can easily obtain that *Img ∂*_{k + 1} is a sub-linear space of *Ker ∂*_{k}. The complementary space is called as the homology group of the graph.

**Definition 2**: Suppose

*∂*

_{k}:

*C*

_{k}(

*G*, ℤ) →

*C*

_{k − 1}(

*G*, ℤ) is the

*k*-dimensional boundary operator on a graph

*G*.

*Ker ∂*

_{k}= {

*ε*

_{k}∈

*C*

_{k}(

*G*, ℤ)|

*∂*

_{k}

*ε*

_{k}= 0} is the kernel of

*∂*

_{k}.

*Img ∂*

_{k + 1}= {

*ε*

_{k}∈

*C*

_{k}| ∃

*ε*

_{k + 1}∈

*C*

_{k + 1},

*ε*

_{k}=

*∂*

_{k + 1}

*ε*

_{k + 1}} is the image of

*∂*

_{k + 1}. The

*k*-dimensional homology group of the graph

*G*is defined as the quotient group, satisfying

*H*

_{k}(

*G*,

*Z*) is called a

*k*-dimensional homologous class. A closed

*k*-chain

*ε*

_{k}∈

*ker ∂*

_{k}represents a homologous class, denoted as [

*ε*

_{k}] ∈

*H*

_{k}(

*G*,

*Z*). Each element in the homologous class can form the basis of homology group. The similar algorithm in [12] is used to compute the homology basis of a discrete surface. Figures 1 and 2 show the homology basis of two graphs with different topology.

In order to do computation on graphs, we define the cohomology group which is the dual of the homology group. In fact, cohomology group is the space of linear functions defined on homology group. The following discussions are considered in parallel because all the objects are dual of the above counterparts such as co-chain and exterior differential operator.

*k*-dimensional co-chain space of the graph

*G*is a linear function defined on

*k*-dimensional chain space and given by

We call each element in *C*^{k}(*G*, *R*) as a *k*-form.

The dual of the boundary operator *∂* is called as the exterior differential operator *d*.

**Definition 3**: The exterior differential operator

*d*

_{k}:

*C*

^{k}(

*G*, ℝ) →

*C*

^{k + 1}(

*G*, ℝ) is defined as follows:

where *ω*^{k} ∈ *C*^{k}(*G*, *R*), *ε*_{k + 1} ∈ *C*_{k + 1}(*G*, *Z*).

*k*-form in the kernel of

*d*

_{k}satisfies

We call each element in *Ker d*_{k} as a closed *k*-form.

*k*-form in the image of

*d*

_{k}satisfies

We call each element in *Ker d*_{k} as an exact *k*-form.

From **Property 1**, we obviously obtain the following property.

**Property 2:**The exterior differential operator

*d*

_{k}:

*C*

^{k}(

*G*, ℝ) →

*C*

^{k + 1}(

*G*, ℝ) is a linear operator with the property

According to property 2, we have that exact forms are closed. Therefore, *Img d*_{k − 1} is a sub-linear space of *Ker d*_{k}. The complementary space is defined as the cohomology group of the graph.

**Definition 4**: Suppose the linear operator

*d*

_{k}:

*C*

^{k}(

*G*, ℝ) →

*C*

^{k + 1}(

*G*, ℝ) is the exterior differential operator on a graph

*G*.

*Ker d*

_{k}= {

*ω*

^{k}∈

*C*

^{k}(

*G*, ℝ)|

*d*

_{k}

*ω*

^{k}= 0} is the kernel of

*d*

_{k}.

*Img d*

_{k − 1}= {

*ω*

^{k}∈

*C*

^{k}(

*G*,

*R*)| ∃

*ω*

^{k − 1}∈

*C*

^{k − 1}(

*G*,

*R*),

*ω*

^{k}=

*d*

_{k − 1}

*ω*

^{k − 1}} is the image of

*d*

_{k − 1}. The

*k*-dimensional cohomology group of the graph

*G*is defined as the quotient group, satisfying

A graph *G* embedded on a topological surface *S* can induce a CW-cell decomposition. The Poincare dual of *G* is denoted as \( \overline{G} \) which is also a graph embedded on *S* such that each vertex *v*_{i} of *G* corresponds to a face \( {\overline{v}}_i \) of \( \overline{G} \) and each face *f*_{j} of *G* corresponds to a vertex \( {\overline{f}}_j \) of \( \overline{G} \). Furthermore, each oriented edge *e* of *G* corresponds to an oriented edge \( \overline{e} \) on \( \overline{G} \) as follows: if *e* = [*v*_{i}, *v*_{j}], \( \overline{e}={\overline{v}}_i\cap {\overline{v}}_j \). Furthermore, if the left face of *e* is *f*_{l} and the right face of *e* is *f*_{r}, \( \overline{e}=\left[{\overline{f}}_r,{\overline{f}}_l\right] \).

**Definition 5**: Suppose a graph and its dual are denoted as

*G*and \( \overline{G} \) respectively. The Hodge star operator \( {}^{\ast }:{C}^k\left(G,\mathrm{\mathbb{R}}\right)\to {C}^{2-k}\left(\overline{G},\mathrm{\mathbb{R}}\right) \) is a linear operator, satisfying

where *ω* ∈ *C*^{k}(*G*, *R*), *ε* ∈ *C*_{k}(*G*, *Z*), \( \overline{\varepsilon}\in {C}_{2-k}\left(\overline{G},Z\right) \), and \( {}^{\ast}\omega \in {C}^{2-k}\left(\overline{G},R\right) \).

*σ*

_{k}:

*C*

^{k}(

*G*,

*R*) →

*C*

^{k − 1}(

*G*,

*R*), satisfying

The operator *σ*_{k} can be viewed as the adjoint of the exterior differential operator, called as the co-differential operator. Combining operator *d*_{k} and *σ*_{k}, we can define the Laplace operator.

**Definition 6**: Suppose

*d*

_{k}:

*C*

^{k}(

*G*, ℝ) →

*C*

^{k + 1}(

*G*, ℝ) is the exterior differential operator on a graph \( G.{}^{\ast }:{C}^k\left(G,\mathrm{\mathbb{R}}\right)\to {C}^{2-k}\left(\overline{G},\mathrm{\mathbb{R}}\right) \) is the Hodge star operator on

*G*.

*σ*

_{k}=

^{∗}

*d*

_{x}

^{∗}:

*C*

^{k}(

*G*,

*R*) →

*C*

^{k − 1}(

*G*,

*R*) is the co-differential operator on

*G*. The Laplace operator

*Δ*:

*C*

^{k}(

*G*,

*R*) →

*C*

^{k}(

*G*,

*R*) is defined as

*ξ*,

*η*∈

*C*

^{k}(

*G*,

*R*),

We mainly consider functions whose values are equal to zero with the action of the Laplace operator. These functions have inspiring mathematical and physical properties. In this paper, we will compute these functions on the graph setting. Based on the combinatorial structure of a graph, the solution of harmonic functions is equivalent to solving combinatorial Laplace matrices on the graph. Finally, these combinatorial harmonic functions can give out the combinatorial conformal invariant of a given graph, called as conformal module, which can be used as the fingerprint of a given graph. So, the conformal module can be applied into matching applications on graphs.

**Definition 7**: Suppose

*d*

_{k}:

*C*

^{k}(

*G*, ℝ) →

*C*

^{k + 1}(

*G*, ℝ) is the exterior differential operator on a graph

*G*.

*σ*

_{k}=

^{∗}

*d*

_{x}

^{∗}:

*C*

^{k}(

*G*,

*R*) →

*C*

^{k − 1}(

*G*,

*R*) is the co-differential operator on

*G*.

*Δ*=

*σ*

_{k + 1}

*d*

_{k}+

*d*

_{k − 1}

*σ*

_{k}:

*C*

^{k}(

*G*,

*R*) →

*C*

^{k}(

*G*,

*R*) is the Laplace operator on

*G*. A

*k*-form

*ω*∈

*C*

^{k}(

*G*, ℝ) is called to be harmonic, if

Since (*Δω*, *ω*) = |*d*_{k}*ω*|^{2} + |*σ*_{k}*ω*|^{2} = 0, we can easily have the following property.

**Property 3:**A harmonic

*k*-form

*ω*∈

*C*

^{k}(

*G*, ℝ) satisfies

From the physical viewpoint, the exterior differential operator *d*_{k} is the curl operator and the co-differential operator *σ*_{k} is the divergence operator. So a harmonic *k*-form means that it is both curl-free and divergence-free.

*k*-forms is denoted as \( {H}_{\Delta}^k\left(G,\mathrm{\mathbb{R}}\right) \), namely

According to Hodge theory, any a *k*-form can be decomposed into an exact form, a co-exact form and a harmonic form.

**Theorem 1**: Suppose

*ω*is a

*k*-form.

*d*

_{k − 1}:

*C*

^{k − 1}(

*G*, ℝ) →

*C*

^{k}(

*G*, ℝ) is the exterior differential operator on a graph

*G*.

*σ*

_{k}=

^{∗}

*d*

_{x}

^{∗}:

*C*

^{k}(

*G*,

*R*) →

*C*

^{k − 1}(

*G*,

*R*) is the co-differential operator on

*G*. Then there is a unique (

*k*− 1)-form

*α*, (

*k*+ 1)-form

*β*and a harmonic

*k*-form

*γ*, such that

moreover, such kind of decomposition is unique.

The detailed proof can be checked in [12].

*k*= 2, suppose

*G*is embedded on a closed surface with genus

*g*.{

*γ*

_{1},

*γ*

_{2}, ⋯,

*γ*

_{2g}} is the generator of 1-dimensional homology group

*H*

_{1}(

*G*,

*Z*). Let

*ω*

_{1},

*ω*

_{2}be two harmonic 1-forms with respect to the homology equivalence class, namely

So *ω*_{1} − *ω*_{2} is harmonic and \( {\int}_{\gamma_k}\left({\omega}_1-{\omega}_2\right)={\int}_{\gamma_k}{\omega}_1-{\int}_{\gamma_k}{\omega}_2=0.\left({\omega}_1-{\omega}_2\right) \) is exact. There is a function *f* : *G* → *R* such that *ω*_{1} − *ω*_{2} = *df*. According to the maximum principle and the empty boundary, *f* has no extremal value and is constant. That means the uniqueness holds: *ω*_{1} − *ω*_{2} = *df* = 0.

*ω*∈

*H*

^{1}(

*G*,

*R*) is closed and

*f*:

*G*→

*R*is a smooth function. So

*ω*+

*df*and

*ω*lie in the same cohomology class. The solution of

*σ*(

*ω*+

*df*) = 0 is equivalent to the solution of Poisson equation:

In order to compute harmonic functions on a graph, we need define the combinatorial Laplace matrix.

**Definition 8**: Suppose

*G*= {

*v*

_{1},

*v*

_{2}, ⋯,

*v*

_{n}} is a planar graph. The combinatorial Laplace matrix Δ = (

*δ*

_{ij}) on

*G*is defined as

where *v*_{i}~*v*_{j} means two vertices are adjacent and *d*(*v*_{i}) is the degree of *v*_{i} in the graph. According to theorem 1, the following Corollary is straightforward.

**Corollary:**The group of all harmonic

*k*-forms on

*G*is isomorphic to the

*k*-dimensional cohomology group, that is

*ω*

_{1}and

*ω*

_{2}are cohomological

*k*-forms, then there is a (

*k*− 1)-form

*η*, such that

*ω*

_{1}=

*ω*

_{2}+

*dη*. Assume the Hodge decomposition of

*ω*

_{1}is

*dω*

_{1}= 0,

*dδβ*

_{1}= 0, so

*δβ*

_{1}= 0. Directly, we get

That means two cohomological forms share the same harmonic form. Therefore, each cohomological class in *H*^{k}(*G*, *R*) corresponds to a unique harmonic form in \( {H}_{\Delta}^k\left(G,R\right) \), which establishes the isomorphism.

**Combinatorial Conformal Invariants:**Suppose a graph

*G*is embedded on a genus

*g*surface

*S*.

*ω*is a combinatorial harmonic 1-form on

*G*and

*γ*is a closed loop. We can choose a tubular neighborhood of

*γ*,

We can periodically embed *N*(*γ*) to the plane. Each period is represented as a complex number, denoted as ∫_{γ}*ω*.

*H*

_{1}(

*G*,

*Z*) as {

*a*

_{1},

*b*

_{1},

*a*

_{2},

*b*

_{2}, ⋯,

*a*

_{g},

*b*

_{g}}. Namely, they satisfy the following conditions

*a*⋅

*b*represents the algebraic intersection number. Suppose a closed differential 1-form is

*ω*. The integral of

*ω*on

*a*

_{i}

*a*-period of

*ω*. Similarly,

is called the *b*-period of *ω*.

*a*

_{i}, slicing the surface

*S*to get

*S*

_{i}, the boundary of

*S*

_{i}is \( \partial {S}_i={a}_i^{+}-{a}_i^{-} \). We construct a function

*f*

_{i}such that

and denote *α*_{i} = *df*_{i}.

*b*

_{j}, slicing the surface

*S*to get

*S*

_{j}, the boundary of

*S*

_{j}is \( \partial {S}_j={b}_j^{+}-{b}_j^{-} \). We construct a function

*g*

_{j}such that

and denote *β*_{j} = *dg*_{j}.

*G*is as following:

For genus one case, the graph *G* can be embedded into the plane as a parallelogram. The combinatorial period matrix *Mod*(*G*) represents the shape of the parallelogram. For high genus graph *G*, the combinatorial period matrix is the generalization. The combinatorial period matrix can be used as the fingerprint of the embedded graph *G*.

*f*

_{0}on the graph

*G*is selected as the unique exterior face. Four vertices on the boundary of

*f*

_{0}are chosen, denoted as {

*v*

_{0},

*v*

_{1},

*v*

_{2},

*v*

_{3}}, sorted counter clockwisely. The graph

*G*with four corner vertices is called as a topological quadrilateral. The corner vertices divide the boundary of the exterior face into four segments, denoted as left, top, right, and bottom segments, such that

*v*

_{0}is the left lower corner. Then we can compute a harmonic function

*u*:

*G*→ ℝ with Dirichlet boundary condition, namely

In fact, the harmonic 1-form *ω* is the exterior differential of *u*, *ω* = *du*.

**Theorem 2**: Suppose

*G*is a topological quadrilateral which is a planar graph with an exterior face

*f*

_{0}and four corner vertices {

*v*

_{0},

*v*

_{1},

*v*

_{2},

*v*

_{3}}. The harmonic 1-form maps the topological quadrilateral to a rectangle with width

*w*and height

*h*, then the conformal module of the topological quadrilateral is

This theorem can be proven by an extremal length theory. More detailed proof is shown in [30].

*ϕ*which maps a topological quadrilateral into a canonical rectangle and maps four vertices {

*v*

_{0},

*v*

_{1},

*v*

_{2},

*v*

_{3}} into four corner vertices of the rectangle. The ratio of width and height is the conformal module. Four vertices {

*v*

_{0},

*v*

_{1},

*v*

_{2},

*v*

_{3}} divide the boundary of the topological quadrilateral into four segments, denoted as Γ

_{0}, Γ

_{1}, Γ

_{2}, Γ

_{3}and

*∂*Γ

_{k}=

*v*

_{k + 1}−

*v*

_{k}. Solving two Dirichlet equations, there are two harmonic functions

*f*

_{1},

*f*

_{2}:

*G*→

*R*which satisfy the Dirichlet boundary and the Neumann boundary:

*n*is the normal vector. The gradient fields ∇

*f*

_{1}and ∇

*f*

_{2}are mutually vertical lines but are not conjugate to each other. A constant

*λ*is necessary for the conformal mapping

*ϕ*with \( \phi =\lambda {f}_2+\sqrt{-1}{f}_1 \). So

*λ*is the width of the canonical rectangle. Due to the harmonic function with conformal invariant property, there is a mapping energy

## 4 Experimental results and discussions

All experiments are conducted on a Windows 7 platform, with a single 2.3-GHz Intel CPU, 16-GB RAM memory. All algorithms have been developed in generic C++, compiled using Visual Studio 2010. The sparse linear systems are solved using the numerical library Eigen [31].

The experimental work is to apply the combinatorial Hodge theory to compute the harmonic functions on the graph setting and use these harmonic functions to get the combinatorial conformal module of a given graph. The combinatorial conformal module can be viewed as the fingerprint of a graph. For each image, we make a feature graph for it. Based on the combinatorial conformal modules of graphs, matching work can be done by finding correspondences between two sets of features extracted from images. We test the proposed method in the following applications.

### 4.1 Viewpoint detection

### 4.2 Posture detection

*XOY*,

*YOZ*and

*ZOX*plane respectively and calculate the corresponding Delaunay triangulation which is showed in the 3D posture scatter figures. We project the 3D posture onto the plane to find the most left vertex, the most bottom vertex, the most right vertex and the most top vertex as the corners. The proposed method is applied to compute the combinatorial conformal module of a posture graph. The combinatorial conformal module can be used to distinguish these postures. We use the proposed method to process the whole video sequence with 2751 frames and use K-nearest neighbor method to classify the postures. The correct rate of posture classification result is nearly 90%.

## 5 Conclusion

Graph matching has been intensively applied in computer vision for finding correspondences between two sets of features extracted from images. Most of the existing graph matching methods require high time complexity and are not usable to large graphs. Furthermore, these methods can hardly handle the circumstances where two graphs to be matched have large non-rigid transformations. In this work, we propose a novel graph matching and classification method by the combinatorial conformal module.

We generalize the classical Hodge theory to the combinatorial graph setting. The process is that we apply combinatorial Hodge theory to compute the combinatorial harmonic functions on graphs. By defining combinatorial Laplace matrices on graphs, the solution of harmonic functions is converted into solving linear Laplace matrices. The combinatorial harmonic 1-forms can give the combinatorial conformal invariant of a given graph, called as conformal module. According to the mathematical result, the conformal module can be used as the fingerprint of a given graph. Thus we can use the conformal module to do some matching work. Comparing to other graph matching methods, the proposed method can discover the combinatorial invariant of a given graph. Furthermore, the proposed method can tolerate the non-rigid transformation on a given graph. Due to considering the proposed method from the algebraic topology viewpoint, the computation of matching problem can be done in linear time. More importantly, our proposed method is adaptable to not only planar graphs but also graphs embedded in high genus surfaces.

In the future work, we purpose to explore the inexact graph matching method based on the conformal module and we hope the proposed method can be used to more applications in computer vision.

## Notes

### Acknowledgements

The authors thank the editor and anonymous reviewers for their helpful comments and valuable suggestions.

### Funding

Not applicable.

### Availability of data and materials

Please contact author for data requests.

### Authors’ contributions

All authors take part in the discussion of the work described in this paper. These authors contributed equally to this work. Both authors read and approved the final manuscript.

### Authors’ information

Jialing Zhang was born in Yunnan, P.R. China, in 1981. She received her Master of Science degree from Yunnan Normal University in 2006. She has been a faculty in Kunming University of Science and Technology since 2006. Her main research interests focus on Computational Geometry-Theory and Applications.

Kun Qian was born in Yunnan, P.R. China, in 1982. He received his M.S. degree in Mechanical and Electronic Engineering from Xidian University in China in 2009. He was a visiting scholar at the State University of New York at Stony Brook and University of Louisiana at Lafayette, US from January 2015 to January 2016. He is currently pursuing his Ph.D. degree in Kunming University of Science and Technology in China. His research interests are in the area of computational geometry, computer graphics and computer vision.

### Competing interests

The authors declare that they have no competing interests.

### Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## References

- 1.F. Zhou, F. De la Torre, Factorized graph matching. IEEE Conference on Computer Vision and Pattern Recognition.
**38**, 127–134 (2012)Google Scholar - 2.O. Duchenne, A. Joulin, J. Ponce, A graph-matching kernel for object categorization. International Conference on Computer Vision (2011), pp. 1792–1799Google Scholar
- 3.H. Jiang, S. X. Yu, D. R. Martin, Linear scale and rotation invariant matching. IEEE Trans. Pattern Anal. Mach. Intell.
**33**(7), 1339–1355 (2011)Google Scholar - 4.J. Hays, M. Leordeanu, A. A. Efros, Y. Liu, Discovering texture regularity as a higher-order correspondence problem. European Conference on Computer Vision. (Springer, Berlin, Heidelberg, 2006), pp. 522–535.Google Scholar
- 5.U. Gaur, Y. Zhu, B. Song, A. Roychowdhury, A "string of feature graphs" model for recognition of complex activities in natural videos. IEEE International Conference on Computer Vision (ICCV), (IEEE, 2011), pp. 2595-2602Google Scholar
- 6.W. Brendel, S. Todorovic, Learning spatiotemporal graphs of human activities. IEEE International Conference on Computer Vision (2011), pp. 778–785Google Scholar
- 7.D. Conte, P. Foggia, C. Sansone, M. Vento, Thirty years of graph matching in pattern recognition. Int. J. Pattern Recognit. Artif. Intell.
**18**(03), 265–298 (2004)CrossRefGoogle Scholar - 8.F. Zhou, F. De la Torre, Deformable graph matching. IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 2922–2929Google Scholar
- 9.W. Zeng, J. Marino, K. Chaitanya Gurijala, X. Gu, A. Kaufman, Supine and prone colon registration using quasi-conformal mapping. IEEE Trans. Vis. Comput. Graph.
**16**(6), 1348–1357 (2010)CrossRefGoogle Scholar - 10.W. Zeng, Y. Zeng, Y. Wang, X. Yin, X. Gu, D. Samaras, 3d non-rigid surface matching and registration based on holomorphic differentials. European Conference on Computer Vision (Springer, Marseille, France, October 12–18, 2008), pp. 1–14Google Scholar
- 11.L. Lui, W. Zeng, S. Yau, X. Gu, Shape analysis of planar multiplyconnected objects using conformal welding. IEEE Trans. Pattern Anal. Mach. Intell.
**36**(7), 1384–1401 (2014)CrossRefGoogle Scholar - 12.X. D. Gu, S. T. Yau, Computational conformal geometry. (Somerville, Mass, USA: International Press, 2008)Google Scholar
- 13.J. R. Ullmann, An algorithm for subgraph isomorphism. J. ACM
**23**(1), 31–42 (1976)Google Scholar - 14.L. P. Cordella, P. Foggia, C. Sansone, F. Tortorella, M. Vento, Graph matching: a fast algorithm and its evaluation. International Conference on Pattern Recognition.
**2**, 1582–1584 (1998)Google Scholar - 15.L. P. Cordella, P. Foggia, C. Sansone, M. Vento, Fast graph matching for detecting cad image components. 15th International Conference on Pattern Recognition.
**2**, 1034–1037 (2000)Google Scholar - 16.B. D. McKay, Practical graph isomorphism[J]. J. Symb. Comput.
**60**(1), 94–112 (2014)Google Scholar - 17.C. Irniger, H. Bunke, Graph matching: filtering large databases of graphs using decision trees[C]. Graph-Based Representations in Pattern Recognition (2001), pp. 239-249Google Scholar
- 18.M. Lazarescu, H. Bunke, S. Venkatesh, Graph matching: fast candidate elimination using machine learning techniques. Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR). (Springer, Berlin, Heidelberg, 2000), pp. 236-245Google Scholar
- 19.E. M. Loiola, N. M. M. de Abreu, P.O. Boaventura-Netto, P. Hahn, T. Querido, A survey for the quadratic assignment problem. Eur. J. Oper. Res.
**176**(2), 657–690 (2007)Google Scholar - 20.S. Umeyama, An eigendecomposition approach to weighted graph matching problems. IEEE Trans. Pattern Anal. Mach. Intell.
**10**(5), 695–703 (1988)CrossRefGoogle Scholar - 21.L. S. Shapiro, J. Michael Brady, Feature-based correspondence: an eigenvector approach. Image Vis. Comput.
**10**(5), 283–288 (1992)Google Scholar - 22.T. Caelli, S. Kosinov, An eigenspace projection clustering method for inexact graph matching. IEEE Trans. Pattern Anal. Mach. Intell.
**26**(4), 515–519 (2004)CrossRefGoogle Scholar - 23.M. Leordeanu, M. Hebert, A spectral technique for correspondence problems using pairwise constraints. Tenth IEEE International Conference on Computer Vision.
**2**, 1482–1489 (2005)Google Scholar - 24.H. Qiu, E. R. Hancock, Graph matching and clustering using spectral partitions. Pattern Recogn.
**39**(1), 22–34 (2006)Google Scholar - 25.S. Gold, A. Rangarajan, A graduated assignment algorithm for graph matching[J]. IEEE Trans. Pattern Anal. Mach. Intell.
**18**(4), 377–388 (1996)CrossRefGoogle Scholar - 26.M. Zaslavskiy, F. Bach, J. P. Vert, A path following algorithm for the graph matching problem. IEEE Trans. Pattern Anal. Mach. Intell.
**31**(12), 2227–2242 (2009)Google Scholar - 27.R. Zass, A. Shashua, Probabilistic graph and hypergraph matching. IEEE Conference on Computer Vision and Pattern Recognition(CVPR), (IEEE, 2008), pp. 1–8Google Scholar
- 28.M. Chertok, Y. Keller, Efficient high order matching. IEEE Transactions on Pattern Analysis and Machine Intelligence.
**32**(12), 2205-2215(2010)Google Scholar - 29.O. Duchenne, F. Bach, I. S. Kweon, J. Ponse, A tensor-based algorithm for high-order graph matching. IEEE transactions on pattern analysis and machine intelligence.
**33**(12), 2383-2395(2011)Google Scholar - 30.O. Schramm, Square tilings with prescribed combinatorics[J]. Israel Journal of Mathematics.
**84**(1–2), 97–118 (1993)MathSciNetCrossRefGoogle Scholar - 31.G. Guennebaud, B. Jacob, et al., Eigen v3. http://eigen.tuxfamily.org/index.php?title=Main_Page (2010)

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.