Coarse geometric kernels for networks embedding

We develop embedding kernels based on the Forman–Ricci curvature and intertwined Bochner–Laplacian and employ them for the detection of the coarse structure of networks, as well as for network visualization with applications to support-vector machines (SVMs).


Introduction
A common theme in Geometry and in many of its applications (in some cases still in a non-deliberate manner) is the embedding, with least distortion, of a given structure/space, in a familiar, larger ambient space. For instance, in classical Geometry and Topology, this ambient space is usually R n , for some n large enough, whereas any metric space is isometrically embeddable in l ∞ . Also, more recently, embeddings of networks in the Hyperbolic Plane H 2 or Space H 3 have become quite common (see, e.g. [1] and the references therein).
People study and make appeal to the coarse geometry of networks, constantly and for a long time now, whether they do it conscientiously (rarely) or not (in most cases). We shall demonstrate this shortly, however, to make this even clear, let us specify what we, informally, mean by "Coarse Geometry": The study of the geometric (topological) properties, without "looking at" the small scales. In other words, one does not discern between objects that, viewed from sufficiently far away, look the same. Perhaps the simplest and immediate example of such a large scale geometry (or, in John Lott's suggestive words, "Mr. Magoo geometry") is given by the integer grid in the Euclidean plane. (For more insights and technical definitions and results, see, e.g [2][3][4], as well the Appendix.) The Complex Networks community has been exposed to this approach-and makes by now use of it-via the notion of Gromov hyperbolicity, itself stemming from Gromov's seminal work on hyperbolic groups [5]. The-perhaps somewhat theoretical in the eyes of many practitioners-notion above is connected to the notion of network backbone, underlying the long-distance relations between major network regions. Indeed, the connection between negative curvature and these "communication highways" in networks has been emphasized in [6].
It is therefore most natural to ask, not only if a coarse embedding (viewed as a metric measure space) of a weighted graph exists, but also whether there exists an "automatic" procedure to achieve such an embedding. The present paper shows that the answer to both these questions is positive.
The reminder of the paper is structured as follows: in Sect. 2, we bring the definitions of Forman's Bochner-Laplacians and Curvature functions on which our embedding kernels rest upon; in Sect. 3, we discuss coarse embeddings and their use to support-vector machines (SVMs), while in Sect. 4, we apply our kernels to networks' visualization, concluding in Sect. 5, with an overview of the paper and an outlook. For the benefit of the readers we also added an appendix including a very brief overview of coarse spaces.

The Forman-Laplacian and curvature functions
We bring below the general formulas for the Forman-Laplacian and curvature functions. For more background on C W complexes we refer the reader to Forman's paper [8] as well as to [7].
Let M be a (positively) weighted quasiconvex regular C W complex, let α = α p ∈ M a p-dimensional cell and let w(α) denote its weight. While general weights are possible, making the combinatorial Ricci curvature extremely versatile, it is suffices (cf. [8]), Theorem 2.5 and Theorem 3.9) to restrict oneself only to so called standard weights:

Definition 1
The set of weights {w α } is called a standard set of weights iff there exist w 1 , w 2 > 0 such that given a p-cell α p , the following holds: (Note that the combinatorial weights w α ≡ 1 represent a set of standard weights, with w 1 = w 2 = 1.) Using standard weights, we obtain the following formula for the Forman-Laplacian where, for instance α < β denotes that α is a face of β, and α 1 β , α 2 β , γ α 1 , γ α 2 ∈ {−1, +1} represent the respective incidence numbers of the cells β relative to α i , and α i relative to γ , i = 1, 2, respectively. (see [7]).
The case p = 1 is especially important, since in the classical, i.e. smooth manifold case, F 1 coincides with Ricci curvature. Therefore, by analogy with the classical case, one defines the discrete (weighted) Forman-Ricci curvature on α = α 1 , i.e. 1-cells (edges), as More generally, the operators p and F p are interrelated via the defining discrete Bochner-Weitzenböck formula-see [8], namely where (For further details see [8].) In the dimensionally degenerate-yet essential in practice and applications-1dimensional case, that is of graphs/networks, the daunting Formula (3) reduces to the following elementary one: where • e denotes the edge under consideration between two nodes v 1 and v 2 ; • w(e) denotes the weight of the edge e under consideration; Here we denote by F the Graph Forman-Ricci curvature for networks. (Notice the change of notation adopted for clarity reasons.) We do not elaborate here more on the intuition behind these formulas, especially in the case of 1-dimensional complexes, which is the most important case herein, since we dwelt on them in detail in our original papers [9,10], as well in the quite recent article [11], which represents a companion paper of the present one.

Coarse embedding and an application to SVM
By a coarse embedding of a metric space (X , d) into another metric space (Y , ρ), we mean a map i : X → Y , such that there exist increasing, unbounded functions η 1 , η 2 : R → R, such that for any x 1 , x 2 ∈ X . The "automatic" embedding method we alluded to above is one that is canonical in SVM techniques, and classical in Fourier Analysis and its various applications, in particular in Signal and Image Processing, namely that of reproducing kernels. Recall the following definition: Definition 3 Given a set X , symmetric kernel is a symmetric function k : X × X → R, i.e. k(x, y) = k(y, x), for any x, y ∈ X .
A kernel k is said to have Positive and negative type kernels are interrelated by the following classical result: Proposition 1 (Schoenberg's Lemma) Let k be a symmetric kernel on set X . Then the following statements are equivalent: 1. The kernel k is of negative type; 2. The kernel κ = exp(−tk) is of positive type, for each t > 0.
(For a proof see, e.g. [3], Proposition 11.12. See also [13].) From Schoenberg's Lemma and the fact that k a can be written as where C = C a is a positive constant, it follows (cf. [3]) that following corollary holds: Corollary 2 Let k ≥ 0 be a kernel of negative type on X . Then κ = κ a is also of negative type, for any 0 < a < 1.
(See, e.g. [3], Example 11.13.) Remark 1 An (effectively coarse) embedding method for networks as well as higher dimensional spaces was proposed in [12]. While the approach suggested therein applies to more general spaces than networks and, the embedding space is the familiar R n , it still is less than intuitive, since it makes appeal to a family of quasi-metrics. While this approach allows for the network to be studied at many scales, it is also more complicated then the one adopted here (and based on the path degree metric). Furthermore, the method introduced herein has also the advantage of coming in conjunction with a reproducing kernel, that allows for the automation required in SVM related applications. Therefore, the corollary above shows that a way of studying networks at many scales, akin to the one in [12], exists also in the coarse embedding (kernel) approach.
Since the operators 1 and B 1 are symmetric as functions of the end nodes u, v of an edge e = (u, v), they define (in analogy to the classical Laplacian) reproducing kernels k = 1 , k B = B 1 on any given network. Moreover, since by its very construction/definition B p is positive semidefinite, it follows that k p is also positive semidefinite (i.e. a "classical" reproducing kernel). By a direct application of a classical result (for a proof, see, e.g. [3], Theorem 11.15), we get the following theorem: Theorem 3 Let k be a symmetric kernel on X . Then,

If k is of positive type, then there exists a map Φ : X → H, where H is a real
Hilbert space, such that k(x, y) =< Φ(x), Φ(y) >, for any x, y ∈ X. 2. If k is of negative type and, furthermore, k(x, x) = 0 for any x ∈ X , then there exists a map Φ : X → H, where H is a real Hilbert space, such that k(x, y) =|| Φ(x) − Φ(y) || 2 , for any x, y ∈ X.
Moreover, F and Ric F are both symmetric functions (again viewed as acting on pairs of nodes defining edges), thus kernels k F and k Ric F can be defined. However, neither version of Forman's Ricci curvature is positive, therefore a mapping into a real Hilbert space, as for the Laplacians, is not possible for these curvature-based kernels. In fact, F is not negative only if both its end nodes have degree 2, that is only the degenerate case of cycles and graph "spurious" vertices of degree 2, i.e graph subdivisions homeomorphic to a "good" graph can have edges of non-negative curvature. It can, therefore, be surmised that proper networks have pure Forman curvature F < 0. (Note that this does not hold for Ric F .) Unfortunately, part 2 of the theorem above (the one regarding negative type operators) does not hold, since it requires that k(x, x) = 0, for all x ∈ X , which, of course, does not hold neither for 1 , nor for B 1 . However, one can still map the kernels k , k B to a Hilbert space precisely as above, after modifying them into fitting related positive operators, by putting k * = e −k ; k * B = e −k B . 1 Moreover (and perhaps more important in our context) there exists a coarse embedding of any (finite) network into a (real) Hilbert space. This follows from the fact that, for instance, e −k F is a positive kernel, and from the fact that the kernel k F , is effective i.e. the edges {e = (x, y) | | k F (e) | < K }, K > 0 generate the coarse structure of the network (see [3] for technical details), which, given in the case at hand, of finite networks, can naturally to be taken as the discrete coarse structure generated by sets that contain only a finite number of points (nodes) off the diagonal. Therefore we can apply the following result: Theorem 4 [3,Theorem 11.16] Let X be a coarse space. Then the following statements are equivalent: 1. X can be coarsely embedded into a Hilbert space.

There exists an effective negative type kernel on X .
In fact, the observation above can be strengthened in more than one way: On the one hand, the result can be applied to any of the kernels above, not only the one considered above. Furthermore, due to finiteness, one can relax the effectiveness condition by dispensing with the absolute value in the defining inequality. On the other hand, the Forman-Ricci curvature kernels are effective even if one considers ε-nets obtained from the geometric sampling of non-compact Riemannian manifolds with Ricci curvature bounded from below, given that the sampling procedure is essentially the same as for the compact ones (see [4] for the classical case and [14] for the generalization to metric measure spaces). The only difference between the compact case and the one at hand is that one has to relax somewhat the coarseness isometry definition and the resulting ε-net, endowed with the combinatorial metric (i.e. with edge lengths ≡ 1), will be only roughly isometric to the given manifold, where rough isometry is defined as follows: Definition 4 Let (X , d) and (Y , δ) be two metric spaces, and let f : X → Y (not necessarily continuous). f is called a rough isometry iff 1. There exist a ≥ 1 and b > 0, such that (that is f is ε 1 -full.) It follows that all types of kernels based on Forman's discretization of the Bochner-Weitzenböck may be applied in the variety of SVM problems where kernels are usually employed, such as clustering and classification. In particular, one can compute the so called kernel distance [15,16]: Definition 5 Given a similarity function (kernel) K : R d × R d → R, such that K ( p, p) = 1, for any p ∈ R d , and given sets P, Q ⊂ R d , the kernel distance D K (P, Q) is defined as  K ( p, q)), thus 1 − K ( p, q) can be viewed as a proper squared distance, since 1 − K ( p, p) = 0.

Remark 3
The quantity κ(P, Q) = p∈P q∈Q K ( p, q) is called the crosssimilarity (of P and Q), thus D K (P, Q) can be expressed, more simply, in terms of the cross similarity, rather then in means of the kernel as κ(P, P) − 2κ(P, Q) + κ(Q, Q).

First results: network visualization
A first application of the kernel distance is in the visualization of kernel spaces and data in general [15,17]. The embedding method employed here is the so called mutidimensional scaling (MDS) one (see, e.g. [18]). More precisely, given N points p 1 , . . . , p N ,  Fig. 1 Two views of the "Kangaroo" Graph Forman-Ricci curvature derived, kernel-based, embedding in R 3 which reveal that the network essentially consist from one large cluster. The attained minimum for the cost function in this case is J = 23.841747 we can approximate their position in a kernel space H ⊂ R d by y 1 , . . . , y N ∈ R d , by minimizing the cost-function where || · || denote the Euclidean norm on R d and D i j = D K ( p i , p j ). We illustrate this application below, using the Graph Forman-Ricci curvature, on two real-life networks, namely the simple, illustrative "Kangaroo" network, and on the larger "Les Misérables" one. As the second example proves, this visualization method is quite efficient at distinguishing various clusters in a large network, due to the high degree of separation that higher dimensional spaces afford. In these examples, to numerically determine the minimum of the cost-function J we made appeal to the Fig. 2 Two views of the Graph Forman-Ricci curvature derived, kernel-based embedding of the "Les Misérables" characters network. Here the attained minimum for the cost function is J = 42.264625. Note that the kernel-based embedding into R 3 clearly distinguishes the clusters around each of the main characters in Hugo's novel, connected by the relations between them classical Douglas-Rachford algorithm [19]. We have illustrated these examples in Figs. 1 and 2.
Furthermore, we have implemented the 1 based embedding. Perhaps contrary to the common wisdom, and thus somewhat unexpected, the Graph Forman-Ricci curvature renders a far better separation into clusters than the considered Laplacian. See Fig. 3.
The kernel embedding method not only is useful in the visualization of networks, it also (and perhaps more importantly) allows us to embed large complex networks into the familiar Euclidean space. Consequently one can employ well established and intuitive sampling methods of data points and sets in the usual ambient space.  1 (below) kernel-based embedding of the "C. Elegans" network [20]. Note the far better separation properties of the curvature based embedding. This ability has as a penalty a higher minimum of the cost function J , namely 404.342321 for the curvature-based kernel, as compared to 283.640003 for the Laplacian-based kernel

Conclusion and outlook
We have proposed a coarse geometric approach to Forman curvature and Bochner-Laplacian kernel based network embedding, and we have illustrated these ideas on a number of representative small and medium size networks, as well as on test images. Even though the experiments were quite restricted in scope and functioning more as illustrative cases and "proof of feasibility", some first conclusions can be drawn already, as we have noted throughout the text: • One should experiment and compare our proposed method with other embedding techniques than the one used herein, for instance the so called local linear embedding [21].
• Besides the kernel embedding method embraced in the present paper, one should explore another, and perhaps better established approach, that is the one using the eigenvectors and eigenfunctions of the Laplacian for embedding and sampling, where the classical graph Laplacian is replaced by the Bochner and rough ones. Furthermore, given that they exist for each and every dimension up to the maximal one, they suggest themselves as a potentially powerful method of studying hypernetworks/complexes at many scales. Additional and more systematic approaches can also be found in [22]. • Perhaps the most rewarding, from the applications viewpoint, direction of further study is the employment of the approach introduced herein to the fields of Imaging and Graphics. Indeed, the motivation for the search for possible curvature-based kernels stems from these settings (as a natural counterpoint of curvature-based sampling and reconstruction [23]). Clearly, in this case the networks to be studied are the 1-skeleta of the triangular or quadrilateral meshes, whereas the curvature to be employed could be the ones considered herein, metric curvatures [24] or the classical defect one (see, e.g. [25]).