Hypergraph co-optimal transport: metric and categorical properties

Abstract

Hypergraphs capture multi-way relationships in data, and they have consequently seen a number of applications in higher-order network analysis, computer vision, geometry processing, and machine learning. In this paper, we develop theoretical foundations for studying the space of hypergraphs using ingredients from optimal transport. By enriching a hypergraph with probability measures on its nodes and hyperedges, as well as relational information capturing local and global structures, we obtain a general and robust framework for studying the collection of all hypergraphs. First, we introduce a hypergraph distance based on the co-optimal transport framework of Redko et al. and study its theoretical properties. Second, we formalize common methods for transforming a hypergraph into a graph as maps between the space of hypergraphs and the space of graphs, and study their functorial properties and Lipschitz bounds. Finally, we demonstrate the versatility of our Hypergraph Co-Optimal Transport (HyperCOT) framework through various examples.

Appendix

1.1 Proofs from Sect. 2.3

To prove Theorem 1, we begin with a technical result.

Lemma 24

The infimum in the definition of $d_{\mathcal {H},p}$ is realized.

Proof

For a Polish space X, let $\textrm{Prob}(X)$ denote the set of Borel probability measures over X, endowed with the weak topology.

Let $H = (X,\mu ,Y,\nu ,\omega ), H' = (X',\mu ',Y',\nu ',\omega ') \in \mathcal {H}$. As X and $X'$ are Polish spaces and $\mu$ and $\mu '$ are Borel probability measures, we have that $\mathcal {C}(\mu ,\mu ')$ is compact in $\textrm{Prob}(X\times X')$, by Chowdhury and Mémoli (2019, Lemma 10). It follows that $\mathcal {C}(\mu ,\mu ') \times \mathcal {C}(\nu ,\nu ')$ is compact in $\textrm{Prob}(X\times X') \times \textrm{Prob}(Y \times Y')$.

Now we will show that the distortion functional $\textrm{dis}_p^{\mathcal {H}} = \textrm{dis}_{H,H',p}^\mathcal {H}$ is continuous on $\mathcal {C}(\mu ,\mu ') \times \mathcal {C}(\nu ,\nu ')$, beginning with the $1\le p < \infty$ case. Since we are working in Polish spaces with finite measures, we have that bounded continuous functions are dense. So for every $n\in \mathbb {N}$, we can choose continuous bounded functions $\omega _n\in L^p(\mu \otimes \nu )$ and $\omega '_n\in L^p(\mu '\otimes \nu ')$ such that the following holds for all $x\in X, y\in Y, x'\in X',$ and $y'\in Y'$:

$$\begin{aligned} \Vert \omega (x,y)-\omega _n(x,y)\Vert _{L^p(\mu \otimes \nu )} \le \frac{1}{n} \quad \text{ and } \quad \Vert \omega '(x',y')-\omega '_n(x',y')\Vert _{L^p(\mu '\otimes \nu ')} \le \frac{1}{n}. \end{aligned}$$

Then for each $n\in \mathbb {N}$ we define a n-distortion functional $\textrm{dis}_p^n: \mathcal {C}(\mu ,\mu ')\times \mathcal {C}(\nu ,\nu ')\longrightarrow \mathbb {R}_+$ as $\textrm{dis}_p^n(\pi ,\xi ):= \Vert \omega _n(x,y) - \omega '_n(x',y') \Vert _{L^p(\pi \otimes \xi )}$

Now let us show that the n-distortion functional is continuous. Let $(\pi ,\xi )\in \mathcal {C}(\mu ,\mu ')\times \mathcal {C}(\nu ,\nu ')$ and let $(\pi _m)_{m\in \mathbb {N}}$ and $(\xi _m)_{m\in \mathbb {N}}$ be sequences in $\mathcal {C}(\mu ,\mu ')$ and $\mathcal {C}(\nu ,\nu ')$ that converge to $\pi$ and $\xi$ with respect to the weak topology. Then

$$\begin{aligned}&\lim _{m\rightarrow \infty }\text {dis}_p^n(\pi _m,\xi _m) \end{aligned}$$

(29)

$$\begin{aligned}&\quad = \lim _{m\rightarrow \infty }\left( \int _{Y\times Y'}\int _{X\times X'} |\omega _n(x,y) - \omega '_n(x',y') |^p \pi _m (dx \times dx')\xi _m(dy \times dy')\right) ^{\frac{1}{p}} \end{aligned}$$

(30)

$$\begin{aligned}&\quad = \left( \int _{Y\times Y'}\int _{X\times X'} |\omega _n(x,y) - \omega '_n(x',y') |^p\pi (dx \times dx')\xi (dy \times dy')\right) ^{\frac{1}{p}} \nonumber \\&\quad = \text {dis}_p^n(\pi ,\xi ). \end{aligned}$$

(31)

So we have that $\text {dis}_p^n$ is sequentially continuous. As $\mathcal {C}(\mu ,\mu ')\times \mathcal {C}(\nu ,\nu ') \subseteq$ Prob$(X\times X'\times Y\times Y')$, it is a metrizable space (Ambrosio et al. (2005), Remark 5.1.1). Therefore, as $\text {dis}_p^n$ is sequentially continuous in a metrizable space, it is also continuous.

Now let us see that $\text {dis}_p^n$ converges to $\text {dis}^\mathcal {H}_p$ uniformly. Let $(\pi ,\xi )\in \mathcal {C}(\mu ,\mu ')\times \mathcal {C}(\nu ,\nu ')$ and observe

$$\begin{aligned}&|\text {dis}_p^{\mathcal {H}}(\pi ,\xi ) - \text {dis}_p^n(\pi ,\xi ) |\\&\quad = |\Vert \omega (x,y) - \omega '(x',y') \Vert _{L^p(\pi \otimes \xi )}-\Vert \omega _n(x,y) - \omega '_n(x',y') \Vert _{L^p(\pi \otimes \xi )} |\\&\quad \le \Vert \omega (x,y) - \omega '(x',y') - \omega _n(x,y) + \omega '_n(x',y') \Vert _{L^p(\pi \otimes \xi )} \\&\quad \le \Vert \omega (x,y)-\omega _n(x,y)\Vert _{L^p(\mu \otimes \nu )} +\Vert \omega '(x',y')-\omega '_n(x',y')\Vert _{L^p(\mu '\otimes \nu ')} \le \frac{2}{n}. \end{aligned}$$

Since $\textrm{dis}_p^{\mathcal {H}}$ is the uniform limit of continuous functions, it is also continuous. It follows that the infimum in (4) is realized with a minimal coupling.

For the case in which $p=\infty$, let $(\pi ,\xi )$ be couplings in $\mathcal {C}(\mu ,\mu ')\times \mathcal {C}(\nu ,\nu ')$. As the domain of $\textrm{dis}_p$, $\mathcal {C}(\mu ,\mu ')\times \mathcal {C}(\nu ,\nu ')$, is a subset of Prob$(X\times X'\times Y\times Y')$, Jensen’s Inequality shows that if $1\le p\le q\le \infty$, then we have that $\textrm{dis}_p^{\mathcal {H}} \le \textrm{dis}_q^{\mathcal {H}}$. Also note that $\lim _{p\rightarrow \infty } \textrm{dis}_p^{\mathcal {H}}(\pi ,\xi ) =\textrm{dis}_\infty ^{\mathcal {H}}(\pi ,\xi )$. Then as for $p\in [1,\infty ), \textrm{dis}_p^{\mathcal {H}}$ is continuous, and $\textrm{dis}_{\infty }^{\mathcal {H}} = \sup \{\textrm{dis}_p^{\mathcal {H}} \, \vert \, p \in [1,\infty ) \}$ we have that $\textrm{dis}_\infty ^{\mathcal {H}}$ is lower semicontinuous. As $\textrm{dis}_p^\mathcal {H}$ is at least lower semicontinuous over a compact domain, we have that the infimum in (4) is realized with a minimal coupling for any $p\in [1,\infty ]$. $\square$

Proof of Theorem 1

In the following, let $H=(X,\mu ,Y,\nu ,\omega )$, $H'=(X',\mu ',Y',\nu ',\omega ')$ and $H'' = (X'',\mu '',Y'',\nu '',\omega '')$ be measure hypernetworks.

Pseudometric. Symmetry is obvious. For the triangle inequality, let $(\pi _{12},\xi _{12})$ and $(\pi _{23},\xi _{23})$ be couplings realizing $d_{\mathcal {H},p}(H,H')$ and $d_{\mathcal {H},p}(H',H'')$, respectively (see Lemma 24). Then, using the gluing lemma (see, e.g., (Sturm 2012, Lemma 1.4)), we construct a probability measure $\pi$ on $X \times X' \times X''$ such that the $X \times X'$ marginal of $\pi$ is $\pi _{12}$, the $X' \times X''$ marginal is $\pi _{23}$ and the $X \times X''$ marginal is a coupling of $\mu$ and $\mu ''$, which we denote $\pi _{13}$. Likewise, we construct a probability measure $\xi$ on $Y \times Y' \times Y''$ with analogous properties—in particular, the $Y \times Y''$ marginal is denoted $\xi _{13}$. Then, using the various marginal conditions, we have

$$\begin{aligned}&d_{\mathcal {H},p}(H,H'') \\&\quad \le \textrm{dis}_p^\mathcal {H}(\pi _{13},\xi _{13}) \\&\quad =\Vert \omega -\omega ''\Vert _{L^p(\pi \otimes \xi )}\\&\quad \le \Vert \omega -\omega '\Vert _{L^p(\pi \otimes \xi )}+\Vert \omega '-\omega ''\Vert _{L^p(\pi \otimes \xi )}\\&\quad =\Vert \omega -\omega '\Vert _{L^p(\pi _{12}\otimes \xi _{12})}+\Vert \omega '-\omega ''\Vert _{L^p(\pi _{23}\otimes \xi _{23})} = d_{\mathcal {H},p}(H,H')+d_{\mathcal {H},p}(H',H''). \end{aligned}$$

Complete. We prove completeness by adapting the proof of Sturm (2012, Theorem 5.8). Let $H_n = (X_n,\mu _n,Y_n,\nu _n,\omega _n)$ be a Cauchy sequence of measure hypernetworks. Then we can choose couplings $\pi _n \in \mathcal {C}(\mu _{n},\mu _{n+1})$ and $\xi _n \in \mathcal {C}(\nu _n,\nu _{n+1})$ such that $\textrm{dis}_p^{\mathcal {H}}(\pi _n,\xi _n) \rightarrow 0$. We then apply the generalized Gluing Lemma (Sturm 2012, Lemma 1.4) to $\pi _1,\ldots ,\pi _{N-1}$ (respectively, to $\xi _1,\ldots ,\xi _{N-1}$) to construct a measure $\Pi _N$ on $\prod _{n=1}^N X_n$ (respectively, $\Xi _N$ on $\prod _{n=1}^N Y_N$). We now define $\overline{\mu }$ to be the projective limit $\varprojlim \Pi _N$, a well-defined probability measure on $\prod _{n=1}^\infty X_n$ (e.g., Bochner 2020). Set $\overline{X}:= \textrm{supp}(\overline{\mu })$; then $\overline{X}$ is a closed subspace of a countable product of Polish spaces and is therefore Polish. Similarly, define $\overline{\nu }:= \varprojlim \Xi _N$ and $\overline{Y}:= \textrm{supp}(\overline{\nu })$. Next we define $\Omega _N: \overline{X} \times \overline{Y} \rightarrow {\mathbb {R}}$ by

$$\begin{aligned} \Omega _N((x_i)_{i=1}^\infty ,(y_i)_{i=1}^\infty ) = \omega _N(x_N,y_N). \end{aligned}$$

The sequence $(\Omega _N)_{N=1}^\infty$ is a Cauchy sequence in the Banach space $L^p(\overline{X} \times \overline{Y}, \overline{\mu } \otimes \overline{\nu })$, since

$$\begin{aligned} \Vert \Omega _N - \Omega _{N+1}\Vert _{L^p(\overline{\mu } \otimes \overline{\nu })} = \textrm{dis}_p^{\mathcal {H}}(\pi _N,\xi _N) \rightarrow 0, \end{aligned}$$

and we define $\overline{\omega }$ to be its limit. Each $\omega _N$ is essentially bounded, so it follows that $\overline{\omega }$ is as well. Setting $\overline{H}:= (\overline{X},\overline{\mu },\overline{Y},\overline{\nu },\overline{\omega })$, we have $\overline{H} \in \mathcal {H}$ and

$$\begin{aligned} d_{\mathcal {H},p}(H_n,\overline{H}) \le \Vert \omega _n - \overline{\omega }\Vert _{L^p(\overline{\mu } \otimes \overline{\nu })} \rightarrow 0. \end{aligned}$$

Geodesic. Finally, we show that the metric on $\mathcal {H}$ modulo weak isomorphism is geodesic. For $H \in \mathcal {H}$, let [H] denote its weak isomorphism class. We will construct an explicit geodesic between any two weak isomorphism classes of hypernetworks [H] and $[H']$, following the main idea of the proof of Sturm (2012, Theorem 3.1). Let $\pi \in \mathcal {C}(\mu ,\mu ')$ and $\xi \in \mathcal {C}(\nu ,\nu ')$ be optimal couplings (which exist, by Lemma 24). Consider the path $\gamma :[0,1] \rightarrow \mathcal {H}$ defined by

$$\begin{aligned} \gamma (t) = \left( \textrm{supp}(\pi ), \pi , \textrm{supp}(\xi ), \xi , \omega _t\right) , \end{aligned}$$

(32)

where

$$\begin{aligned} \omega _t((x,x'),(y,y')) = (1-t)\omega (x,y) + t \omega '(x',y'). \end{aligned}$$

Observe that $\gamma (0) \in [H]$ and $\gamma (1) \in [H']$; indeed, coordinate projection maps $\phi :X \times X' \rightarrow X$ and $\psi :Y \times Y' \rightarrow Y$ define a basic weak isomorphism $\gamma (0) \rightarrow H$ and a similar construction works for $\gamma (1) \rightarrow H'$. To show that $\gamma$ defines a geodesic in $[\mathcal {H}]$, it suffices to show that for all $0 \le s \le t \le 1$,

$$\begin{aligned} d_{\mathcal {H},p}(\gamma (s),\gamma (t)) \le (t-s) d_{\mathcal {H},p}(H,H') \end{aligned}$$

(33)

(see, e.g., (Chowdhury and Mémoli 2018, Lemma 1.3)). One can show by an elementary computation that

$$\begin{aligned} \textrm{dis}_{\gamma (s),\gamma (t),p}^{\mathcal {H}}(\pi ,\xi )^p \le (t-s)^p \textrm{dis}_{H,H',p}^{\mathcal {H}}(\pi ,\xi )^p \end{aligned}$$

for any pair of couplings $(\pi ,\xi )$—indeed, this holds because linear interpolations are geodesics in $L^p$ spaces—and (33) follows. $\square$

Remark 12

The proof of the triangle inequality above uses the notion of gluing, which appears in our category theoretic framework in Sect. 3. The coupling $\pi _{13}$ constructed above is written as $\pi _{13} = \pi _{23} \bullet \pi _{13}$ in the notation from Sect. 3. A byproduct of our proof is the following useful fact: for measure networks $N,N',N''$ and couplings $\pi \in \mathcal {C}(\mu ,\mu ')$ and $\pi ' \in \mathcal {C}(\mu ',\mu '')$, we have

$$\begin{aligned} \textrm{dis}_p^\mathcal {N}(\pi ' \bullet \pi ) \le \textrm{dis}_p^\mathcal {N}(\pi ') + \textrm{dis}_p^\mathcal {N}(\pi ). \end{aligned}$$

A similar statement holds in the measure hypernetwork setting.

1.2 Proofs of technical Lemmas from Sect. 3.6

Proof of Lemma 12

We will prove the statement for hypernetworks, as the network case is similar. Let $\pi$ and $\xi$ be couplings realizing $d_{\mathcal {H},p}(H,H')$ (such couplings always exist, see Lemma 24 in the appendix). Define

$$\begin{aligned} \overline{X} = \textrm{supp}(\pi ) \subset X \times X', \quad \overline{\mu } = \pi \mid _{\overline{X}}, \quad \overline{Y} = \textrm{supp}(\xi ) \subset Y \times Y', \quad \overline{\nu } = \xi \mid _{\overline{Y}} \end{aligned}$$

and define hypernetwork functions $\overline{\omega },\overline{\omega }':\overline{X} \times \overline{Y} \rightarrow {\mathbb {R}}$ as follows. Let $p_Z:X \times X' \rightarrow Z$ be coordinate projection for $Z \in \{X,X'\}$ and let $p_Z:Y \times Y' \rightarrow Z$ be coordinate projection for $Z \in \{Y,Y'\}$. The hypernetwork functions are then given by

$$\begin{aligned} \overline{\omega } = \omega \circ (p_X \times p_Y) \qquad \text{ and } \qquad \overline{\omega }' = \omega ' \circ (p_{X'} \times p_{Y'}). \end{aligned}$$

The coordinate projections are measure-preserving, due to the marginal constraints of $\pi$ and $\xi$, and the pair $(p_X,p_Y)$ (respectively, $(p_{X'},p_{Y'})$) therefore defines a basic weak isomorphism (Definition 6) from $\overline{H}$ to H (respectively, $\overline{H'}$ to $H'$).

To see (12), observe first that $d_{\mathcal {H},p}(H,H') = d_{\mathcal {H},p}(\overline{H},\overline{H'})$ follows by weak isomorphism. To see the other equality for $p \in [1,\infty )$, we have

$$\begin{aligned} d_{\mathcal {H},p}(H,H')^p&= \int _{Y \times Y'} \int _{X \times X'} |\omega (x,y) - \omega '(x',y')|^p \; \pi (dx \times dx') \xi (dy \times dy') \\&= \int _{\overline{Y}} \int _{\overline{X}} |\omega (x,y) - \omega '(x',y')|^p \; \overline{\mu }(dx \times dx') \overline{\nu }(dy \times dy'), \\ \end{aligned}$$

since the value of the integral isn’t changed by replacing the full spaces with supports of the measures. Using the change of variables $\overline{x} = (x,x') \in \overline{X}$ and $\overline{y} = (y,y') \in \overline{Y}$, the above is equal to

$$\begin{aligned} \int _{\overline{Y}} \int _{\overline{X}} |\overline{\omega }(\overline{x},\overline{y}) - \overline{\omega }'(\overline{x},\overline{y})|^p \; \overline{\mu }(d\overline{x}) \overline{\nu }(d\overline{y}) = \Vert \overline{\omega } - \overline{\omega }'\Vert _{L^p(\overline{\mu } \otimes \overline{\nu })}^p. \end{aligned}$$

For the $p = \infty$ case, a similar change of variables gives

$$\begin{aligned} d_{\mathcal {H},\infty }(H,H')&= \inf \{c \mid |\omega (x,y) - \omega '(x',y')|\le c \text{ for } \pi \otimes \xi -\text{ almost } \text{ every } (x,y,x',y')\} \\&= \inf \{c \mid |\overline{\omega }(\overline{x},\overline{y}) - \overline{\omega }'(\overline{x},\overline{y})|\\&\le c \text{ for } \overline{\mu } \otimes \overline{\xi }-\text{ almost } \text{ every } (\overline{x},\overline{y})\} \\&= \Vert \overline{\omega } - \overline{\omega }'\Vert _{L^\infty (\overline{\mu } \times \overline{\nu })}. \end{aligned}$$

$\square$

Proof of Lemma 13

By Proposition 2, it suffices to show that if there is a basic weak isomorphism $(\phi ,\psi )$ from H to $H'$, then $\phi$ defines a basic weak isomorphism of $N:= \textsf{Q}_q(H) = (X,\mu ,\omega _{\textsf{Q}_q})$ and $N':= \textsf{Q}_q(H') = (X',\mu ',\omega _{\textsf{Q}_q}')$. By an argument similar to the one used to prove functoriality in Theorem 11, the assumption that $(\phi ,\psi )$ is a basic weak isomorphism implies that for $\mu \otimes \mu \otimes \nu$-almost every $(x_1,x_2,y) \in X \times X \times Y$, $\min _j \omega (x_j,y) = \min _j \omega '(\phi (x_j),\psi (y))$. In the $q < \infty$ case, we deduce that for $\mu \otimes \mu$ almost every $(x_1,x_2)$,

$$\begin{aligned} \omega _{\textsf{Q}_q}'(\phi (x_1),\phi (x_2))^p&= \int _{Y'} \min _j \omega '(\phi (x_j),y')^p \; \nu ' (dy') \nonumber \\&= \int _{Y} \min _j \omega '(\phi (x_j),\psi (y))^p \; \nu (dy)\nonumber \\&= \int _{Y} \min _j \omega (x_j,y)^p \; \nu (dy) \nonumber \\&= \omega _{\textsf{Q}_q}(x_1,x_2)^p, \end{aligned}$$

(34)

where (34) follows by the change-of-variables formula (using that $\psi$ is measure-preserving). The $q=\infty$ case follows by a limiting argument. $\square$

Proof of Lemma 14

This is Mémoli (2011a, Theorem 5.1 (d)), in the metric measure space setting. The proof goes through in the measure network setting. $\square$

1.3 Runtime analysis for hypergraph simplification

We ran the experiments using the following computing infrastructures:

• CPU models: 32 Intel Xeon Silver 4108 CPU 1.80 GHz cores (HT)

• Amount of memory: 132 GB of RAM

• Operating system: OpenSUSE Leap 15.3 (x86_64)

Table 1 summarizes the size of each dataset, including the number of hyperedges and the number of vertices contained in each hypergraph, the runtime of computing the barcode, the runtime of performing the simplification (simp.) at the first step, and the maximum runtime among all simplification steps for each dataset. Figure 12 demonstrates the distribution and the minimum runtime of computing the hypernetwork distances $d_{\mathcal {H},2}(H_0, H_i)$ at each step i. Table 2 summarizes the minimum runtime at the first step as well as the minimum, maximum, mean, and median values of the minimum runtime of computing the distances across all simplification steps for each dataset.

The largest dataset is the Coauthor Network dataset, which contains 506 nodes and 491 hyperedges. Therefore, this dataset gives rise to the largest runtime of computing the barcode, simplified hypergraphs, and hypernetwork distances. The maximum runtime among all steps is around 5 s for the Coauthor Network dataset.

Table 1 The number of vertices (#v), the number of hyperedges (#e), the runtime (in seconds) of computing the barcode, the runtime (in seconds) of performing the simplification at the first step, and the maximum runtime (in seconds) among all simplification steps in datasets ActiveDNS (DNS), Coauthor Network (CN), and Les Misérables (LM)

Fig. 12

Runtime in computing the GW distances for activeDNS (a), Les Misérables (b) and Coauthor Network (c) datasets. The x-axes display the simplification steps. The y-axes are time recorded in seconds

Table 2 The minimum runtime (in seconds) at the first step as well as the minimum, maximum, mean, and median values of the minimum runtime (in seconds) of computing the distances across all simplification steps for each dataset