k-Point semidefinite programming bounds for equiangular lines

Abstract

We propose a hierarchy of k-point bounds extending the Delsarte–Goethals–Seidel linear programming 2-point bound and the Bachoc–Vallentin semidefinite programming 3-point bound for spherical codes. An optimized implementation of this hierarchy allows us to compute 4, 5, and 6-point bounds for the maximum number of equiangular lines in Euclidean space with a fixed common angle.

1 Introduction

Given $D \subseteq [-1,1)$, a subset C of the unit sphere $S^{n-1} = \{\, x \in {\mathbb {R}}^n : \Vert x\Vert = 1\,\}$ is a spherical D-code if $x \cdot y \in D$ for all distinct x, $y \in C$, where $x\cdot y$ is the Euclidean inner product between x and y. The maximum cardinality of a spherical D-code in $S^{n-1}$ is denoted by A(n, D).

Different sets D describe different problems that can be treated with similar techniques. The most important cases are D being an interval and D being a finite set. If $D = [-1, \cos (\pi /3)]$, then A(n, D) is the kissing number, the maximum number of pairwise nonoverlapping unit spheres that can touch a central unit sphere.

A fundamental tool for computing upper bounds for A(n, D) is the linear programming bound of Delsarte et al. [13], which is an adaptation of the Delsarte bound [12] to the sphere. The linear programming bound was one of the first nontrivial upper bounds for the kissing number and is the optimal value of a convex optimization problem. It is a 2-point bound, because it takes into account interactions between pairs of points on the sphere: pairs $\{x, y\}$ with $x \cdot y \notin D$ correspond to constraints in the optimization problem. Bachoc and Vallentin [2] extended the linear programming bound to a 3-point bound by taking into account interactions between triples of points, extending the three-point bound by Schrijver [44] for binary codes. The resulting semidefinite programming bound gives the best known upper bounds for the kissing number for all dimensions $3 \le n \le 24$, although in dimensions $n = 3, 4, 8$, and 24 the optimal values were already known by other methods.

In the same paper in which the linear programming bound was proposed, Delsarte et al. [13] considered its application to bound A(n, D) when D is finite and also to the related problem of bounding A(n, D) for all D with a given size $|D| = s$. The semidefinite programming bound from Bachoc and Vallentin was first computed for these problems by Barg and Yu [4].

In this paper, we give a hierarchy of k-point bounds that extend both the linear and semidefinite programming bounds. We model the parameter A(n, D) as the independence number of a graph, namely the infinite graph with vertex set $S^{n-1}$ in which two vertices x and y are adjacent if $x\cdot y \notin D$. The linear programming bound corresponds to an extension of the Lovász theta number to this infinite graph [1]. In Sect. 2, we derive our hierarchy from a generalization [11] of Lasserre’s hierarchy to a class of infinite graphs that comprises the graph being considered. The first level of our hierarchy is the Lovász theta number, and is therefore equivalent to the linear programming bound; the second level is the semidefinite programming bound by Bachoc and Vallentin, as shown in Sect. 5.2. This puts the 2 and 3-point bounds in a common framework and shows how these relate to the Lasserre hierarchy.

For the case where D is infinite, we give a precise reason why it is difficult to compute the problems in this hierarchy when $k \ge 4$. This might explain why so far nobody has been able to compute a 4-point bound generalization of the 2 and 3-point bounds for the kissing number problem. For the case where D is finite there is no such obstruction, and though our hierarchy is not as strong, in theory, as the Lasserre hierarchy, it is computationally less expensive. This allows us to use it to compute 4, 5, and 6-point bounds for the maximum number of equiangular lines with a certain angle, a problem that corresponds to the case $|D|=2$. Aside from a previous result of de Laat [10], which uses Lasserre’s hierarchy directly, this is the first successful use of k-point bounds for $k > 3$ for geometrical problems; it yields improved bounds for the number of equiangular lines with given angles in several dimensions.

To perform computations, we transform the resulting problems into semidefinite programming problems. To this end, for a given $k \ge 2$ we use a characterization of kernels $K:S^{n-1} \times S^{n-1} \rightarrow {\mathbb {R}}$ on the sphere that are invariant under the action of the subgroup of the orthogonal group that stabilizes $k-2$ given points. For $k=2$, this characterization was given by Schoenberg [43] and for $k = 3$, by Bachoc and Vallentin [2]; Musin [36] extended these two results for $k > 3$; a similar result is given by Kuryatnikova and Vera [42].

Still, a naive implementation of our approach would be too slow even to generate the problems for $k = 5$. The implementation available with the arXiv version of this paper was carefully written to deal with the orbits of k points in the sphere in an efficient way; this allows us to generate problems even for $k = 6$. This implementation could be of interest to others working on similar problems.

1.1 Equiangular lines

A set of equiangular lines is a set of lines through the origin such that every pair of lines defines the same angle. If this angle is $\alpha$, then such a set of equiangular lines corresponds to a spherical D-code where $D = \{a, -a\}$ and $a = \cos \alpha$. So we are interested in finding $A(n, \{a,-a\})$ for a given $a \in [-1,1)$ and also in finding the maximum number of equiangular lines with any given angle, namely

$$\begin{aligned} M(n) = \max \{\, A(n, \{a,-a\}) : a \in [-1,1)\,\}. \end{aligned}$$

The study of M(n) started with Haantjes [24]. He showed that $M(2) = 3$ and that the optimal configuration is a set of lines on the plane having a common angle of $60^{\circ }$. He also showed that $M(3) = 6$; the optimal configuration is given by the lines going through opposite vertices of a regular icosahedron, which have a common angle of $63.43\ldots$ degrees. These two constructions provide lower bounds; in both cases, Gerzon’s bound, which states that $M(n) \le n(n+1)/2$ (see Theorem 6.1 below which is proven for example in Matoušek’s book [35, Miniature 9]), provides matching upper bounds.

In the setting of equiangular lines, the LP bound coincides with van Lint and Seidel’s relative bound ([47], see also, Theorem 6.5). The 3-point SDP bound was first specialized to this setting by Barg and Yu [4]. No k-point bound has been computed or formulated for $k \ge 4$ for equiangular lines or for any other spherical code problem. Gijswijt, Mittelmann, and Schrijver [17] computed 4-point SDP bounds for binary codes and Litjens, Polak, and Schrijver [34] extended these 4-point bounds to q-ary codes.

Next to being fundamental objects in discrete geometry, equiangular lines have applications, for example in the field of compressed sensing: Only measurement matrices whose columns are unit vectors determining a set of equiangular lines can minimize the coherence parameter [16, Chapter 5].

Table 1 Known values for M(n) for small dimensions together with the cosine a of the common angle between the lines

In general, it is a difficult problem to determine M(n) for a given dimension n. Currently, the first open case is dimension $n = 17$ where it is known that M(17) is either 48 or 49; see Table 1. Sequence A002853 in The On-Line Encyclopedia of Integer Sequences [45] is M(n).

2 Derivation of the hierarchy

In this section we derive a hierarchy of bounds for the independence number of a graph. We first derive this for finite graphs and then we show how this can be extended to a larger class, which includes the infinite graphs that we use to model the geometric problems described in the introduction. We provide detailed arguments to justify each step of the derivation, but Proposition 2.1 at the end of the section has a direct and simple proof for the validity of the bound we use in the rest of the paper.

Let $G = (V,E)$ be a graph. A subset of V is independent if it does not contain a pair of adjacent vertices. The independence number of G, denoted by $\alpha (G)$, is the maximum cardinality of an independent set. For an integer $k \ge 0$, let $I_{k}$ be the set of independent sets in G of size at most k and $I_{=k}$ be the set of independent sets in G of size exactly k.

2.1 Definition of the hierarchy for finite graphs

Assume for now that G is finite. We can obtain upper bounds for the independence number via the Lasserre hierarchy [29] for the independent set problem, whose t-th step, as shown by Laurent [30], can be formulated as

$$\begin{aligned} \max \left\{ \,\sum _{S \in I_{=1}} \nu _S : \nu \in {\mathbb {R}}^{I_{2t}}_{\ge 0}, \ \nu _{\emptyset } = 1,\text { and } M(\nu ) \succeq 0\,\right\} , \end{aligned}$$

(1)

where $M(\nu )$ is the matrix indexed by $I_t \times I_t$ such that

$$\begin{aligned} M(\nu )_{J,J'} = {\left\{ \begin{array}{ll} \nu _{J \cup J'} &{}\quad \text {if } J\cup J' \text { is independent};\\ 0 &{}\quad \text {otherwise} \end{array}\right. } \end{aligned}$$

(2)

and $M(\nu ) \succeq 0$ means that $M(\nu )$ is positive semidefinite. It is easily seen that this hierarchy bounds the independence number from above since for an independent set $C \subseteq V$, the vector $\nu \in {\mathbb {R}}^{I_{2t}}$ defined by $\nu _S = 1$ if $S \subseteq C$ and $\nu _S = 0$ otherwise is such that $M(\nu )$ is a principal submatrix of $\nu \nu ^\mathsf{T}$ and hence is a feasible solution to (1) with value $\sum _{S \in I_{=1}} \nu _S = |C|$. It is also shown [30] that this hierarchy converges to the independence number in at most $\alpha (G)$ steps.

To produce an optimization program where the variables are easier to parameterize, we construct in two stages a weaker hierarchy with matrices indexed only by the vertex set of the graph. First, we modify the problem to remove $\emptyset$ from the domain of $\nu$; this gives the possibly weaker problem

$$\begin{aligned} \max \left\{ \,1+2\sum _{S \in I_{=2}} \nu _S : \nu \in {\mathbb {R}}^{I_{2t} {\setminus } \{\emptyset \}}_{\ge 0}, \ \sum _{S \in I_{=1}} \nu _S = 1,\text { and } M(\nu ) \succeq 0\,\right\} , \end{aligned}$$

(3)

where $M(\nu )$ is now considered as a matrix indexed by $(I_t {\setminus } \{\emptyset \}) \times (I_t {\setminus } \{\emptyset \})$. To see how problem (3) is a weaker version of problem (1) and thus still an upper bound for the independence number, let $\nu \in {\mathbb {R}}^{I_{2t}}$ be a feasible solution for (1) and define ${\overline{\nu }} \in {\mathbb {R}}^{I_{2t} {\setminus } \{\emptyset \}}$ as ${\overline{\nu }}_S = \nu _S / (\sum _{Q \in I_{=1}} \nu _Q)$. One can check that ${\overline{\nu }}$ is feasible for (3) and $\sum _{S \in I_{=1}} \nu _S \le 1+2\sum _{S \in I_{=2}} {\overline{\nu }}_S$. To justify this last inequality, apply the Schur complement to the submatrix of $M(\nu )$ indexed by $I_{1}$ to conclude that the matrix

$$\begin{aligned} \big ( \nu _{\{u,v\}} - \nu _{\{u\}}\nu _{\{v\}} \big )_{u, v \in V} \end{aligned}$$

indexed by $I_{=1} \simeq V$ (set $\nu _{\{u,v\}} = 0$ if $\{u,v\}$ is not independent) is positive semidefinite and hence

$$\begin{aligned} \left( \sum _{u \in V} \nu _{\{u\}}\right) ^2 \le \sum _{u,v\in V}\nu _{\{u,v\}}, \end{aligned}$$

which implies the desired inequality.

Second, we construct a weaker hierarchy by only requiring certain principal submatrices of $M(\nu )$ to be positive semidefinite, an approach similar to the one employed by Gvozdenović et al. [23]. For this we fix $k \ge 2$ and, for each $Q \in I_{k-2}$, define the matrix $M_Q(\nu ):V \times V \rightarrow {\mathbb {R}}$ by

$$\begin{aligned} M_Q(\nu )(x,y) = {\left\{ \begin{array}{ll} \nu _{Q \cup \{x, y\}} &{}\quad \text {if } Q \cup \{x, y\} \in I_{k};\\ 0 &{}\quad \text {otherwise} \end{array}\right. } \end{aligned}$$

and replace the condition ‘$M(\nu ) \succeq 0$’ by ‘$M_Q(\nu ) \succeq 0$ for all $Q \in I_{k-2}$’. With these conditions we can restrict the support of $\nu$ to the set $I_k {\setminus } \{\emptyset \}$, obtaining the relaxation

$$\begin{aligned} \max \left\{ \,1+2\sum _{S \in I_{=2}} \nu _S : \nu \in {\mathbb {R}}^{I_k {\setminus } \{\emptyset \}}_{\ge 0}, \ \sum _{S \in I_{=1}} \nu _S = 1,\text { and } M_Q(\nu ) \succeq 0 \text { for } Q \in I_{k-2}\,\right\} . \end{aligned}$$

(4)

We now proceed to the computation of the dual of program (4). For that we use ${\mathbb {R}}^{V^2 \times I_{k-2}}$ to denote a collection of matrices $V \times V \rightarrow {\mathbb {R}}$ indexed by $I_{I_{k-2}}$ and ${\mathbb {R}}^{V^2 \times I_{k-2}}_{\succeq 0}$ to denote that each of these matrices is positive semidefinite. We define a linear operator $M_k :{\mathbb {R}}^{I_k {\setminus } \{\emptyset \}} \rightarrow {\mathbb {R}}^{V^2 \times I_{k-2}}$ by

$$\begin{aligned} M_k(\nu ) = \big (M_Q(\nu )\big )_{Q \in I_{k-2}} \end{aligned}$$

and write the constraints ‘$M_Q(\nu ) \succeq 0$ for all $Q \in I_{k-2}$’ as $M_k(\nu ) \in {\mathbb {R}}^{V^2 \times I_{k-2}}_{\succeq 0}$. The adjoint operator is defined in such a way that the inner product between $M_k(\nu )$ and $T \in {\mathbb {R}}^{V^2 \times I_{k-2}}$ is equal to the inner product between $\nu$ and $M_k^*(T)$:

$$\begin{aligned} \sum _{Q \in I_{k-2}} \sum _{x,y \in V} M_Q(\nu )(x,y) T(x,y,Q)&= \sum _{Q \in I_{k-2}} \; \sum _{\begin{array}{c} x,y \in V \\ Q \cup \{x,y\} \in I_k \end{array}} \nu _{Q \cup \{x,y\}} T(x,y,Q)\\&= \sum _{S \in I_k{\setminus } \{\emptyset \}} \nu _{S} \sum _{\begin{array}{c} Q \subseteq S \\ |Q| \le k-2 \end{array}} \sum _{\begin{array}{c} x, y \in S \\ Q \cup \{x, y\} = S \end{array}} T(x, y, Q), \end{aligned}$$

so we conclude that the expression for $M_k^* :{\mathbb {R}}^{V^2 \times I_{k-2}} \rightarrow {\mathbb {R}}^{I_k {\setminus } \{\emptyset \}}$ is

$$\begin{aligned} M_k^*(T)(S) = \sum _{\begin{array}{c} Q \subseteq S \\ |Q| \le k-2 \end{array}} \sum _{\begin{array}{c} x, y \in S \\ Q \cup \{x, y\} = S \end{array}} T(x, y, Q). \end{aligned}$$

(5)

Using the duality theory of conic optimization as described e.g. by Barvinok [6, Chapter IV], we can derive the following dual problem for (4):

$$\begin{aligned} \min \Big \{\,1 + \lambda : \lambda \in {\mathbb {R}},\ T \in {\mathbb {R}}^{V^2 \times I_{k-2}}_{\succeq 0},\text { and } M_k^*(T) \le \lambda \chi _{I_{=1}} - 2 \chi _{I_{=2}}\,\Big \}, \end{aligned}$$

(6)

where $\chi _{I_{=1}}$ and $\chi _{I_{=2}}$ are the characteristic functions of $I_{=1}$ and $I_{=2}$. It is a consequence of weak duality that program (6) gives an upper bound for the independence number. At the end of the next section we give a direct proof of this fact in a more general context.

2.2 Definition of the hierarchy for infinite graphs

We extend this hierarchy to infinite graphs in the same way that the Lasserre hierarchy is extended by de Laat and Vallentin [11]. This extension can be carried out for compact topological packing graphs; these are graphs whose vertex sets are compact Hausdorff spaces and in which every finite clique is contained in an open clique. The main consequences of this definition are that the independence number is finite and $I_k$, considered with the topology inherited from V, is the disjoint union of the compact and open sets $I_{=s}$ for $s = 0$, ..., k [11, Section 2]. We assume from now on that G is a compact topological packing graph.

The extension relies on the theory of conic optimization over infinite-dimensional spaces presented e.g.  by Barvinok [6]. The first step is to introduce the spaces for the variables of our problem; we will use both the space ${\mathcal {C}}(X)$ of continuous real-valued functions on a compact space X and its topological dual (with respect to the supremum norm) ${\mathcal {M}}(X)$, the space of signed Radon measures.

In the infinite setting, the nonnegative variable $\nu$ from (4) becomes a measure in the dual of the cone ${\mathcal {C}}(I_k {\setminus } \{\emptyset \})_{\ge 0}$ of continuous and nonnegative functions, namely

$$\begin{aligned} {\mathcal {M}}(I_k {\setminus } \{\emptyset \})_{\ge 0} = \{\,\nu \in {\mathcal {M}}(I_k {\setminus } \{\emptyset \}) : \nu (f) \ge 0 \text { for all } f \in {\mathcal {C}}(I_k {\setminus } \{\emptyset \})_{\ge 0}\,\}; \end{aligned}$$

we observe that when V is finite, ${\mathcal {M}}(I_k {\setminus } \{\emptyset \})_{\ge 0}$ can be identified with ${\mathbb {R}}^{I_k {\setminus } \{\emptyset \}}_{\ge 0}$.

Let ${\mathcal {C}}(V^2 \times I_{k-2})_{\mathrm {sym}}$ be the set of continuous real-valued functions on $V^2 \times I_{k-2}$ that are symmetric in the first two coordinates and let ${\mathcal {M}}(V^2 \times I_{k-2})_{\mathrm {sym}}$ be the space of symmetric and signed Radon measures¹. A kernel $K \in {\mathcal {C}}(V^2)$ is positive if for every finite $U \subseteq V$ the matrix $\bigl (K(x, y)\bigr )_{x,y \in U}$ is positive semidefinite. A function $T \in {\mathcal {C}}(V^2 \times I_{k-2})$ is positive if for every $Q \in I_{k-2}$ the kernel $(x, y) \mapsto T(x, y, Q)$ is positive. The set of all positive functions in ${\mathcal {C}}(V^2 \times I_{k-2})$ is a convex cone denoted by ${\mathcal {C}}(V^2 \times I_{k-2})_{\succeq 0}$; its dual cone is denoted by ${\mathcal {M}}(V^2 \times I_{k-2})_{\succeq 0}$.

Instead of extending the operator $M_k$ from the finite case, a key step in this extension is to use its adjoint. Based on formula (5), we define the operator $B_k :{\mathcal {C}}(V^2 \times I_{k-2})_{\mathrm {sym}} \rightarrow {\mathcal {C}}(I_k {\setminus } \{\emptyset \})$ by

$$\begin{aligned} B_k T(S) = \sum _{\begin{array}{c} Q \subseteq S \\ |Q| \le k-2 \end{array}} \sum _{\begin{array}{c} x, y \in S \\ Q \cup \{x, y\} = S \end{array}} T(x, y, Q). \end{aligned}$$

(7)

Note that, though the number of summands in (7) varies with the size of S, the function $B_k T$ is still continuous since, by the assumption that G is a topological packing graph, $I_k {\setminus } \{\emptyset \}$ can be written as the disjoint union of the compact and open subsets $I_{=s}$ for $s = 1$, ..., k and $B_kT$ is continuous in each of these parts. Furthermore, since the number of summands in (7) is bounded by a constant depending only on k, the operator $B_k$ is itself continuous. Thus it has an adjoint $B_k^* :{\mathcal {M}}(I_k {\setminus } \{\emptyset \}) \rightarrow {\mathcal {M}}(V^2 \times I_{k-2})_{\mathrm {sym}}$. Using the adjoint, we define the generalized k-point bound for $k \ge 2$:

$$\begin{aligned}&\Delta _k(G) = \sup \{\,1 + 2\nu (I_{=2}) : \nu \in {\mathcal {M}}(I_k {\setminus } \{\emptyset \})_{\ge 0},\nonumber \\&\quad \nu (I_{=1}) = 1,\text { and } B_k^*\nu \in {\mathcal {M}}(V^2 \times I_{k-2})_{\succeq 0}\, \}. \end{aligned}$$

(8)

Note that for a finite graph with the discrete topology this reduces to (4).

Again, using the duality theory of conic optimization [6, Chapter IV], we can derive the following dual problem for (8):

$$\begin{aligned} \Delta _k(G)^* = \inf \{\,1 + \lambda : \lambda \in {\mathbb {R}},\ T \in {\mathcal {C}}(V^2 \times I_{k-2})_{\succeq 0},\text { and } B_kT \le \lambda \chi _{I_{=1}} - 2 \chi _{I_{=2}}\,\}, \end{aligned}$$

(9)

where $\chi _{I_{=1}}$ and $\chi _{I_{=2}}$ are the characteristic functions of $I_{=1}$ and $I_{=2}$, which are continuous since G is a topological packing graph. From now on, we will denote both the optimal value of (9) and the optimization problem itself by $\Delta _k(G)^*$.

It is a direct consequence of weak duality that $\Delta _k(G)^*$ is an upper bound for the independence number of G, but it is instructive to see a direct proof.

Proposition 2.1

If $G = (V,E)$ is a compact topological packing graph, then $\alpha (G) \le \Delta _k(G)^*$.

Proof

Let $C \subseteq V$ be a nonempty independent set and let $(\lambda , T)$ be a feasible solution of $\Delta _k(G)^*$. On the one hand, since $B_kT \le \lambda \chi _{I_{=1}} - 2 \chi _{I_{=2}}$, we have

$$\begin{aligned} \sum _{\begin{array}{c} S \subseteq C\\ |S| \le k,\ S \ne \emptyset \end{array}} B_{k}T(S) \le \left( {\begin{array}{c}|C|\\ 1\end{array}}\right) \lambda + \left( {\begin{array}{c}|C|\\ 2\end{array}}\right) (-2) = |C|(1 + \lambda - |C|). \end{aligned}$$

On the other hand, since $T \in {\mathcal {C}}(V^2 \times I_{k-2})_{\succeq 0}$, we have

$$\begin{aligned} \begin{aligned} \sum _{\begin{array}{c} S \subseteq C\\ |S| \le k,\ S \ne \emptyset \end{array}} B_{k}T(S)&= \sum _{\begin{array}{c} S \subseteq C\\ |S| \le k,\ S \ne \emptyset \end{array}}\; \sum _{\begin{array}{c} Q \subseteq S \\ |Q| \le k-2 \end{array}}\; \sum _{\begin{array}{c} x, y \in S \\ Q \cup \{x, y\} = S \end{array}} T(x,y,Q)\\&= \sum _{\begin{array}{c} Q \subseteq C\\ |Q| \le k-2 \end{array}}\; \sum _{x, y \in C} T(x,y,Q) \ge 0 \end{aligned} \end{aligned}$$

since, by the definition of ${\mathcal {C}}(V^2 \times I_{k-2})_{\succeq 0}$, the matrices $\bigl (T(x,y,Q)\bigr )_{x,y\in C}$ are positive semidefinite for all $Q \in I_{k-2}$. Putting it all together we get $|C| \le 1 + \lambda$. $\square$

3 Symmetry reduction

Symmetry reduction plays a key role in the computation of $\Delta _k(G)^*$ in our application. We now see how to exploit symmetry to decompose the variable T of (9) in terms of simpler kernels from ${\mathcal {C}}(V^2)$. In this section we keep assuming that G is a compact topological packing graph and delay the specialization to the case where V is a sphere to the next section.

Let $\Gamma$ be a compact group that acts continuously on V and that is a subgroup of the automorphism group² of the graph G. The group $\Gamma$ acts coordinatewise on $V^2$, and this action extends to an action on ${\mathcal {C}}(V^2)$ by

$$\begin{aligned} (\gamma K)(x, y) = K(\gamma ^{-1} x, \gamma ^{-1} y). \end{aligned}$$

The group $\Gamma$ acts continuously on $I_t$ by

$$\begin{aligned} \gamma \emptyset = \emptyset \quad \text {and} \quad \gamma \{x_1,\ldots ,x_t\} = \{\gamma x_1,\ldots ,\gamma x_t\}, \end{aligned}$$

and hence it also acts on ${\mathcal {C}}(V^2 \times I_{k-2})_{\mathrm {sym}}$ by

$$\begin{aligned} (\gamma T)(x, y, S) = T(\gamma ^{-1} x, \gamma ^{-1} y, \gamma ^{-1} S). \end{aligned}$$

If $\Gamma$ acts on a set X, we denote by $X^\Gamma$ the set of elements of X that are invariant under this action. In this way we write ${\mathcal {C}}(V^2)^{\Gamma }$, ${\mathcal {C}}(V^2 \times I_{k-2})_{\succeq 0}^{\Gamma }$, etc.

Given a feasible solution $(\lambda , T)$ of $\Delta _k(G)^*$, the pair $(\lambda , {\overline{T}})$ with

$$\begin{aligned} {\overline{T}}(x,y,S)=\int _\Gamma T(\gamma ^{-1} x, \gamma ^{-1} y, \gamma ^{-1} S) \, \mathop {}\!\mathrm {d}\gamma , \end{aligned}$$

where we integrate against the Haar measure on $\Gamma$ normalized so that the total measure is 1, is also feasible with the same objective value. So we may assume that T is invariant under the action of $\Gamma$.

Let ${\mathcal {R}}_{k-2}$ be a complete set of representatives of the orbits of $I_{k-2}/\Gamma$. For $R \in {\mathcal {R}}_{k-2}$, let $\mathrm {Stab}_{\Gamma }(R) = \{\,\gamma \in \Gamma : \gamma R = R\,\}$ be the stabilizer of R with respect to $\Gamma$ and, for $Q \in \Gamma R$, let $\gamma _Q \in \Gamma$ be a group element such that $\gamma _Q Q = R$. When $I_{k-2}/\Gamma$ is finite, we can decompose the space ${\mathcal {C}}(V^2 \times I_{k-2})^\Gamma$ as a direct sum of simpler spaces.

The next proposition may seem rather technical but the main idea is to use the symmetry of $T \in {\mathcal {C}}(V^2 \times I_{k-2})^\Gamma$ and the assumption that there is just a finite collection of representatives for the last coordinate to write $T(x,y,Q) = T(\gamma _Q x, \gamma _Q y, \gamma _Q Q)$ and express T by finitely many kernels, each of them representing T when its last coordinate is fixed; this is also the place where the stabilizer subgroups come into play.

Proposition 3.1

If $I_{k-2}/\Gamma$ is finite, then

$$\begin{aligned} \Psi :\bigoplus _{R \in {\mathcal {R}}_{k-2}} {\mathcal {C}}(V^2)^{\mathrm {Stab}_{\Gamma }(R)} \rightarrow {\mathcal {C}}(V^2 \times I_{k-2})^\Gamma \end{aligned}$$

given by

$$\begin{aligned} \Psi ((K_R)_{R \in {\mathcal {R}}_{k-2}})(x, y, Q) = K_{\gamma _Q Q}(\gamma _Q x,\gamma _Q y) \end{aligned}$$

is an isomorphism that preserves positivity, that is, if $(K_R)_{R \in {\mathcal {R}}_{k-2}}$ is such that $K_R$ is a positive kernel for each R, then $\Psi ((K_R)_{R \in {\mathcal {R}}_{k-2}})$ is positive.

Proof

We first show that $(V^2 \times I_{k-2}) / \Gamma$ is homeomorphic to the disjoint union

$$\begin{aligned} \bigcup _{R \in {\mathcal {R}}_{k-2}} V^2 / \mathrm {Stab}_{\Gamma }(R) \times \{R\}. \end{aligned}$$

More precisely, we show that

$$\begin{aligned} \psi :\bigcup _{R \in {\mathcal {R}}_{k-2}} V^2 / \mathrm {Stab}_{\Gamma }(R) \times \{R\} \rightarrow (V^2 \times I_{k-2}) / \Gamma \end{aligned}$$

given by $\psi (\mathrm {Stab}_{\Gamma }(R)(x,y),R) = \Gamma (x,y,R)$ is such a homeomorphism with inverse

$$\begin{aligned} \psi ^{-1}(\Gamma (x,y,Q)) = (\mathrm {Stab}_{\Gamma }(\gamma _Q Q)(\gamma _Q x, \gamma _Q y), \gamma _QQ). \end{aligned}$$

(10)

Indeed, the map $\psi$ is well defined because $\Gamma (x,y,R) = \Gamma (\gamma x, \gamma y, R)$ for all $\gamma$ in $\mathrm {Stab}_{\Gamma }(R)$. For each $R \in {\mathcal {R}}_{k-2}$, the map $\psi _R :V^2 / \mathrm {Stab}_{\Gamma }(R) \rightarrow (V^2 \times I_{k-2}) / \Gamma$ given by

$$\begin{aligned} \psi _R\big (\mathrm {Stab}_{\Gamma }(R)(x,y)\big ) = \Gamma (x,y,R) \end{aligned}$$

is continuous, as follows from the definition of quotient topology. By the definition of disjoint union topology, this implies $\psi$ is continuous.

The map (10) is well defined, for if we replace $\gamma _Q$ by $\xi \gamma _Q$, where $\xi \in \mathrm {Stab}_{\Gamma }(\gamma _Q Q)$, then the right-hand side of (10) does not change. Direct verification shows $\psi ^{-1} \circ \psi$ and $\psi \circ \psi ^{-1}$ are the identity maps.

Since ${\mathcal {R}}_{k-2}$ is finite, the domain of $\psi$ is compact. So $\psi$ is a continuous bijection between compact Hausdorff spaces, and hence a homeomorphism.

Now the proposition follows easily. Under the isomorphisms

$$\begin{aligned} {\mathcal {C}}\left( \bigcup _{R \in {\mathcal {R}}_{k-2}} V^2 / \mathrm {Stab}_{\Gamma }(R) \times \{R\}\right) \simeq \bigoplus _{R \in {\mathcal {R}}_{k-2}} {\mathcal {C}}(V^2)^{\mathrm {Stab}_{\Gamma }(R)} \end{aligned}$$

and

$$\begin{aligned} {\mathcal {C}}((V^2 \times I_{k-2}) / \Gamma ) \simeq {\mathcal {C}}(V^2 \times I_{k-2})^\Gamma , \end{aligned}$$

the operator $\Psi$ is equal to

$$\begin{aligned} {\mathcal {C}}\left( \bigcup _{R \in {\mathcal {R}}_{k-2}} V^2 / \mathrm {Stab}_{\Gamma }(R) \times \{R\}\right) \rightarrow {\mathcal {C}}((V^2 \times I_{k-2}) / \Gamma ), \qquad f \mapsto f \circ \psi ^{-1}, \end{aligned}$$

which is a well-defined isomorphism since $\psi$ is a homeomorphism. Finally, it follows directly from the definitions of positive kernels and ${\mathcal {C}}(V^2 \times I_{k-2})^\Gamma _{\succeq 0}$ that $\Psi$ preserves positivity. $\square$

The above proposition shows that to characterize ${\mathcal {C}}(V^2 \times I_{k-2})^\Gamma$ we need to characterize the sets ${\mathcal {C}}(V^2)^{\mathrm {Stab}_{\Gamma }(R)}$ for $R \in {\mathcal {R}}_{k-2}$. In the next section we give this characterization for the case of spherical symmetry.

4 Parameterizing invariant kernels on the sphere by positive semidefinite matrices

From now on we assume $G = (V,E)$ is the graph where $V = S^{n-1}$ and where two distinct vertices x, $y \in S^{n-1}$ are adjacent if $x \cdot y \notin D$ for some $D \subseteq [-1, 1)$. We assume D is closed in order to make G a compact topological packing graph. Taking $\Gamma = \mathrm {O}(n)$, we are in the situation described in the previous section.

We observe that $I_{=m} / \mathrm {O}(n)$ can be represented by the set of $m \times m$ positive semidefinite matrices of rank at most n with ones in the diagonal and elements of D elsewhere, up to simultaneous permutations of the rows and columns. So the condition that $I_{k-2}/\mathrm {O}(n)$ is finite is fulfilled for any set D when $k=2$ or 3 and it only holds for finite D when $k \ge 4$.

Let us see how to parameterize the cones

$$\begin{aligned} {\mathcal {C}}(S^{n-1} \times S^{n-1})_{\succeq 0}^{\mathrm {Stab}_{\mathrm {O}(n)}(R)} \quad \text {for } R \in {\mathcal {R}}_{k-2} \end{aligned}$$

by positive semidefinite matrices. For simplicity, we only consider the case where every $R \in {\mathcal {R}}_{k-2}$ consists of linearly independent vectors; later on we will see that all cases considered in the computations satisfy this assumption.

Let $\{e_1,\ldots ,e_n\}$ be the standard basis of ${\mathbb {R}}^n$ and fix $R \in {\mathcal {R}}_{k-2}$. By rotating a set $R \in {\mathcal {R}}_{k-2}$ if necessary, we may assume that R is contained in $\mathrm {span}(\{e_1,\ldots , e_m\})$, where $m = \dim (\mathrm {span}(R))$. The stabilizer subgroup of $\mathrm {O}(n)$ with respect to R is isomorphic to the direct product of two groups, namely

$$\begin{aligned} \mathrm {Stab}_{\mathrm {O}(n)}(R) \simeq {\mathcal {S}}_R \times \mathrm {Stab}_{\mathrm {O}(n)}(\mathrm {span}(R)), \end{aligned}$$

where ${\mathcal {S}}_R$ is isomorphic to a finite subgroup of $\mathrm {O}(m)$ that acts on the first m coordinates and acts on R as a permutation of its elements and $\mathrm {Stab}_{\mathrm {O}(n)}(\mathrm {span}(R))$ is a group isomorphic to $\mathrm {O}(n-m)$ that acts on the last $n-m$ coordinates. Indeed, any rotation that leaves $\mathrm {span}(R)$ and its orthogonal complement invariant and acts in R as a permutation fixes R as a set and hence is from $\mathrm {Stab}_{\mathrm {O}(n)}(R)$. Conversely, any rotation that fixes R as a set will at most permute its elements and hence by linearity, leaves $\mathrm {span}(R)$ invariant; while by orthogonality, such a rotation also leaves the orthogonal complement invariant and hence is of the prescribed form.

If $k=2$, then $R = \emptyset$ and $\mathrm {Stab}_{\mathrm {O}(n)}(\mathrm {span}(R)) = \mathrm {O}(n)$. By a classical result of Schoenberg [43], each positive $\mathrm {O}(n)$-invariant kernel $K :S^{n-1} \times S^{n-1} \rightarrow {\mathbb {R}}$ is of the form

$$\begin{aligned} K(x,y) = \sum _{l=0}^\infty a_l P_l^n(x \cdot y) \end{aligned}$$

for some nonnegative numbers $a_0$, $a_1$, ... with absolute and uniform convergence, where $P_l^n$ is the Gegenbauer polynomial of degree l in dimension n normalized so that $P_l^n(1) = 1$ (equivalently, $P_l^n$ is the Jacobi polynomial with both parameters equal to $(n-3)/2$).

Kernels invariant under the stabilizer of one point were considered by Bachoc and Vallentin [2] and kernels invariant under the stabilizer of more points were considered by Musin [37]. The analogue of Schoenberg’s theorem for kernels invariant under the stabilizer of one or more points is stated in terms of certain polynomials $P_l^{n,m}$, which were called by Musin [37] “multivariate Gegenbauer polynomials”.

For $0 \le m \le n - 2$, $t \in {\mathbb {R}}$, and u, $v \in {\mathbb {R}}^m$, the polynomial $P_l^{n,m}$ is the $(2m+1)$-variable polynomial

$$\begin{aligned} P_l^{n, m}(t,u,v) = \big ((1-\Vert u\Vert ^2)(1-\Vert v\Vert ^2)\big )^{l/2}P_l^{n-m}\left( \frac{t-u \cdot v}{\sqrt{(1-\Vert u\Vert ^2)(1-\Vert v\Vert ^2)}}\right) , \end{aligned}$$

where $\Vert v\Vert = \sqrt{v \cdot v}$. If we use the convention ${\mathbb {R}}^0 = \{0\}$, then $P_l^n(t) = P_l^{n,0}(t,0,0)$. Since the Gegenbauer polynomials are odd or even according to the parity of l, the function $P_l^{n, m}(t,u,v)$ is indeed a polynomial in the variables u, v, and t. Musin [37] denotes $P_l^{n,m}$ by $G_l^{(n,m)}$ and Bachoc and Vallentin [2] denote $P_l^{n,1}$ by $Q_l^{n-1}$.

Fix $d \ge 0$, let ${\mathcal {B}}_l$ be a basis of the space of m-variable polynomials of degree at most l (e.g. the monomial basis), and write $z_l(u)$ for the column vector containing the polynomials in ${\mathcal {B}}_l$ evaluated at $u \in {\mathbb {R}}^m$. Let $Y_l^{n, m}$ be the matrix of polynomials

$$\begin{aligned} Y_l^{n,m}(t,u,v) = P_l^{n,m}(t,u,v) z_{d-l}(u)z_{d-l}(v)^\mathsf{T}. \end{aligned}$$

The choice of d makes $Y_l^{n,m}$ a $\left( {\begin{array}{c}d-l+m\\ m\end{array}}\right) \times \left( {\begin{array}{c}d-l+m\\ m\end{array}}\right)$ matrix with $(2m+1)$-variable polynomials of degree at most 2d as its entries.

Given a matrix X with linearly independent columns, set $L(X) = B^{-1} X^\mathsf{T}$, where B is the matrix such that $BB^\mathsf{T}$ is the Cholesky factorization of $X^\mathsf{T}X$, which is unique since $X^\mathsf{T}X$ is positive definite. For each $R \in \mathcal R_{k-2}$, fix a matrix $A_R$ whose columns are the vectors of R in some order. The rows of $L(A_R)$ span the same space as the columns of $A_R$ because B is invertible, and by construction the rows of $L(A_R)$ are orthonormal:

$$\begin{aligned} L(A_R) L(A_R)^\mathsf{T} = B^{-1} A_R^\mathsf{T} A_R B^{-\mathsf T} = B^{-1} BB^\mathsf{T} B^{-\mathsf T} = I. \end{aligned}$$

Therefore, for $x \in {\mathbb {R}}^n$, $L(A_R)x$ is a vector with the coordinates of the projection of x onto $\mathrm {span}(R)$ with respect to an orthonormal basis of the linear span.

The following theorem is a restatement of a result of Musin [37, Corollary 3.2] in terms of invariant kernels and with adapted notation. We will only use the sufficiency part of the statement, proved in Appendix A for completeness.

For square matrices A, B of the same dimensions, write $\langle A, B \rangle = {{\,\mathrm{tr}\,}}(A^\mathsf{T}B)$ for their Frobenius inner product.

Theorem 4.1

Let $R \subseteq S^{n-1}$ with $m = \dim (\mathrm {span}(R)) = |R| \le n-2$ and let $A_R$ be a matrix whose columns are the vectors of R in some order. Fix $d \ge 0$ and, for each $0 \le l \le d$, let $F_l$ be a positive semidefinite matrix with $\left( {\begin{array}{c}d-l+m\\ m\end{array}}\right)$ rows and columns. Then $K :S^{n-1} \times S^{n-1} \rightarrow {\mathbb {R}}$ given by

$$\begin{aligned} K(x,y) = \sum _{l = 0}^d \big \langle F_l, Y_l^{n,m}(x \cdot y, L(A_R) x, L(A_R) y) \big \rangle \end{aligned}$$

(11)

is a positive, continuous, and $\mathrm {Stab}_{\mathrm {O}(n)}(\mathrm {span}(R))$-invariant kernel. Conversely, every $\mathrm {Stab}_{\mathrm {O}(n)}(\mathrm {span}(R))$-invariant kernel $K \in {\mathcal {C}}(S^{n-1} \times S^{n-1})_{\succeq 0}$ can be uniformly approximated by kernels of the above form.

Theorem 4.1 gives us a parameterization of $\mathrm {Stab}_{\mathrm {O}(n)}(\mathrm {span}(R))$-invariant kernels. To get a parameterization of $\mathrm {Stab}_{\mathrm {O}(n)}(R)$-invariant kernels we still have to deal with the symmetries in ${\mathcal {S}}_R$. By construction, for an orthogonal matrix $\sigma \in {\mathcal {S}}_R$ there is a permutation matrix $P_\sigma$ such that $\sigma A_R = A_R P_\sigma$. Since $\sigma \in \mathrm {O}(n)$ and $A_R^\mathsf{T}A_R = A_R^\mathsf{T}\sigma ^\mathsf{T}\sigma A_R = P_\sigma ^\mathsf{T}A_R^\mathsf{T}A_RP_\sigma$, the elements of ${\mathcal {S}}_R$ correspond precisely to the symmetries of the Gram matrix $A_R^\mathsf{T}A_R$ under simultaneous permutations of rows and columns. Indeed, if the Gram matrix $A_R^\mathsf{T}A_R$ is invariant under a certain simultaneous permutation of rows and columns, then since R is linearly independent, this permutation defines a linear transformation of $\mathrm {span}(R)$ that preserves all inner products between vectors of R, whence it is orthogonal and therefore corresponds to an element of ${\mathcal {S}}_R$. This observation leads to the following corollary.

Corollary 4.2

Let $R \subseteq S^{n-1}$ with $m = \dim (\mathrm {span}(R)) = |R| \le n-2$ and let $A_R$ be a matrix whose columns are the vectors of R in some order. Fix $d \ge 0$ and, for each $0 \le l \le d$, let $F_l$ be a positive semidefinite matrix with $\left( {\begin{array}{c}d-l+m\\ m\end{array}}\right)$ rows and columns. Then $K :S^{n-1} \times S^{n-1} \rightarrow {\mathbb {R}}$ given by

$$\begin{aligned} K(x,y) = \sum _{l = 0}^d \big \langle F_l,{\mathcal {F}}_l(R)(x,y) \big \rangle , \end{aligned}$$

(12)

where

$$\begin{aligned} {\mathcal {F}}_l(R)(x,y) = \frac{1}{|{\mathcal {S}}_R|} \sum _{\sigma \in {\mathcal {S}}_R} Y_l^{n,m}(x \cdot y, L(A_R P_\sigma ) x, L(A_RP_\sigma )y), \end{aligned}$$

is a positive, continuous, and $\mathrm {Stab}_{\mathrm {O}(n)}(R)$-invariant kernel.

Proof

If K is given by (12), then by writing

$$\begin{aligned} K(x,y) = \frac{1}{|{\mathcal {S}}_R|} \sum _{\sigma \in {\mathcal {S}}_R} \sum _{l = 0}^d \big \langle F_l, Y_l^{n,m}(x \cdot y, L(A_R P_\sigma ) x, L(A_R P_\sigma ) y) \big \rangle \end{aligned}$$

we see using Theorem 4.1 that K is a sum of $|{\mathcal {S}}_R|$ positive, continuous, and $\mathrm {Stab}_{\mathrm {O}(n)}(\mathrm {span}(R))$-invariant kernels, and hence it is itself positive, continuous, and $\mathrm {Stab}_{\mathrm {O}(n)}(\mathrm {span}(R))$-invariant.

Since, for every $\sigma \in {\mathcal {S}}_R$,

$$\begin{aligned} L(A_RP_{\sigma })x = B^{-1}P_{\sigma }^\mathsf{T}A_R^\mathsf{T}x = B^{-1}A_R^\mathsf{T} \sigma ^\mathsf{T} x = L(A_R)\sigma ^\mathsf{T} x \end{aligned}$$

(recall $BB^\mathsf{T}$ is the Cholesky decomposition of $A_R^\mathsf{T} A_R = (A_R P_\sigma )^\mathsf{T}(A_R P_\sigma )$), and since $x \cdot y = (\sigma ^\mathsf{T} x) \cdot (\sigma ^\mathsf{T} y)$, we have that

$$\begin{aligned} K(x,y) = \frac{1}{|{\mathcal {S}}_R|} \sum _{\sigma \in {\mathcal {S}}_R} K'(\sigma ^\mathsf{T} x, \sigma ^\mathsf{T} y), \end{aligned}$$

(13)

where

$$\begin{aligned} K'(x,y) = \sum _{l = 0}^d \big \langle F_l, Y_l^{n,m}(x \cdot y, L(A_R) x, L(A_R) y) \big \rangle . \end{aligned}$$

Now it follows directly from (13) that K is $\mathrm {Stab}_{\mathrm {O}(n)}(R)$-invariant. $\square$

5 Semidefinite programming formulations

Before giving the semidefinite programming formulations, let us discuss how the matrix-valued function ${\mathcal {F}}_l(R)(x,y)$ can be computed. We have

$$\begin{aligned} L(A_R P_\sigma ) x = B^{-1} P_\sigma ^\mathsf{T} A_R^\mathsf{T} x = B^{-1} P_\sigma ^\mathsf{T} (A_R^\mathsf{T} x), \end{aligned}$$

where $BB^\mathsf{T}$ is the Cholesky decomposition of $A_R^\mathsf{T} A_R = (A_R P_\sigma )^\mathsf{T}(A_R P_\sigma )$. This shows that $L(A_R P_\sigma ) x$ depends only on the inner products between the vectors in the set $R \cup \{x\}$ and on the ordering of the columns of $A_R$. Since the size of R is bounded by $k-2$, this also shows that all computations for setting up the problem can be done in a relatively small dimension depending on k and not on n.

5.1 An SDP formulation for spherical finite-distance sets

To write the full semidefinite programming formulation corresponding to (9), we use Corollary 4.2 together with the isomorphism from Proposition 3.1. Let ${\mathcal {S}}_{\succeq 0}^N$ denote the cone of $N \times N$ positive semidefinite matrices. If for $R \in {\mathcal {R}}_{k-2}$ and $0 \le l\le d$ we have $F_{R, l} \in {\mathcal {S}}_{\succeq 0}^N$, where $N = \left( {\begin{array}{c}d-l+|R|\\ |R|\end{array}}\right)$, then $T :S^{n-1} \times S^{n-1} \times I_{k-2} \rightarrow {\mathbb {R}}$ given by

$$\begin{aligned} T(x,y,Q) = \sum _{l = 0}^d \big \langle F_{\gamma _Q Q, l}, {\mathcal {F}}_l(\gamma _QQ)(\gamma _Qx,\gamma _Qy) \big \rangle \end{aligned}$$

is a function in ${\mathcal {C}}(S^{n-1} \times S^{n-1} \times I_{k-2})_{\succeq 0}^{\mathrm {O}(n)}$ and hence, for $S \in {\mathcal {R}}_{k} {\setminus } \{\emptyset \}$, the expression for $B_k T(S)$ becomes

$$\begin{aligned} B_k T(S)&= \sum _{\begin{array}{c} Q \subseteq S \\ |Q| \le k-2 \end{array}} \; \sum _{\begin{array}{c} x, y \in S \\ \{x, y\} \cup Q = S \end{array}} T(x, y, Q)\\&= \sum _{\begin{array}{c} Q \subseteq S \\ |Q| \le k-2 \end{array}} \; \sum _{\begin{array}{c} x, y \in S \\ \{x, y\} \cup Q = S \end{array}} \sum _{l = 0}^d \big \langle F_{\gamma _Q Q, l},{\mathcal {F}}_l(\gamma _QQ)(\gamma _Qx,\gamma _Qy) \big \rangle \\&= \sum _{\begin{array}{c} Q \subseteq S \\ |Q| \le k-2 \end{array}}\; \sum _{l = 0}^d \big \langle F_{\gamma _Q Q, l}, \sum _{\begin{array}{c} x, y \in S\\ \{x, y\} \cup Q = S \end{array}} {\mathcal {F}}_l(\gamma _Q Q)(\gamma _Q x,\gamma _Q y) \big \rangle . \end{aligned}$$

Since the action of $\mathrm {O}(n)$ on $S^{n-1} \simeq I_{=1}$ is transitive, the quotient $I_{=1} / \mathrm {O}(n)$ has only one element. We set $\mathcal R_1 {\setminus } {\mathcal {R}}_0 = \{e_1\}$, where $e_1$ is the first canonical basis vector of ${\mathbb {R}}^n$. We replace the objective $1+\lambda$ in (9) by $1 + B_kT(\{e_1\})$, which we can further simplify by noticing that $Y_0^{n,1}(1,1,1)$ is the all-ones matrix $J_{d+1}$ of size $(d+1) \times (d+1)$ and $Y_l^{n,1}(1,1,1)$ is the zero matrix for $l > 0$. This gives the semidefinite programming formulation

$$\begin{aligned}&\min \bigg \{\, 1 + \sum _{l = 0}^d F_{\emptyset ,l} + \langle F_{\{e_1\},0}, J_{d+1}\rangle : F_{R, l} \in {\mathcal {S}}_{\succeq 0}^{\left( {\begin{array}{c}d-l+|R|\\ |R|\end{array}}\right) } \text { for } 0 \le l\le d \text { and } R \in {\mathcal {R}}_{k-2},\\&\quad \sum _{\begin{array}{c} Q \subseteq S \\ |Q| \le k-2 \end{array}} \sum _{l = 0}^d \big \langle F_{\gamma _Q Q, l},\sum _{\begin{array}{c} x,y \in S \\ \{x, y\} \cup Q = S \end{array}}{\mathcal {F}}_l(\gamma _Q Q)(\gamma _Q x,\gamma _Q y)\big \rangle \le -2 \chi _{I_{=2}}(S) \text { for } S \in {\mathcal {R}}_k {\setminus } {\mathcal {R}}_1\,\bigg \}. \end{aligned}$$

For each fixed d this gives an upper bound for $\Delta _k(G)^*$ that converges to $\Delta _k(G)^*$ as d tends to infinity.

We give an efficient Julia [7] implementation to generate the semidefinite programming input files for the solver, which was essential to make computations with $k = 6$. This includes an efficient function for generating the representatives of the independent sets, a function for checking whether two sets of vectors are in the same orbit, an implementation of the function ${\mathcal {F}}$ that works entirely in dimension k, and finally a function for setting up the semidefinite programming problems, which works for general n, finite D, and k.

5.2 A precise connection between the Bachoc–Vallentin bound and the Lasserre hierarchy

The bound $\Delta _2(G)^*$ immediately reduces to the generalization of the Lovász $\vartheta$ number as given by Bachoc et al. [1], which coincides with the LP bound [13] after symmetry reduction. Here we show that $\Delta _3(G)^*$ can be interpreted as a nonsymmetric version of the Bachoc–Vallentin 3-point bound [2].

Suppose T is feasible for $\Delta _3(G)^*$. If $S = \{a, b\}$ with $a\ne b$, then

$$\begin{aligned} B_3T(\{a, b\})&= \sum _{\begin{array}{c} Q \subseteq S\\ |Q| \le 1 \end{array}} \; \sum _{\begin{array}{c} x, y \in S \\ Q \cup \{x,y\} = S \end{array}} T(x,y,Q)\\&= T(a,b,\emptyset ) + T(b,a,\emptyset ) + T(a,b,\{a\}) + T(b,a,\{a\})\\&\quad + T(b,b,\{a\}) + T(a,b,\{b\}) + T(b,a,\{b\}) + T(a,a,\{b\}). \end{aligned}$$

By using $T(x,y,\emptyset ) = \sum _{l = 0}^d F_{\emptyset ,l} P_l^n(x \cdot y)$ and

$$\begin{aligned} T(x, y, \{z\})&= \sum _{l = 0}^d\big \langle F_{\{e_1\}, l}, {\mathcal {F}}_l\big (\{e_1\}\big )(\gamma _{\{z\}} x, \gamma _{\{z\}}y)\big \rangle \\&= \sum _{l = 0}^d\big \langle F_{\{e_1\}, l}, Y_l^{n,1}(x \cdot y, x \cdot z, y \cdot z)\big \rangle , \end{aligned}$$

we see that

$$\begin{aligned} B_3T(\{a, b\}) = 2\sum _{l = 0}^d F_{\emptyset ,l} P_l^n(a \cdot b) + 6\sum _{l = 0}^d\big \langle F_{\{e_1\}, l}, S_l^n(a\cdot b, a\cdot b, 1)\big \rangle , \end{aligned}$$

where we use the notation $S_l^n = \frac{1}{6}\sum _{\sigma \in S_3} \sigma Y_l^{n, 1}$, in which $\sigma$ runs through the group of all permutations of three variables and acts on $Y_l^{n, 1}$ by permuting its arguments.

If $|S| = 3$, say $S = \{a, b, c\}$, then

$$\begin{aligned} B_3T(\{a, b, c\})&= \sum _{\begin{array}{c} Q \subseteq S\\ |Q| \le 1 \end{array}}\; \sum _{\begin{array}{c} x, y \in S\\ Q \cup \{x,y\} = S \end{array}} T(x,y,Q)\\&= T(a,b,\{c\}) + T(b,a,\{c\}) + T(a,c,\{b\})\\&\quad + T(c,a,\{b\}) + T(b,c,\{a\}) + T(c,b,\{a\})\\&= 6\sum _{l = 0}^d\big \langle F_{\{e_1\}, l}, S_l^n(a \cdot b, a \cdot c, b \cdot c)\big \rangle . \end{aligned}$$

Using the above expressions we see that the constraints $B_3T(S) \le -2$ for $S \in I_{= 2}$ and $B_3T(S) \le 0$ for $S \in I_{= 3}$ in $\Delta _3(G)^*$ are exactly the ones that appear in Theorem 4.2 of Bachoc and Vallentin [2]. Except for the presence of an ad hoc $2\times 2$ matrix variable b that comes from a separate argument, both bounds are identical.

Remark 5.1

Recall that for our method it is essential that $I_{k-2}/\mathrm {O}(n)$ be finite and that $I_{=m} / \mathrm {O}(n)$ can be represented by the set of $m \times m$ positive semidefinite matrices of rank at most n with ones in the diagonal and elements of D elsewhere, up to simultaneous permutations of the rows and columns. So $I_{k-2}/\mathrm {O}(n)$ is finite for $k=2$, 3, but infinite whenever D is infinite and $k \ge 4$. This explains why it is not clear how to compute a 4-point bound generalization of the LP [13] and SDP [2] bounds for the size of spherical codes with given minimal angular distance. For the spherical finite-distance problem, however, the set $I_{k-2}/\mathrm {O}(n)$ is always finite, so that we can perform computations beyond $k=3$.

6 Two-distance sets and equiangular lines

If $D = \{a, -a\}$ for some $0< a < 1$, then the vectors in a spherical D-code correspond to a set of equiangular lines with common angle $\arccos a$. We set

$$\begin{aligned} M_a(n) = A\big (n, \{a,-a\}\big ) \end{aligned}$$

and write

$$\begin{aligned} M(n) = \max _{0< a < 1} M_a(n) \end{aligned}$$

for the maximum number of equiangular lines in ${\mathbb {R}}^n$ with any common angle.

A semidefinite programming bound based on the method of Bachoc and Vallentin [2], and hence equivalent to $\Delta _3(G)^*$, was applied to this problem by Barg and Yu [5] (see also the table computed by King and Tang [27]) which, together with previous results, determines M(n) for most $n \le 43$.

Barg and Yu present [4, Eqs. (14)–(17)] a semidefinite programming formulation that corresponds exactly to the formulation given in Sect. 5.1 when $k = 3$ (except for an ad hoc $2 \times 2$ matrix). In the other papers [5, 27, 41, 48] where this semidefinite program is considered, a primal version is given instead, which is less convenient from the perspective of rigorous verification of bounds.

In this paper we compute new upper bounds for $M_a(n)$ for $a = 1/5$, 1/7, 1/9, and 1/11 and many values of n using $\Delta _k(G)^*$ with $k = 4$, 5, and 6. The results do not improve the known bounds for M(n) but greatly improve the known bounds for $M_a(n)$ for certain ranges of dimensions; these results are presented in Sect. 6.2.

6.1 Overview of the literature

The literature on equiangular lines is vast; here is a summary.

6.1.1 Bounds for M(n)

The interest in M(n) started with Haantjes [24], who showed $M(3) = M(4) = 6$ in 1948. Since then, much progress has been made using different techniques, and M(n) has been determined for many values of $n \le 43$. Table 1 presents the known values for M(n) for small dimensions.

The most general bound for M(n), called the absolute bound, is due to Gerzon:

Theorem 6.1

(Gerzon, cf. Lemmens and Seidel [31]) We have

$$\begin{aligned} M(n) \le \frac{n(n+1)}{2}. \end{aligned}$$

Moreover, if equality holds, then $n = 2$, $n=3$, or $n = l^2 - 2$ for some odd integer l and the cosine of the common angle is $a = 1/l$.

The four cases where it is known that the bound is attained are $n = 2$, 3, 7, and 23. Delsarte et al. [13, Example 8.3] show that equality holds if and only if the union of the code with its antipodal code is a tight spherical 5-design, and in this case Cohn and Kumar [9] show this union is a universally optimal code (which means it minimizes every completely monotonic potential function in the squared chordal distance). Bannai et al. [3] and Nebe and Venkov [39] show that there are infinitely many odd integers l for which no tight spherical 5-design exists in $S^{n-1}$ with $n = l^2 - 2$, so that Gerzon’s bound cannot be attained in those dimensions. This list starts with $l = 7$, 9, 13, 21, 25, 45, 57, 61, 69, 85, 93, ... (resp. $n = 47$, 79, 167, 439, 623, 2023, 3247, 3719, 4759, 7223, 8647, ...). For the remaining possible dimensions, attainability is an open problem.

For the dimensions that are not of the form $l^2 - 2$ for some odd integer l, the absolute bound can be improved:

Theorem 6.2

(Glazyrin and Yu [18] and King and Tang [27]) Let l be the unique odd integer such that $l^2-2 \le n \le (l+2)^2-3$. Then,

$$\begin{aligned} M(n) \le {\left\{ \begin{array}{ll} \displaystyle \frac{n(l+1)(l+3)}{(l+2)^2-n}, &{} n = 44,45,46,76,77,78,117,118,166,222,286,358;\\ \displaystyle \frac{(l^2-2)(l^2-1)}{2}, &{} \text {for all other } n \ge 44. \end{array}\right. } \end{aligned}$$

Furthermore, if the bound is attained, then the cosine of the angle between the lines is $a = 1/(l+2)$ for the first case and $a = 1/l$ for the second.

Glazyrin and Yu also proved another theorem [18, Theorem 4] about the codes that attain the bound from Theorem 6.2:

Theorem 6.3

(Glazyrin and Yu [18]) Suppose l is a positive odd integer. If X is a $\{1/l,-1/l\}$-spherical code of size $(l^2-2)(l^2-1)/2$ contained in $S^{n-1}$ with $n \le 3l^2-16$, then X must belong to a $(l^2-2)$-dimensional subspace.

Since $(l+2)^2-3 \le 3l^2-16$ for $l \ge 5$, this last theorem implies that if the second bound from Theorem 6.2 is attained, then Gerzon’s bound also has to be attained for $n = l^2-2$. For the first two cases where tight spherical 5-designs do not exist, this implies $M(n) \le 1127$ for $47 \le n \le 75$ and $M(n) \le 3159$ for $79 \le n \le 116$. The following theorem is adapted from Larman, Rogers, and Seidel [28, Theorem 2]:

Theorem 6.4

(Larman et al. [28]) We have

$$\begin{aligned} M(n) \le \max \{2n+3, M_{1/3}(n), M_{1/5}(n), \ldots , M_{1/l}(n)\}, \end{aligned}$$

where l is the largest odd integer such that $l \le \sqrt{2n}$.

Most of the results for M(n) rely on Theorem 6.4, which shows that to bound M(n) one just has to consider finitely many angles. This motivates the consideration of $M_a(n)$ when 1/a is an odd integer.

6.1.2 Bounds for $M_a(n)$

Bounds for fixed a are known as relative bounds, as opposed to Gerzon’s absolute bound from Theorem 6.1. The first relative bound is due to van Lint and Seidel [47]:

Theorem 6.5

(van Lint and Seidel [47]) If $n < 1/a^2$, then

$$\begin{aligned} M_a(n) \le \frac{n(1-a^2)}{1 - na^2}. \end{aligned}$$

As shown by Glazyrin and Yu [18, Theorem 5], Theorem 6.5 can be derived from the positivity of the Gegenbauer polynomials $P_2^n$, and indeed this is the bound given by the semidefinite programming techniques when $n \le 1/a^2 - 2$. This bound is also the first case of Theorem 6.2.

After computing the semidefinite programming bound for many values of n and a, Barg and Yu [5] observed long ranges $1/a^2-2\le n \le 3/a^2 - 16$ where the bound remained stable, matching Gerzon’s bound (Theorem 6.1) at $n = 1/a^2 - 2$. Based on this observation, Yu [48] proved the following theorem:

Theorem 6.6

rm (Yu [48]) If $n \le 3/a^2 - 16$ and $a \le 1/3$, then

$$\begin{aligned} M_a(n) \le \frac{(1/a^2-2)(1/a^2-1)}{2}. \end{aligned}$$

An alternative proof for the previous theorem is given by Glazyrin and Yu [18, Theorem 6], where the use of the positivity of the Gegenbauer polynomials $P_1^{n-1}$ and $P_3^{n-1}$ is made more explicit. The bounds given by the semidefinite programming method were also used to prove the following theorem:

Theorem 6.7

(Okuda and Yu [41]) If $3/a^2 - 16 \le n \le 3/a^2+6/a+1$, then

$$\begin{aligned} M_a(n) \le 2 + \frac{(n-2)(1/a+1)^3}{(3/a^2-6/a+2)-n}. \end{aligned}$$

The bounds from Theorems 6.5, 6.6, and 6.7 coincide with the values given by the semidefinite programming formulation when $k = 3$ (see the points labeled “$\Delta _3(G)^*$ [5, 27]” in Figs. 1, 2, 3 and 4). Another source of relative bounds is a technique called pillar decomposition, introduced by Lemmens and Seidel [31] and used to determine $M_{1/3}(n)$:

Theorem 6.8

(Lemmens and Seidel [31]) If $n \ge 15$, then

$$\begin{aligned} M_{1/3}(n) = 2n - 2. \end{aligned}$$

For $a = 1/5$, they obtained results that lead to the following conjecture:

Conjecture 6.9

(Lemmens and Seidel [31]) We have

$$\begin{aligned} M_{1/5}(n) = {\left\{ \begin{array}{ll} 276 &{} \quad \text {for } 23\le n\le 185;\\ \lfloor \frac{3}{2}(n-1)\rfloor &{}\quad \text {for } n \ge 185. \end{array}\right. } \end{aligned}$$

Note that 276 is the bound given by Theorem 6.6 when $a = 1/5$ and this shows (together with the fact that there exists a $\{-1/5, 1/5\}$-code of size 276 in dimension $n = 23$) that the conjecture is true for $n \le 59$. In fact, the semidefinite programming bound computed by Barg and Yu [5] also shows $M_{1/5}(60) = 276$. Neumaier [40] (see also [20, Corollary 6.6]) proved that there exists a large N such that $M_{1/5}(n) = \lfloor \frac{3}{2}(n-1)\rfloor$ for all $n > N$. Neumaier claimed, without a proof, that N should be at most 30251.

Recently, Lin and Yu [33] made progress in this conjecture by proving some claims from Lemmens and Seidel [31]. The only case still open is when the code has a set with 4 unit vectors with mutual inner products $-1/5$ and no such set with 5 unit vectors (up to replacement of some vectors by their antipodes).

Glazyrin and Yu [18] introduced a new method to derive upper bounds for spherical finite-distance sets. By using Gegenbauer polynomials together with the polynomial method, they proved a theorem that, specialized for two-distance sets, is:

Theorem 6.10

(Glazyrin and Yu [18]) For all a, b, and n, we have

$$\begin{aligned} A\big (n, \{a,b\}\big ) \le \frac{n+2}{1 - (n-1)/(n(1-a)(1-b))} \end{aligned}$$

whenever the right-hand side is positive.

With this result, they proved the following relative bound, which provides the best bounds for moderately large values of n (see Figs. 2, 3 and 4):

Theorem 6.11

(Glazyrin and Yu [18]) If $a \le 1/3$, then

$$\begin{aligned} \begin{aligned} M_a(n)&\le n\bigg (\frac{(a^{-1}-1)(a^{-1}+2)^2}{3a^{-1}+5} + \frac{(a^{-1}+1)(a^{-1}-2)^2}{3a-5}+2\bigg )+2\\&\le n\bigg (\frac{2}{3}a^{-2}+\frac{4}{7}\bigg )+2. \end{aligned} \end{aligned}$$

King and Tang [27] improved the pillar decomposition technique and got a better bound for $M_{1/5}(n)$ [27, Theorem 7]. Recently, Lin and Yu [33] further improved parts of their argument; by combining [33, Proposition 4.5] with the proof of [27, Theorem 7] we get:

Theorem 6.12

(Lin and Yu [33]) If $n \ge 63$, then

$$\begin{aligned} M_{1/5}(n) \le 100 + 3 A(n-4, \{1/13, -5/13\}). \end{aligned}$$

The previous results give three competing methods to bound $M_{1/5}(n)$, each one being the best for a different range of dimensions. One can either use semidefinite programming to bound $M_{1/5}(n)$ directly, use Theorem 6.12 together with semidefinite programming to bound $A\big (n-4, \{1/13, -5/13\}\big )$, or use Theorem 6.10. King and Tang [27] made this comparison, computing the semidefinite programming bound $\Delta _3(G)^*$. See in Table 2 and in Fig. 1 the comparison with the new semidefinite programming bound $\Delta _6(G)^*$.

Regarding asymptotic results, while it is known that M(n) is asymptotically quadratic in n (a quadratic lower bound in which the cosine of the angle between the lines, a, tends to zero as n increases can be found in [20, Corollary 2.8], while Theorem 6.1 gives a quadratic upper bound), for fixed a we have that $M_a(n)$ is linear in n. Bukh [8] was the first to show a bound for $M_a(n)$ of the form $M_a(n) \le c n$, although with a large constant c. Theorem 6.11 has another linear bound good to give results for intermediate values of n, while the best asymptotic result is due to Jiang et al. [25]. They completely settled the value of $\lim _{n \rightarrow \infty } M_a(n) / n$ for every a in terms of a parameter called the spectral radius order $r(\lambda )$, which is defined for $\lambda > 0$ as the smallest integer r so there exists a graph with r vertices and adjacency matrix with largest eigenvalue exactly $\lambda$, and is defined $r(\lambda ) = \infty$ in case no such graph exists.

Theorem 6.13

(Jiang et al. [25]) Fix $0< a < 1$. Let $\lambda = (1-a)/(2a)$ and $r = r(\lambda )$ be its spectral radius order. The maximum number $M_a(n)$ of equiangular lines in ${\mathbb {R}}^n$ with common angle $\arccos a$ satisfies

• (a) $M_a(n) = \lfloor r(n-1)/(r-1) \rfloor$ for all sufficiently large $n > n_0(a)$ if $r < \infty$.

• (b) $M_a(n) = n + o(n)$ as $n \rightarrow \infty$ if $r = \infty$.

Jiang et al. remarks that the $n_0(a)$ from their theorem may be really big, though. When $a = 1/(2r-1)$ for some positive integer r, then $\lambda = r-1$ and $r(\lambda ) = r$ (since the complete graph on r vertices has spectral radius $r-1$). Theorem 6.13 confirms a conjecture made by Bukh [8]:

Corollary 6.14

(Jiang et al. [25]) If $a = 1/(2r-1)$ for some positive integer $r \ge 2$, then for all n sufficiently large,

$$\begin{aligned} M_{a}(n) = \left\lfloor \frac{r(n-1)}{r-1} \right\rfloor . \end{aligned}$$

There is a simple construction that achieves the value from Corollary 6.14. Let $a = 1/(2r-1)$ for some positive integer r and t, s be arbitrary positive integers. Then one can show that a matrix with t diagonal blocks, each of size r, and s diagonal blocks of size 1, with diagonal entries equal to 1, off-diagonal entries inside each block equal to $-a$, and all other entries equal to a is the Gram matrix of a $\{-a,a\}$-code in $S^{(r-1)t+s}$ of size $rt+s$. Letting $t = \lfloor (n-1)/(r-1) \rfloor$ and $s = n-1 - (r-1)\lfloor (n-1)/(r-1) \rfloor$ we get the desired size.

6.2 New semidefinite programming bounds

As observed in Sect. 6, the semidefinite programming bounds computed by Barg and Yu [5] and King and Tang [27] correspond to $\Delta _3(G)^*$. In this paper we compute new upper bounds for $M_a(n)$ for $a = 1/5$, 1/7, 1/9, and 1/11 and many values of n using $k = 4$, 5, and 6. Since every two-distance set with these angles and at most $k-2 \le 4$ vectors is linearly independent, the assumption made in Sect. 4 is satisfied. We always use degree $d = 5$ for the polynomials since, as reported by Barg and Yu [4], no improvement is observed for larger values of d (but this may change if sets D with cardinality greater than 2 are considered). The semidefinite programs were produced using a script written in Julia [7] using Nemo [15], were solved with SDPA-GMP [38], and the results were rigorously verified using the interval arithmetic library Arb [26]. The rigorous verification procedure is much simpler than that for similar problems [14]. The scripts used to generate the programs and verify the results can be found with the arXiv version of this paper.

Fig. 1

Relative bounds for $M_{1/5}(n)$. In fact, King and Tang [27] computed a bound using $\Delta _3(G)^*$ together with a theorem [27, Theorem 7] weaker than Theorem 6.12; the result is similar though

Fig. 2

Relative bounds for $M_{1/7}(n)$

Fig. 3

Relative bounds for $M_{1/9}(n)$

Fig. 4

Relative bounds for $M_{1/11}(n)$

The results are presented in Figs. 1, 2, 3 and 4 and Tables 2, 3, 4 and 5, where we compile the bounds for $M_a(n)$ for each n that is a multiple of 5; the best bounds are displayed in boldface. While it takes only a few seconds to generate and solve a single instance of the semidefinite programming problem for $k = 3$, the process takes about 5 days using a single core of an Intel i7-8650U processor for $k = 6$; that is why the tables have some missing values for $\Delta _6(G)^*$.

No improvements were obtained for $n \le 3/a^2 -16$; we observed in this case that $\Delta _6(G)^* = \Delta _3(G)^*$ which is equal to the values given by Theorems 6.5 and 6.6. Since this is the range of dimensions that influences M(n), no improvements for M(n) were obtained. We obtained great improvements for all dimensions $n > 3/a^2 -16$, making the semidefinite programming bound competitive with the other methods (like Theorem 6.11) for more dimensions. Asymptotically, the semidefinite programming bounds behave badly, loosing even to Gerzon’s bound.

In particular, we improved the range of dimensions for which the bound remains stable, showing that $n = 3/a^2 -16$ from Theorem 6.6 is not optimal. Table 6 shows how much this range is increased for the values of a considered. This observation motivates the following two questions, where a is such that 1/a is an odd integer:

• (1) What is the smallest n such that $M_a(n) = (1/a^2-2)(1/a^2-1)/2$?

• (2) What is the smallest n such that $M_a(n) > (1/a^2-2)(1/a^2-1)/2$?

Table 2 Upper bounds for $M_{1/5}(n)$ by diverse methods, including new results with $\Delta _6(G)^*$

Table 3 Upper bounds for $M_{1/7}(n)$ by diverse methods, including new results with $\Delta _6(G)^*$

Table 4 Upper bounds for $M_{1/9}(n)$ by diverse methods, including new results with $\Delta _6(G)^*$

Table 5 Upper bounds for $M_{1/11}(n)$ by diverse methods, including new results with $\Delta _6(G)^*$

Table 6 By considering $\Delta _k(G)^*$ for $k \ge 4$ we find out that the maximum dimension n for which the bound $M_a(n) \le (1/a^2-2)(1/a^2-1)/2$ is valid is larger than $3/a^2-16$ as given by Theorem 6.6 and $\Delta _3(G)^*$; the table shows the improved dimensions

Question (1) is the more interesting of the two since if the smallest n is $1/a^2 - 2$, then Gerzon’s bound is attained. Theorem 6.3 makes progress in this direction, showing that Gerzon’s bound is also attained if the smallest n is at most $3/a^2-16$; this is known not to be the case for many a (due to the nonexistence of some tight spherical 5-designs, as mentioned after Theorem 6.1), which implies $M_{1/7}(n) \le 1127$ for $n \le 131$ and $M_{1/9}(n) \le 3159$ for $n \le 227$. Table 6 also suggests that the constraint $n \le 3/a^2-16$ in Theorem 6.3 may not be optimal.

Question (2) seems interesting because Table 6 shows that $n = 3/a^2 - 15$ is not a good candidate solution. In fact, the smallest n is likely much larger, as suggested by Conjecture 6.9 for $M_{1/5}(n)$ and the construction described after Corollary 6.14. Using this construction, we know that $(1/a^2-2)(1/a^2-1)/2$ is achieved when $n = (1/a^2-2)(1/a-1)^2/2+1$, which corresponds to the dimensions 185, 847, 2529, and 5951 for $a = 1/5$, 1/7, 1/9, and 1/11 respectively.

We also improve the bounds computed by King and Tang [27] for $M_{1/5}(n)$ by replacing their theorem [27, Theorem 7] by Theorem 6.12 and by using $\Delta _6(G)^*$ to compute better bounds for $A(n, \{1/13, -5/13\})$. Lin and Yu [33] observed that $A(n, \{1/13, -5/13\}) \ge 3n/2-3$ and therefore there is a limit to the power of this approach: it will never be able to prove Conjecture 6.9 no matter how much we increase k. In general, it is not clear how good the bound $\Delta _k(G)^*$ can be for $M_{a}(n)$ if one allows k to increase; in contrast, de Laat and Vallentin [11, Theorem 2] show that their version of the Lasserre hierarchy for compact topological packing graphs converges to the independence number if enough steps are computed. Whether such a convergence result holds for $\Delta _k(G)^*$ is an open question; in any case, it takes days to compute $\Delta _k(G)^*$ for $k = 6$, so one can expect that solving the resulting semidefinite programs for $k > 6$ will be hard in practice.

Appendix A. Proof of the sufficiency part of Theorem 4.1

We now prove, for the sake of completeness, a theorem that, together with the linear transformation $L(A_R)$ used to compute the coordinates of the projection of a vector with respect to an orthonormal basis of $\mathrm {span}(R)$, amounts to the sufficiency part of Theorem 4.1, which is the direction used in this paper. It is a restatement of a proposition of Musin [37, Corollary 3.1].

Recall that $P_l^n$ is the Gegenbauer polynomial of degree l in dimension n normalized so that $P_l^n(1) = 1$. For $0 \le m \le n - 2$, $t \in {\mathbb {R}}$, and u, $v \in {\mathbb {R}}^m$, the polynomial $P_l^{n,m}$ is the $(2m+1)$-variable polynomial

$$\begin{aligned} P_l^{n, m}(t,u,v) = \big ((1-\Vert u\Vert ^2)(1-\Vert v\Vert ^2)\big )^{l/2}P_l^{n-m}\left( \frac{t-u \cdot v}{\sqrt{(1-\Vert u\Vert ^2)(1-\Vert v\Vert ^2)}}\right) , \end{aligned}$$

(14)

where $\Vert v\Vert = \sqrt{v \cdot v}$. If we use the convention ${\mathbb {R}}^0 = \{0\}$, then $P_l^n(t) = P_l^{n,0}(t,0,0)$. Fix $d \ge 0$, let ${\mathcal {B}}_l$ be a basis of the space of m-variable polynomials of degree at most l (e.g. the monomial basis), and write $z_l(u)$ for the column vector containing the polynomials of ${\mathcal {B}}_l$ evaluated at $u \in {\mathbb {R}}^m$. The matrix $Y_l^{n, m}$ is the matrix of polynomials

$$\begin{aligned} Y_l^{n,m}(t,u,v) = P_l^{n,m}(t,u,v) z_{d-l}(u)z_{d-l}(v)^\mathsf{T}. \end{aligned}$$

Theorem A.1

(Musin [37]) Let $R \subseteq S^{n-1}$ with $m = \dim (\mathrm {span}(R)) \le n-2$ and let E be an $m \times n$ matrix whose rows form an orthonormal basis for $\mathrm {span}(R)$. Fix $d \ge 0$ and, for each $0 \le l \le d$, let $F_l$ be a positive semidefinite matrix of size $\left( {\begin{array}{c}d-l+m\\ m\end{array}}\right) \times \left( {\begin{array}{c}d-l+m\\ m\end{array}}\right)$. Then $K :S^{n-1} \times S^{n-1} \rightarrow {\mathbb {R}}$ given by

$$\begin{aligned} K(x,y) = \sum _{l = 0}^d \big \langle F_l, Y_l^{n, m}(x \cdot y, Ex, Ey)\big \rangle \end{aligned}$$

is a positive, continuous, and $\mathrm {Stab}_{\mathrm {O}(n)}(\mathrm {span}(R))$-invariant kernel.

First we prove that the polynomials $P_l^{n,m}$ satisfy the following positivity property [37, Theorem 3.1].

Proposition A.2

(Musin [37]) For $0 \le m \le n - 2$, let E be an $m \times n$ matrix whose rows form an orthonormal set in ${\mathbb {R}}^n$ and C be a finite subset of $S^{n-1}$. Then, for every nonnegative integer l, the matrix $\big ( P_l^{n,m}(x \cdot y, Ex, Ey) \big )_{x, y \in C}$ is positive semidefinite.

Proof

If $l = 0$ then all polynomials evaluate to 1 and the proposition holds, so we assume $l \ne 0$. Let L be the space spanned by the rows of E and z be a unit vector in $L^\perp$. For each $x \in C$, write $x = x_1 + x_2$ with $x_1 \in L$ and $x_2 \in L^\perp$. If $\Vert x_2\Vert > 0$, then let ${\bar{x}} = x_2 / \Vert x_2\Vert$, otherwise write ${\bar{x}} = z$. If $\Vert x_2\Vert$, $\Vert y_2\Vert \ne 0$, then

$$\begin{aligned} {\bar{x}} \cdot {\bar{y}} = \frac{x_2\cdot y_2}{\Vert x_2\Vert \Vert y_2\Vert } = \frac{x\cdot y - x_1 \cdot y_1}{\sqrt{(1-\Vert x_1\Vert ^2)(1-\Vert y_1\Vert ^2)}}. \end{aligned}$$

Since the rows of E are orthonormal, we have $x_1 \cdot y_1 = (Ex)\cdot (Ey)$ and hence $\Vert x_2\Vert ^l\Vert y_2\Vert ^lP_l^{n-m}({\bar{x}} \cdot {\bar{y}}) = P_l^{n,m}(x \cdot y, Ex, Ey)$.

If, say, $\Vert x_2\Vert = 0$, then $\Vert x_2\Vert ^l\Vert y_2\Vert ^l P_l^{n-m}({\bar{x}} \cdot {\bar{y}}) = 0$, while $P_l^{n,m}(x \cdot y, Ex, Ey)$ is also 0 as can be seen from (14) since $x \cdot y - x_1 \cdot y_1 = x_2 \cdot y_2 = 0$.

Now $\{\,{\bar{x}} : x \in C\,\}$ is contained in an embedding of $S^{n-m-1}$ into $S^{n-1}$ and by Schoenberg’s theorem [43] we have that $\big (P_l^{n-m}({\bar{x}} \cdot {\bar{y}})\big )_{x, y \in C}$ is positive semidefinite. Since $\big (\Vert x_2\Vert ^l\Vert y_2\Vert ^l\big )_{x, y \in C}$ is positive semidefinite, so is $\big ( \Vert x_2\Vert ^l \Vert y_2\Vert ^l P_l^{n-m}({\bar{x}} \cdot {\bar{y}})\big )_{x, y \in C}$, and we are done. $\square$

Proof of Theorem A.1

Since all entries of $Y_l^{n,m}$ are polynomials, K is continuous, and since $x \cdot y$, Ex, and Ey are invariant under the action of $\mathrm {Stab}_{\mathrm {O}(n)}(\mathrm {span}(R))$ on (x, y), K is invariant. To prove positivity, let C be a finite subset of $S^{n-1}$ and $w :C \rightarrow {\mathbb {R}}$ be a function. We have

$$\begin{aligned} \sum _{x, y \in C}w_x w_y K(x,y) = \sum _{l = 0}^d \bigg \langle F_l, \sum _{x,y \in C} w_xw_y Y_l^{n,m}(x \cdot y, Ex, Ey)\bigg \rangle . \end{aligned}$$

To show this quantity is nonnegative, we will show that for all $l = 0$, ..., d the matrix $\sum _{x, y \in C}w_x w_y Y_l^{n,m}\big (x \cdot y, Ex, Ey \big )$ is positive semidefinite. For this, write it as a product of matrices: if B is the matrix whose columns are given by $z_{d-l}(Ex)$ for $x \in C$, then

$$\begin{aligned}&\sum _{x, y \in C}w_x w_y Y_l^{n,m}(x \cdot y, Ex, Ey)\\&\quad = \sum _{x, y \in C}w_x w_y z_{d-l}(Ex)z_{d-l}(Ey)^\mathsf{T}P_l^{n,m}(x \cdot y, Ex, Ey)\\&\quad = B \big (P_l^{n,m}(x \cdot y, Ex, Ey)\big )_{x, y \in C} B^\mathsf{T}, \end{aligned}$$

and, since the matrix $\big (P_l^{n,m}(x \cdot y, Ex, Ey)\big )_{x, y \in C}$ is positive semidefinite by Proposition A.2, we are done. $\square$