On the condition number of Vandermonde matrices with pairs of nearly-colliding nodes

We prove upper and lower bounds for the spectral condition number of rectangular Vandermonde matrices with nodes on the complex unit circle. The nodes are “off the grid,” pairs of nodes nearly collide, and the studied condition number grows linearly with the inverse separation distance. Such growth rates are known in greater generality if all nodes collide or for groups of colliding nodes. For pairs of nodes, we provide reasonable sharp constants that are independent of the number of nodes as long as non-colliding nodes are well-separated.


Introduction
Vandermonde matrices with complex nodes appear in polynomial interpolation problems and many other fields of mathematics (see, e.g., the introduction of [2] and its references). In this paper, we are interested in rectangular Vandermonde matrices with nodes on the complex unit circle and with a large polynomial degree. These matrices generalize the classical discrete Fourier matrices to non-equispaced nodes The original article was revised: Figure 2 image is correctly presented in this paper.

Stefan Kunis skunis@uos.de
Dominik Nagel dnagel@uos.de 1 and the involved polynomial degree is also called bandwidth. The condition number of those matrices has recently become important in the context of stability analysis of super-resolution algorithms like Prony's method [6,15], the matrix pencil method [12,18], the ESPRIT algorithm [20,21], and the MUSIC algorithm [17,22]. If the nodes of such a Vandermonde matrix are all well-separated, with minimal separation distance greater than the inverse bandwidth, bounds on the condition number are established for example in [2,5,14,18].
If nodes are nearly colliding, i.e., their distance is smaller than the inverse bandwidth, the behavior of the condition number is not yet fully understood. The seminal paper [9] coined the term (inverse) super-resolution factor for the product of the bandwidth and the separation distance of the nodes. For nodes on a grid, the results in [7,9] imply that the condition number grows like the super-resolution factor raised to the power of 1 if all nodes nearly collide. More recently, the practically relevant situation of groups of nearly colliding nodes was studied in [1,4,16,19]. In different setups and oversimplifying a bit, all of these refinements are able to replace the exponent 1 by the smaller number 1, where denotes the number of nodes that are in the largest group of nearly colliding nodes. The authors of [1,19] focus on quite specific quantities in an optimization approach and in the so-called Prony mapping, respectively. In contrast, the condition number or the relevant smallest singular value of Vandermonde matrices with "off the grid" nodes on the unit circle is studied in [4,16]. While [4] provided the exponent 1 for the first time, the proof technique leads to quite pessimistic constants and more restrictively asks all nodes (including the well-separated ones) to be within a tiny arc of the unit circle. More recently, the second version of [16] provided a quite general framework and reasonable sharp constants, but involves a technical condition which prevents the separation distance from going to zero for a fixed number of nodes and a fixed bandwidth.
Here, we present upper and lower bounds for the condition number of Vandermonde matrices with pairs of nearly colliding nodes, i.e., the special case 2. We achieve the expected linear order and all constants are reasonably sharp and absolute. In contrast to the more general quoted results [4,16], the nodes can be placed on the full unit circle and the separation distance is allowed to approach zero. Our mild technical conditions, which seem to be artifacts of our proof technique, are: (i) A logarithmic growth in the separation distance of the well-separated nodes (which can be dropped at a price of a larger constant for the condition number estimate), (ii) A uniformity condition that colliding nodes behave similarly (they have the same separation distance up to a predefined constant), and (iii) An a priori upper bound on the separation distance of the colliding nodes.
The outline of this paper is as follows: Section 2 fixes the notation, recalls results for the case of well-separated nodes, and provides lower bounds for the condition number. In Section 3, we establish upper bounds for nodes that are well-separated from each other except for one pair of nodes that is nearly colliding. Section 4 goes one step further and studies the more general case where an arbitrary number of pairs of nodes nearly collide. Theoretical and numerical comparisons with [3,4,8,16] can be found at the end of Section 4 and in Section 5. ). On the other hand, if two nodes are equal, then two rows of are the same and by continuity the condition number diverges if two nodes collide. The (wrap around) distance of two nodes is given by: min .
and we introduce the normalized separation distance of the node set as: min .
We call the case 1 critical separation, i.e., min 1 , and the cases 1 and 1 nearly colliding and well-separated, respectively. Figure 1 illustrates the situation for 4 nodes on the unit circle. The parameter describes a minimum separation distance of involved non-colliding nodes assumed in the theorems.
A reasonable result for well-separated nodes is as follows.
The key idea is to see the set of nodes as a union of two well-separated subsets and use the existing bounds for these. In contrast to the next chapter, here, one of the sets only consists of a single node. We start by noting that Theorem 2.1 and (3.1) Proof The vector can be approximated by the first column of 2 in the sense that: 3 . . . We have 1 and for 2 1 the mean value theorem yields: Note that, in the worst case, half of the nodes can be as close as possible (under the assumed separation condition) to 2 not only on its right but also on its left. Hence, for 2 2 , 1 and Lemma A.1 lead to: Thus, for all nodes, we get: First of all, we establish an upper bound for the norm of the triangular matrix. Equation (3.1) and Theorem 2.1 imply: Together with Lemma A.6, we obtain: The next step is to bound Proof The bound follows from Lemmata 3.3 and 3.5 with 6 6.5.
Lower and upper bounds in Theorems 2.2 and 3.6 yield: 1 cond 4 for 0.46 and 6 . The condition on implies that for specific configurations of nodes, our result becomes effective as early as 6 -this is in contrast to the results [4,16], where has to be much larger.   While it is clear that the Schur complement 1 1 2 is strictly positive definite, establishing a lower bound on its smallest singular value similar to the proofs of Lemmata 3.4 and 3.5 seems considerably harder. Already, the linear approximation in Lemma 3.4 then needs to be replaced by a higher order approximation for the matrix .

Pairs of nearly colliding nodes
We now study the situation in which the Vandermonde matrix comes from pairs of nearly colliding nodes. The proof technique we use is analogous to the one we used in the case of two nearly colliding nodes. The difference is that we have a matrix 1    . Additionally, we set 1 . We bound the spectral norm of 1 by the one of the real symmetric matrix 1 using Lemma A.2 and proceed by:  Proof First, note that: for . For each fixed off-diagonal entry , the matrix 2 has no contribution. We write the node 2 as a perturbation of by 2 and expand the Dirichlet kernel by its Taylor polynomial of degree 2 in the point 2 . Using: for some , the constant term, as well as the linear term, cancels out and we get:

Figure 5 visualizes the values of the constant with respect to and . Please note that (i) increasing the constant by a factor 2 has to be compensated approximately by halving and doubling and (ii) increasing the number of nodes from 4 to 64 has to be compensated approximately by tripling .
Proof We proceed analogously to Lemma 3.5 and apply Lemma A.4 to the matrix decomposed as in (4.1) and obtain:  (i) First, note that † 1 1 is an orthogonal projector and thus Theorem 2.1 implies: We apply Lemma A.3 with 2 , use the identities 1 1 and 2 2 , apply the triangular inequality, and the sub-multiplicativity of the matrix norm to get: (iv) We use the estimates for the Dirichlet kernel 2 in ii) and  Proof In Lemma 4.6, the constant is monotone increasing in and monotone decreasing in . Hence, after plugging in the bounds for and in our assumptions, it is easy to see that the constant 1 4 2 10 2 log 4 1 is monotone decreasing in and , respectively. Therefore, we get 1 4 10 1 4 11.3, so that 1 11.3 1 2 . Together with the bound 22 10 2.2 from Lemma 4.3, we obtain the result.
If each pair of nearly colliding nodes has the same separation distance, i.e., 1, we can improve the upper bound in the sense that restrictions on except for 1 can be dropped. In order to obtain the same constant, we have to increase the restrictions on slightly. In three summands, we can factor out and use the estimate 2 , leading to a larger bound after inverting the expression in the end. Afterwards, in the third summand is left, for which we use the rough bound . In the fourth summand, we use 1 for the single . The same argument as in (3.4) shows that this also bounds the maximum in (4.2) and we get the result.  The lower bound is tight and the numerical value 5 in the upper bound follows from our proof technique and can be improved (see Fig. 6). The uniformity condition 1 4 2 is artificial and, except for the special cases in Theorems 3.6 and 4.9, prevents letting 1. Moreover, the technical condition min 25 log 4 1 in Theorem 4.7 is due to the slow decay of the Dirichlet kernel and can be weakened by a preconditioning technique which however leads to a somewhat larger constant in the final result. 1  . 1 We thank one of the peer reviewers for this clever hint.
Under the conditions of Definition 4.1 with 1, 1 4 2 and min 11 2 , we finally have cond 14 . Note that this approach also allows to drop the logarithmic factor in Remark 3.8 (ii).
The absolute constant 5 in the upper bound of the condition number (or † 11.3 3.4) follows from our proof technique and we give a numerical comparison to the approaches [3,4,8,16] in Fig. 7. A short theoretical comparison including different assumptions on , , , and is given below.  3. Recently, we refined this approach in [13], dropped the mentioned dependencies on and could weaken the condition (4.4) considerably.
Remark 4.13 (Comparison to [8]) This approach deals with pairs of nearly colliding nodes but differs completely from ours and the ones in [3,4,16], and rather generalizes the construction of certain extremal functions in [18] to pairs of nearly colliding nodes and subsets of them. The proven constant in the upper bound given in [8,Cor. 4.2] is † 9 6 7.0 and thus is slightly larger than ours ( 11.3 3.4). Using the stronger assumption on from our setting in the proof of [8,Thm. 3.6] and improving estimates in [8,Eq. (8)] provides the best result ( 1.7) for pairs of nearly colliding nodes. The conditions 1 and 3 are quasi-optimal. Provided all technical results prove right, this approach is superior.  However, note that for two nearly colliding pairs 1 2 1 2 , a direct computation (avoiding a so-called limit basis used in [3]) yields the off-diagonal estimate: 1  Altogether, the improved variant of this technique can be used for nearly colliding pairs, but leads to a stronger assumption on for all moderate uniformity constants .

Numerical examples
All computations were carried out using MATLAB R2019b. As a test for the bounds in the case of one pair of nearly colliding nodes, we use the following configuration. Let the number of nodes 20 and 200 be fixed, respectively. Moreover, we choose 1 12 1 which ensures that all nodes fit on the unit interval. We choose 10 11 1 logarithmically uniformly at random and 3 [6 12] uniformly at random. Then, we set the nodes 1 [0 1 such that 1 0, 2 and for 3 , 1 . Afterwards, the condition number of the corresponding Vandermonde matrix is computed. This procedure is repeated 100 times and the results are presented in Fig. 6 (left).
For pairs of nearly colliding nodes, we use the following configuration. Let the number of nodes 20 and 200 be fixed, respectively. Moreover, we choose the parameter 2 and and min as in Theorem 4.7. To ensure that all nodes fit on the unit interval, we choose as the smallest odd integer bigger than 2 min 2. Then, we choose 10 11 1 logarithmically uniformly at   11 1 logarithmically uniformly at random and min 2 2 uniformly at random. Afterwards, the inverse of the smallest singular value (norm of Moore-Penrose pseudo inverse) of the corresponding Vandermonde matrix is computed. This procedure is repeated 100 times and the results normalized by are presented in Fig. 7 4 . We note that our bound remains valid for 1 but the restriction on becomes more severe.

Summary
We proved upper and lower bounds for the spectral condition number of rectangular Vandermonde matrices with nodes on the complex unit circle. If pairs of nodes nearly collide, the studied condition number grows linearly with the inverse separation distance. In contrast to the more general results [4,16], we provide reasonable sharp and absolute constants but have to admit that our technique most likely will not generalize to more than two nodes nearly colliding. Note that our easy to achieve lower bound seems to capture the situation more accurately than the upper bound. We posed mild technical conditions in our proofs, which cannot be confirmed to be necessary numerically. While [4] provided the right growth order for the first time, some of the imposed conditions are very restrictive and the involved constants are quite pessimistic. The second version of [16] provided a quite general framework and presented decent results with only a mild artificial growth of the condition number with respect to the number of nodes. Moreover, a technical condition there prevents the separation distance from going to zero for a fixed number of nodes and a fixed bandwidth. We believe that both problems can be fixed at least partially and thus [16] seems to be a good framework for understanding node configurations with nearly colliding nodes. Recently, the manuscript [8] came to our attention-it considers pairs of nearly colliding nodes and weakens the assumptions considerably and gives, after modifications, stronger bounds on the smallest singular value. The taken approach differs completely from ours and the ones in [4,16], but rather generalizes the construction of [18] to pairs of nearly colliding nodes.