The strict superlinear order can be faster than the infinite order

Abstract

The sequences with strict superlinear convergence are the output of numerous algorithms; such speed is clearly much faster than linear, but is it also slower than, say, quadratic? In this note we show that actually there are four distinct classes of strict superlinear order: “weak”, “medium”, “strong” and “mixed”. The speed of the sequences from the first three classes is increasingly much faster (term-by-term big Oh, i.e., $|x^*-x_k|=\mathcal {O}(|y^*-y_k|^{\alpha })$, as $k\rightarrow \infty , \forall \alpha >1$ given), whereas the speed of the “mixed” class cannot be assessed. We prove that the speed of the sequences from the “medium” and “weak” classes is term-by-term slower than the speed of the sequences with high classical C-orders $p>1$ (in the sense of big Oh above), while an example shows that certain sequences from the “mixed” class may be term-by-term faster than some sequences with infinite C-order. We also show that for a given sequence with strict superlinear convergence, one can evaluate numerically to which class it belongs. Some recent results of Rodomanov and Nesterov (Math Program 194(1-2, Ser. A):159–190, 2022), resp. Ye et al. [17] show that certain classical quasi-Newton methods (DFP, BFGS and SR1) belong to the “weak” class.