QuickXsort: A Fast Sorting Scheme in Theory and Practice

Abstract

QuickXsort is a highly efficient in-place sequential sorting scheme that mixes Hoare’s Quicksort algorithm with X, where X can be chosen from a wider range of other known sorting algorithms, like Heapsort, Insertionsort and Mergesort. Its major advantage is that QuickXsort can be in-place even if X is not. In this work we provide general transfer theorems expressing the number of comparisons of QuickXsort in terms of the number of comparisons of X. More specifically, if pivots are chosen as medians of (not too fast) growing size samples, the average number of comparisons of QuickXsort and X differ only by o(n)-terms. For median-of-k pivot selection for some constant k, the difference is a linear term whose coefficient we compute precisely. For instance, median-of-three QuickMergesort uses at most $n \lg n - 0.8358n + {\mathcal {O}}(\log n)$ comparisons. Furthermore, we examine the possibility of sorting base cases with some other algorithm using even less comparisons. By doing so the average-case number of comparisons can be reduced down to $n \lg n - 1.4112n + o(n)$ for a remaining gap of only 0.0315n comparisons to the known lower bound (while using only ${\mathcal {O}}(\log n)$ additional space and ${\mathcal {O}}(n\log n)$ time overall). Implementations of these sorting strategies show that the algorithms challenge well-established library implementations like Musser’s Introsort.

Appendices

Appendix

A Notation

1.1 Generic Mathematics

${\mathbb {N}}$, ${\mathbb {N}}_0$, ${\mathbb {Z}}$, ${\mathbb {R}}$:

Natural numbers ${\mathbb {N}}= \{1,2,3,\ldots \}$, ${\mathbb {N}}_0 = {\mathbb {N}}\cup \{0\}$, integers ${\mathbb {Z}}= \{\ldots ,-2,-1,0,1,2,\ldots \}$, real numbers ${\mathbb {R}}$.

${\mathbb {R}}_{>1}$, ${\mathbb {N}}_{\ge 3}$ etc.:

Restricted sets $X_\text {pred} = \{x\in X : x \text { fulfills } \text {pred} \}$.

$\ln (n)$, $\lg (n)$, $\log n$:

Natural and binary logarithm; $\ln (n) = \log _e(n)$, $\lg (n) = \log _2(n)$. We use $\log$ for an unspecified (constant) base in ${\mathcal {O}}$-terms

X :

To emphasize that X is a random variable it is Capitalized.

[a, b):

Real intervals, the end points with round parentheses are excluded, those with square brackets are included.

[m..n], [n]:

Integer intervals, $[m..n] = \{m,m+1,\ldots ,n\}$; $[n] = [1..n]$.

$\llbracket \text {stmt}\rrbracket$, $\llbracket x=y\rrbracket$:

Iverson bracket, $\llbracket \text {stmt}]\rrbracket = 1$ if stmt is true, $\llbracket \text {stmt}\rrbracket = 0$ otherwise.

$H_{n}$ :

nth harmonic number; $H_{n} = \sum _{i=1}^n 1/i$.

$x \pm y$ :

x with absolute error |y|; formally the interval $x \pm y = [x-|y|,x+|y|]$; as with ${\mathcal {O}}$-terms, we use one-way equalities $z=x\pm y$ instead of $z \in x \pm y$.

$a^{{\underline{b}}}$, $a^{{{\overline{b}}}}$:

Factorial powers; “a to the b falling resp. rising”; e.g., $x^{{\underline{3}}} = x(x-1)(x-2)$, $x^{\underline{-3}} = 1/((x+1)(x+2)(x+3))$.

$\left( {\begin{array}{c}n\\ k\end{array}}\right)$ :

Binomial coefficients; $\left( {\begin{array}{c}n\\ k\end{array}}\right) = n^{{\underline{k}}} \big / k!$.

$\text {B}(\lambda ,\rho )$ :

For $\lambda ,\rho \in {\mathbb {R}}_+$; the beta function, $\text {B}(\lambda ,\rho ) = \int _0^1 z^{\lambda -1}(1-z)^{\rho -1}\, dz$; see also Eq. (9) on page 42.

$I_{x,y}(\lambda ,\rho )$ :

The regularized incomplete beta function; $I_{x,y}(\lambda ,\rho )= \int _x^y \frac{z^{\lambda -1}(1-z)^{\rho -1}}{\text {B}(\lambda ,\rho )} dz$ for $\lambda ,\rho \in {\mathbb {R}}_+$, $0\le x\le y\le 1$.

1.2 Stochastics-Related Notation

${\mathbb {P}}[E]$, ${\mathbb {P}}[X=x]$:

Probability of an event E resp. probability for random variable X to attain value x.

${\mathbb {E}}[X]$ :

Expected value of X; we write ${\mathbb {E}}[XY]$ for the conditional expectation of X given Y, and to emphasize that expectation is taken w.r.t. random variable X.

:

Equality in distribution; X and Y have the same distribution.

${\mathcal {U}}(a,b)$ :

Uniformly in $(a,b)\subset {\mathbb {R}}$ distributed random variable.

$\text {Beta}(\lambda ,\rho )$ :

Beta distributed random variable with shape parameters $\lambda \in {\mathbb {R}}_{>0}$ and $\rho \in {\mathbb {R}}_{>0}$.

$\text {Bin}(n,p)$ :

Binomial distributed random variable with $n\in {\mathbb {N}}_0$ trials and success probability $p\in [0,1]$.

$\text {BetaBin}(n,\lambda ,\rho )$ :

Beta-binomial distributed random variable; $n\in {\mathbb {N}}_0$, $\lambda ,\rho \in {\mathbb {R}}_{>0}$;

1.3 Specific Notation for Algorithms and Analysis

n :

Length of the input array, i.e., the input size.

k, t:

Sample size $k\in {\mathbb {N}}_{\ge 1}$, odd; $k=2t+1$, $t\in {\mathbb {N}}_0$; we write k(n) to emphasize that k might depend on n.

w :

Threshold for recursion, for $n\le w$, we sort inputs by X; we require $w\ge k-1$.

$\alpha$ :

$\alpha \in [0,1]$; method X may use buffer space for $\lfloor \alpha n\rfloor$ elements.

c(n):

Expected costs of QuickXsort; see Sect. 9.2.

x(n), a, b:

Expected costs of X, $x(n) = a n \lg n + b n \pm o(n)$; see Sect. 9.2.

$J_1$, $J_2$:

(Random) Subproblem sizes; $J_1+J_2 = n-1$; $J_1 = t + I_1$;

$I_1$, $I_2$:

(Random) Segment sizes in partitioning; ; $I_2 = n-k-I_1$; $J_1 = t + I_1$

R :

(One-based) Rank of the pivot; $R=J_1+1$.

s(k):

(Expected) cost for pivot sampling, i.e., cost for choosing median of k elements.

$A_1$, $A_2$, A:

Indicator random variables; $A_1 = \llbracket \text {left subproblem sorted recursively}\rrbracket$; see Sect. 9.2.

B Mathematical Preliminaries

In this appendix, we restate some known results for the reader’s convenience.

1.1 Hölder Continuity

A function $f:I\rightarrow R$ defined on a bounded interval I is Hölder-continuous with exponent $\eta \in (0,1]$ if

$$\begin{aligned} \exists C\; \forall x,y\in I\;: \bigl | f(x) - f(y) \bigr | \;\le C |x-y|^\eta . \end{aligned}$$

Hölder-continuity is a notion of smoothness that is stricter than (uniform) continuity but slightly more liberal than Lipschitz-continuity (which corresponds to $\eta =1$). $f:[0,1]\rightarrow {\mathbb {R}}$ with $f(z) = z \ln (1/z)$ is a stereotypical function that is Hölder-continuous (for any $\eta \in (0,1)$) but not Lipschitz (see Lemma 9.1 below).

One useful consequence of Hölder-continuity is given by the following lemma: an error bound on the difference between an integral and the Riemann sum ([51, Proposition 2.12–(b)]).

Lemma B.1

(Hölder integral bound) Let $f:[0,1] \rightarrow {\mathbb {R}}$ be Hölder-continuous with exponent $\eta$. Then

$$\begin{aligned} \int _{0}^1 f(x) \, dx&\;\;= \frac{1}{n} \sum _{i = 0}^{n-1} f(i / n) \;\;\pm {\mathcal {O}}(n^{-\eta }), \qquad (n\rightarrow \infty ). \end{aligned}$$

$\square$

Remark B.2

(Properties of Hölder-continuity) We considered only the unit interval as the domain of functions, but this is no restriction: Hölder-continuity (on bounded domains) is preserved by addition, subtraction, multiplication and composition (see, e.g., [48, Section 4.6] for details). Since any linear function is Lipschitz, the result above holds for Hölder-continuous functions $f:[a,b] \rightarrow {\mathbb {R}}$.

If our functions are defined on a bounded domain, Lipschitz-continuity implies Hölder-continuity and Hölder-continuity with exponent $\eta$ implies Hölder-continuity with exponent $\eta ' < \eta$. A real-valued, differentiable function is Lipschitz if its derivative is bounded.

1.2 Chernoff Bound

We write if X is has a binomial distribution with $n\in {\mathbb {N}}_0$ trials and success probability $p\in [0,1]$. Since X is a sum of independent random variables with bounded influence on the result, Chernoff bounds imply strong concentration results for X. We will only need a very basic variant given in the following lemma.

Lemma B.3

(Chernoff Bound, Theorem 2.1 of [36]) Let and $\delta \ge 0$. Then

(26)

$\square$

1.3 Continuous Master Theorem

For solving recurrences, we build upon Roura’s master theorems [44]. The relevant continuous master theorem is restated here for convenience:

Theorem B.4

(Roura’s Continuous Master Theorem (CMT)) Let $F_n$ be recursively defined by

$$\begin{aligned} F_n \;\;= {\left\{ \begin{array}{ll} b_n\,, &{}\hbox { for } 0 \le n < N ; \\ t_n \,+ \smash {\sum _{j=0}^{n-1} w_{n,j} \, F_j}, &{}\hbox { for }n \ge N\,, \end{array}\right. } \end{aligned}$$

(27)

where $t_n$, the toll function, satisfies $t_n \sim K n^\sigma \log ^\tau (n)$ as $n\rightarrow \infty$ for constants $K\ne 0$, $\sigma \ge 0$ and $\tau > -1$. Assume there exists a function $w:[0,1]\rightarrow {\mathbb {R}}_{\ge 0}$, the shape function, with $\int _0^1 w(z) dz \ge 1$ and

$$\begin{aligned} \sum _{j=0}^{n-1} \,\biggl | w_{n,j} \,- \! \int _{j/n}^{(j+1)/n} w(z) \, dz \biggr | \;\;= {\mathcal {O}}(n^{-d}), \qquad (n\rightarrow \infty ), \end{aligned}$$

(28)

for a constant $d>0$. With $\displaystyle H {:}{=}1 - \int _0^1 \!z^\sigma w(z) \, dz$, we have the following cases:

• 1. If $H > 0$, then $\displaystyle F_n \sim \frac{t_n}{H}$.

• 2. If $H = 0$, then $\displaystyle F_n \sim \frac{t_n \ln n}{{{\widetilde{H}}}}$ with $\displaystyle {{\widetilde{H}}} = -(\tau +1)\int _0^1 \!z^\sigma \ln (z) \, w(z) \, dz$.

• 3. If $H < 0$, then $F_n = {\mathcal {O}}(n^c)$ for the unique $c\in {\mathbb {R}}$ with $\displaystyle \int _0^1 \!z^c w(z) \, dz = 1$.

$\square$

Theorem B.4 is the “reduced form” of the CMT, which appears as Theorem 1.3.2 in Roura’s doctoral thesis [43], and as Theorem 18 of [35]. The full version (Theorem 3.3 in [44]) allows us to handle sublogarithmic factors in the toll function, as well, which we do not need here.

C Pseudocode

In this appendix, we list explicit pseudocode for the most important merging procedures.

1.1 Simple Merge by Swaps

Algorithm C.1 is a merging procedure with an in-place interface: upon termination, the merge result is found in the same area previously occupied by the merged runs. The method works by moving the left run into a buffer area first. More specifically, we move $A[\ell ..m-1]$ into the buffer area $B[b..b+n_1-1]$ and then merge it with the second run A[m..r]—still residing in the original array—into the empty slot left by the first run. By the time this first half is filled, we either have consumed enough of the second run to have space to grow the merged result, or the merging was trivial, i.e., all elements in the first run were smaller.

This method is mostly for demonstration purposes, how little changes are required to use Mergesort in QuickXsort. It is not the best solution, though.

1.2 Ping-Pong Mergesort

In this section, we give detailed code for the “ping-pong” Mergesort variant that we use in our QuickMergesort implementation. It also uses $\alpha =\frac{1}{2}$, i.e., buffer space to hold half of the input. By using a special procedure, MergesortOuter, for the outermost call following the strategy illustrated in Fig. 3, we reduce the task to the case of $\alpha =1$ for two subproblems. These are solved by moving elements from the input area to the output area (while sorting them), which is easily achieved by a recursive procedure (MergesortInner).

1.3 Reinhardt’s Merge

In this section, we give the code for Reinhardt’s merge [42], which allows a smaller buffer $\alpha = \frac{1}{4}$. We do not present Reinhardt’s whole Mergesort algorithm based on it but merely the actual merging routine as illustrated in Fig. 4. The organization of the different calls to merging is easy but requires many tedious index calculations, which do not add much insight for the reader.