Range Majorities and Minorities in Arrays

Abstract

The problem of parameterized range majority asks us to preprocess a string of length n such that, given the endpoints of a range, one can quickly find all the distinct elements whose relative frequencies in that range are more than a threshold $\tau$. This is a more tractable version of the classical problem of finding the range mode, which is unlikely to be solvable in polylogarithmic time and linear space. In this paper we give the first linear-space solution with optimal $\mathcal {O}\!\left( {1 / \tau } \right)$ query time, even when $\tau$ can be specified with the query. We then consider data structures whose space is bounded by the entropy of the distribution of the symbols in the sequence. For the case when the alphabet size $\sigma$ is polynomial on the computer word size, we retain the optimal time within optimally compressed space (i.e., with sublinear redundancy). Otherwise, either the compressed space is increased by an arbitrarily small constant factor or the time rises to any function in $(1/\tau )\cdot \omega (1)$. We obtain the same results on the complementary problem of parameterized range minority.

Appendix: Finding \(\tau '\)-Majorities on Tiny Alphabets

We show how to find $\tau '$-majorities in time $\mathcal {O}\!\left( {1/\tau '} \right)$ on ranges of length $\mathcal {O}\!\left( {(1/\tau ')w^\beta } \right)$, over alphabet $[0..\sigma '-1]$, with $\sigma '=w^\beta$, in the case $1/\tau ' < \sigma '$. We will compute an array of $\sigma '$ counters with the frequency of the symbols in the range, and then report those exceeding the threshold. The maximum size of the range is $(4/\tau ')w^\beta /4 \le \sigma ' w^\beta =w^{2\beta }$, and thus $2\beta \lg w$ bits suffice to represent each counter. The $\sigma '$ counters then require $2\beta w^\beta \lg w$ bits and can be maintained in a computer word (although we will store them somewhat spaced for technical reasons). We can read the elements in $S_v$ by chunks of $w^\beta$ symbols, and compute in constant time the corresponding counters for those symbols. Then we sum the current counters and the counters for the chunk, all in constant time because they are fields in a single computer word. The range is then processed in time $\mathcal {O}\!\left( {1/\tau '} \right)$.

To compute the counters corresponding to $w^\beta$ symbols, we extend the popcounting algorithm of Belazzougui and Navarro [6, Sec. 4.1]; assume we extract the $w^\beta$ symbols from $S_v$ and have them packed in the lowest $k\ell$ bits of a computer word X, where $k=w^\beta$ is the number of symbols and $\ell =\lg \sigma '$ is the number of bits used per symbol. We first create $\sigma '$ copies of the sequence at distance $2k\ell$ of each other: $X \leftarrow X \cdot (0^{2k\ell -1}1)^{\sigma '}$. In each copy we will count the occurrences of a different symbol. To have the $(i+1)$th copy count the occurrences of symbol i, for $0 \le i < \sigma '$, we perform

$$\begin{aligned} X ~\leftarrow ~ X~~\textsc {xor}~~0^{k\ell } ((\sigma '-1)_\ell )^k \ldots 0^{k\ell } (2_\ell )^k ~ 0^{k\ell } (1_\ell )^k ~ 0^{k\ell } (0_\ell )^k, \end{aligned}$$

where $i_\ell$ is number i written in $\ell$ bits. Thus in the $(i+1)$th copy the symbols equal to i become zero and the others nonzero. We then set a 1 at the highest bit of the symbols equal to i in the $(i+1)$th copy, with

$$\begin{aligned} X ~\leftarrow ~ (Y - (X~\textsc {and~not}~Y))~\textsc {and}~Y~\textsc {and~not}~X, \end{aligned}$$

where $Y=(0^{k\ell } (10^{\ell -1})^k)^{\sigma '}$.³ Now we add all the 1s in each copy with $X \leftarrow X \cdot 0^{k\ell (2\sigma '-1)} (0^{\ell -1}1)^k$. This spreads several sums across the $2k\ell$ bits of each copy, and in particular the kth sum adds up all the 1s of the copy. Each sum requires $\lg k$ bits, which is precisely the $\ell$ bits we have allocated per field. Finally, we isolate the desired counters using $X \leftarrow X~\textsc {and}~(0^{k\ell }1^\ell 0^{(k-1)\ell })^{\sigma '}$. The $\sigma '$ counters are not contiguous in the computer word, but we still can afford to store them spaced: we use $2k\ell \sigma ' = 2 \beta w^{2\beta }\lg w$ bits, which since $\beta \le 1/4$, is always less than w.

Since the range is of length at most $w^{2\beta }$, the cumulative counters need $\lg (w^{2\beta }) = 2\ell$ bits. We will store them in a computer word A separated by $2k\ell$ bits so that we can directly add the resulting word X after processing a chunk of $w^\beta$ symbols of the range in $S_v$: $A \leftarrow A+X$. If the last chunk is of length $l < w^\beta$, we complete it with zeros and then subtract those spurious $w^\beta -l$ occurrences from the first counter, $A \leftarrow A - (w^\beta -l) \cdot 2^{(k-1)\ell }$.

The last challenge is to output the counters that are at least $y = \lfloor \tau ' (j-i+1) \rfloor +1$ after processing the range. We use

$$\begin{aligned} A \leftarrow A + (2^{2\ell }-y) \cdot (0^{k\ell +\ell -1} 1 0^{(k-1)\ell })^{\sigma '} \end{aligned}$$

so that the counters reaching y will overflow to the next bit. We isolate those overflow bits with $A \leftarrow A~\textsc {and}~(0^{(k-1)\ell -1} 1 0^{(k+1)\ell })^{\sigma '}$, so that we have to report the symbol i if and only if $A~\textsc {and}~0^{(k(2\sigma '-2i-1)-1)\ell -1} 1 0^{(k(2i+1)+1)\ell } \not = 0$. We then repeatedly isolate the lowest bit of A with

$$\begin{aligned} D \leftarrow (A~\textsc {xor}~(A-1))~\textsc {and}~ (0^{(k-1)\ell -1} 1 0^{(k+1)\ell })^{\sigma '}, \end{aligned}$$

and then remove it with $A \leftarrow A~\textsc {and}~(A-1)$, until $A=0$. Once we have a position isolated in D, we find the position in constant time by using a monotone minimum perfect hash function over the set $\{ 2^{(k(2i+1)+1)\ell },~0 \le i < \sigma '\}$, which uses $\mathcal {O}\!\left( {\sigma ' \lg w} \right) =o(w)$ bits [2]. Only one such data structure is needed for all the sequences, and it takes less space than a single systemwide pointer.