When does 0–1 Principle Hold for Prefix Sums?

Knuth’s 0–1 principle argues that the correctness of any swap-based sorting network can be verified by testing arbitrary sequences over Boolean values (i.e., 0 and 1). Voigtländer (Proceedings of the 35th ACM SIGPLAN-SIGACT symposium on principles of programming languages, POPL 2008, San Francisco, California, USA, January 7–12, 2008. ACM, New York, NY, pp 29–35, 2008. https://doi.org/10.1145/1328438.1328445) proved a similar result for prefix-sum networks that consist of associative binary operators: the correctness can be verified by testing arbitrary sequences and associative binary operators over three values, namely 0, 1, and 2. He raised the question of whether testing over Boolean values is sufficient if the binary operator is idempotent in addition to associative. This paper answers his question. First, there is an incorrect prefix-sum network for associative idempotent operators, the flaw of which cannot be detected by testing over Boolean values. Second, testing over Boolean values is sufficient if the binary operators are restricted to commutative in addition to associative and idempotent.


Introduction
A sorting network consists of wires and comparators.A wire propagates a value, and a comparator connects two wires and rearranges the propagated values in order.Figure 1 shows an example.
There are many sorting networks.It is not easy to verify whether a sorting network correctly reorders the input.One may consider testing every permutation of different values.This naive method is inefficient and requires n! test cases for verifying an n-wires sorting network.Knuth's 0-1 principle [1] argues that it is sufficient to consider sequences over 0 and 1 s for verifying sorting networks.It significantly reduces the number of necessary test cases from n! to 2 n .This test method is referred to as two-values testing.
This paper investigates the prefix sum (also known as scan, specifically in the functional programming community).Given an associative binary operator ⊕ and series of values, [x 0 , x 1 , . . ., x n−1 ], the prefix sum calculates the sum of every prefix: It is especially important in the context of parallel computing [2], and its hardware-level implementations are called prefix sum networks (Fig. 2) Similar to sorting, there are many variations for the prefix sum.The minimumwork algorithm requires only n − 1 calculations of ⊕ for an input of size n but is perfectly sequential (Fig. 2, left).Parallel algorithms (e.g., Fig. 2, right) should handle the trade-off between the amount of work and parallel scalability.It is nontrivial to develop efficient and correct prefix-sum algorithms, and several studies [3][4][5][6][7][8] proposed methods for supporting their design and verification.
Voigtländer [4] focused on the similarity between sorting networks and prefixsum networks and proved that the prefix-sum algorithms have the 0-1-2 principle.We can verify their correctness by three-values testing, namely by testing arbitrary combinations of 0, 1, and 2 s, and associative binary operators over them.This enables us to avoid testing arbitrary sequences.
He raised the question of whether two-values testing is sufficient if the binary operator is idempotent in addition to associative.This paper answers his question.In fact, idempotency with associativity is insufficient: there is an incorrect implementation for which we cannot detect its flaw by two-values testing (Sect.3).Two-values testing is sufficient if the binary operator is commutative in addition to associative and idempotent (Sect.4).

Prefix Sum
Following Voigtländer [4], this paper uses notations inspired by the Haskell [9] functional programming language.The parentheses for function applications are omitted; hence, f x denotes an application of function f to its argument x.Multi-parameter functions are written in the curried form: f x y is an application of function f to two arguments, x and y.Binary operators are variants of two-parameter functions: writing x ⊕ y is equivalent to (⊕) x y.Function applications precede operators; thus, x ⊕ f y means x ⊕ ( f y).
A value x of type A is denoted as x::A.The type of function f that takes a value of type A and returns one of type B is denoted as f ::A → B. If f takes two values of type A 1 and A 2 in this order, its type is f :: The type of a list with elements having type A is denoted as applies the given function to every element in the given list: Throughout this paper, only programs that terminate without raising any errors are considered.
Following the Haskell standard library, the prefix-sum function is called scanl.Its type is given below. 1canl::∀α.
It takes two parameters, a binary operator and list.The type α is universally quantified by ∀α because scanl only provides a computation pattern (in other words, the shape of a network) and relies on the binary operator for the actual summarization.
The operator passed to a prefix-sum function is assumed to be associative.Associativity is necessary for its efficient parallel implementation.Idempotency and commutativity have also been exploited for better parallel implementations.Idempotency enables somewhat redundant computations, thus makes it simpler to deal with exceptional cases.Lynch and Swartzlander [10] developed a redundant parallel adder by taking idempotency into account.Commutativity is particularly useful for summarizing more than two inputs because it enables us to disregard the order of input elements and brings more flexibility to the scheduling of computations.Beaumont-Smith and Lim [11] studied parallel prefix sum networks with such many-in operators.
Associative commutative idempotent operators include the disjunction and conjunction (∨ and ∧) on Boolean values or bit-vectors, binary maximum and minimum operators on numerals, and union and intersection on sets.Associative commutative (but not idempotent) operators include the exclusive-or on Boolean values or bitvectors, addition and multiplication on numerals, and union on multisets.Associative idempotent (but not commutative) operators include the "left" operator (i.e., such that x y = x) and "right" operator.The prefix-sum computation with these operators appears in several application domains.For example, the visibility testing problem (called "line-of-sight") and quicksort contain prefix-sum computations with the binary maximum operator and left operator, respectively [2].
There are many possible implementations of scanl.This paper focuses on the following observational characterization.
implements scanl for associative (associative idempotent and associative commutative idempotent) operators if the following equation holds for every associative (associative idempotent and associative commutative idempotent, respectively) binary operator ⊕, every list x = [x 0 , x 1 , . . ., x n−1 ] of length n, and every natural number 0 ≤ m ≤ n − 1.
In the following, scanl is used as a name of a reference implementation of scanl.

0-1-2 Principle
Let Two and Three be types that correspond to {0, 1} and {0, 1, 2}, respectively.Twovalues testing checks whether f (⊕) x = scanl (⊕) x holds for arbitrary x::[Two]2 and (⊕)::Two → Two → Two satisfying the required algebraic properties.Similarly, three-values testing checks the equality for arbitrary x::[Three] and (⊕)::Three → Three → Three satisfying the required algebraic properties.It is trivial that f will pass both if f implements scanl.We are interested in whether the converse holds.
Voigtländer [4] showed that two-values testing is insufficient.Consider a function . Apparently, f is not equivalent to scanl.Nevertheless, two-values testing cannot detect this flaw: every associative (⊕)::Two → Two → Two satisfies the following equation for any a 0 , a 1 ::Two: ( He then showed that three-values testing is sufficient: if f passed the three-valued testing, it implements scanl for associative operators.The proof is omitted because its detail is not relevant to this paper.

Prefix Sum with Associative Idempotent Operator
This paper shows that two-values testing cannot distinguish an incorrect implementation of scanl for associative idempotent operators.
Eq.( 2) Note A, C, and I denote associativity, commutativity, and idempotency, respectively.When ⊕ is not associative, columns about Eqs. ( 1) and ( 2) are filled with − because they are meaningless Consider function f such that f (⊕) [a 0 , a 1 , a 2 , a 3 ] !! 3 = a 0 ⊕ a 2 ⊕ a 1 ⊕ a 3 .This implementation is incorrect because the order of a 1 and a 2 are reversed. 3However, twovalues testing cannot detect this flaw: for any associative (⊕)::Two → Two → Two and a i ::Two (0 ≤ i ≤ 3), the following equation holds. ( Since each hand-side expression uses every value at most once, idempotency is irrelevant.Table 1lists all binary operators on Two.There are 16 operators in total, eight are associative and four are associative and idempotent.From an exhaustive case analysis, we can verify that every associative idempotent operator satisfies Eq. ( 2).
It is possible to provide another, more axiomatic proof to Eq. ( 2).Because it trivially holds when a 1 ⊕ a 2 = a 2 ⊕ a 1 , we assume that ⊕ is not commutative.Since ⊕ is idempotent, there are only two possibilities.

Prefix Sum with Associative Commutative Idempotent Operator
If the binary operator satisfies commutativity in addition to associativity and idempotency, two-values testing is sufficient.The following proof takes the same approach as those of Knuth's 0-1 principle by Day et al. [12] and 0-1-2 principle by Voigtländer [4].
The proof is based on the intensive use of relational parametricity [13] (also known as Wadler's free theorem [14]) for polymorphic types.In particular, the following lemma is the key.

Lemma 1 For any function f
, and function g:: A → B, the following equation holds, Proof It is an immediate consequence of the relational parametricity applied to the polymorphic type of f .The sufficiency of two-values testing can be formulated as the following theorem.Note that this paper shows a stronger argument: it is sufficient to test only the binary disjunction (∨) defined below.

Proof
The "only if" direction is trivial.The proof of the "if" direction consists of the following two propositions.
Proposition 1 is an instance of a more general theory for testing polymorphic functions [15,16]. 4It is proven here to make the presentation self-contained.The strategy is to show that f (∪) (map wrap = scanl (∪) (map wrap x) indicates f (⊕) x = scanl (⊕) x for any associative, commutative, and idempotent ⊕.In the following reasoning, (•) denotes the function composition operator: The definition of sum ⊕ assumes the associativity and commutativity of ⊕.Moreover, the use of Lemma 1 exploits idempotency: if ⊕ is not idempotent, sum ⊕ (x 1 ∪ x 2 ) = sum ⊕ x 1 ⊕ sum ⊕ x 2 does not hold when x 1 and x 2 share some elements in common.
Next, Proposition 2 is proven.Assume One might wonder whether two-values testing suffices if the binary operator is associative and commutative but not idempotent.It does not.Equation ( 1) is invalid even if ⊕ is commutative.[4] showed that O(n 2 ) test cases are sufficient for verifying a prefix sum for associative operators with n input elements.In fact, n test cases are sufficient when the operator is associative, commutative, and idempotent.The proof of Theorem 2 essentially checks whether a particular element, b * , is in a certain position, m, by setting 1 on the element corresponding to b * and 0 on others.

Related Work
There are a few other studies on testing-based verification of prefix-sum algorithms.Sheeran [5] proposed a test method similar to Proposition 1 in the proof of Theorem 2: preparing labels that stand for the elements then checking whether the output consists of appropriate sets of labels.It requires only a single test, the input and output of which, respectively, consume 2(n log n) and 2(n 2 log n) spaces, where n is the number of input elements.It is applicable even when the operator is commutative and/or associative.
Chong et al. [6] improved Sheeran's method by using intervals instead of sets of labels.For example, given a list [x 0 , x 1 , x 2 , x 3 ], their approach represents x 0 ⊕ x 1 ⊕ x 2 ⊕ x 3 by an interval, namely, a pair of the first and last indices, (0, 3).The output of their method consumes only 2(n log n) spaces.However, we cannot use it when the operator ⊕ is commutative in addition to associative.For example, if we calculate x 0 ⊕ x 1 ⊕ x 2 ⊕ x 3 by (x 0 ⊕ x 2 ) ⊕ (x 1 ⊕ x 3 ), neither x 0 ⊕ x 2 nor x 1 ⊕ x 3 corresponds to any interval.
This paper has developed a verification with n test cases for a prefix sum for associative commutative idempotent operators.By merging them into one with n-bits vectors, one can conduct an equivalent verification by a single test, the input and output of which require 2(n 2 ) spaces.The efficiency of this approach is incomparable with Sheeran's.

Conclusion
This paper proved that two-values testing is sufficient for the prefix sum algorithms with associative, commutative, and idempotent operators but not for those with associative and idempotent ones.It thereby answered the question raised by Voigtländer [4].These results will provide more understanding behind Voigtländer's 0-1-2 principle and may lead to similar principles for other computation patterns.

Fig. 1 Fig. 2
Fig. 1 Sorting network with four wires and five comparators Define contain b * and is b * as follows.containb * z = if b * ∈ z then 1 else 0 is b * a = if b * == a then1 else 0 Since only one of r 1 and r 2 contains b * , contain b * r 1 = contain b * r 2 .The following routine reasoning proves f (∨) y = scanl (∨) y, where y = map is b * x.

Table 1
Properties of binary operators on Two