Proving Divide and Conquer Complexities in Isabelle/HOL

Manuel Eberl

doi:10.1007/s10817-016-9378-0

Journal of Automated Reasoning

April 2017, Volume 58, Issue 4, pp 483–508 | Cite as

Proving Divide and Conquer Complexities in Isabelle/HOL

Authors
Authors and affiliations

Manuel Eberl

Article

First Online: 13 June 2016

216 Downloads
3 Citations

Abstract

The Akra–Bazzi method (Akra and Bazzi in Comput Optim Appl 10(2):195–210, 1998. doi: 10.1023/A:1018373005182), a generalisation of the well-known Master Theorem, is a useful tool for analysing the complexity of Divide and Conquer algorithms. This work describes a formalisation of the Akra–Bazzi method (as generalised by Leighton in Notes on better Master theorems for divide-and-conquer recurrences, 1996. http://courses.csail.mit.edu/6.046/spring04/handouts/akrabazzi.pdf) in the interactive theorem prover Isabelle/HOL and the derivation of a generalised version of the Master Theorem from it. We also provide some automated proof methods that facilitate the application of this Master Theorem and allow mostly automatic verification of $\varTheta $-bounds for these Divide and Conquer recurrences. To our knowledge, this is the first formalisation of theorems for the analysis of such recurrences.

Keywords

Isabelle/HOL Master Theorem Akra–Bazzi Divide and Conquer algorithms Recurrences Complexity Landau symbols

This is a preview of subscription content, to check access.

Notes

Acknowledgments

Tobias Nipkow and Johannes Hölzl commented on an early draft of this work. Louay Bazzi made a very helpful suggestion that allowed us to fix the missing case in Leighton’s proof. We also thank the anonymous reviewers for their helpful suggestions and insightful questions.

Appendix: The Class of Admissible Integrals

We will now provide a proof of our claim from Sect. 3.3 that any admissible integral must coincide with the Riemann integral for all piecewise-continuous functions. First of all, we shall state again what admissibility means formally: Consider an integral notion ${\mathcal {I}}$. Formally, ${\mathcal {I}}$ consists of

A functional ${\mathcal {I}}: \mathbb {R}^ \mathbb {R}\times \mathbb {R}^2 \rightarrow \mathbb {R}$ that maps a function f and interval bounds a and b to the ${\mathcal {I}}$-integral of f from a to b, denoted by
$$\begin{aligned} \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} f(x)\,\text {d}x. \end{aligned}$$
A set ${\mathcal {I}} \subseteq \mathbb {R}^\mathbb {R}\times \mathbb {R}^2$, the set of ${\mathcal {I}}$-integrable functions. We say that f is ${\mathcal {I}}$-integrable on [a; b] if $(f,a,b) \in {\mathcal {I}}$.

For the remainder of this section, we will always indicate for every use of the integral sign which notion of integration is meant by writing the name of the integral underneath the integral as we did above.

Now assume that ${\mathcal {I}}$ fulfils the following four properties (which are the same that were stated in Sect. 3.3, but more formally):

$$\begin{aligned}&\forall a,b,c\in \mathbb {R}. \quad a \le b \ \wedge \ c \ge 0\ \longrightarrow \ ((\uplambda \, x.\ c), a, b) \in {\mathcal {I}} \ \wedge \ \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} c \text {d}x = c\cdot (a - b) \end{aligned}$$

(9)

$$\begin{aligned}&\forall a,b,a',b'\in \mathbb {R}.\ \forall f\in \mathbb {R}^\mathbb {R}. \quad a\le a' \le b' \le b \ \wedge \ (f,a,b)\in {\mathcal {I}}\ \longrightarrow \ (f,a',b')\in {\mathcal {I}} \end{aligned}$$

(10)

$$\begin{aligned}&\forall a,b\in \mathbb {R}.\ \forall f,g\in \mathbb {R}^\mathbb {R}. \quad (f,a,b)\in {\mathcal {I}} \ \wedge \ (g,a,b)\in {\mathcal {I}}\ \wedge \ (\forall x\in [a;b].\ f(x)\le g(x))\nonumber \\&\qquad \ \longrightarrow \ \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} f(x)\,\text {d}x \le \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} g(x)\,\text {d}x \end{aligned}$$

(11)

$$\begin{aligned}&\forall a,b,c\in \mathbb {R}.\ \forall f\in \mathbb {R}^\mathbb {R}.\quad a \le b \le c \ \wedge \ (f,a,c)\in {\mathcal {I}}\nonumber \\&\qquad \ \longrightarrow \ \underset{{\mathcal {I}} \phantom {c}}{\int _{a}^{c}} f(x)\,\text {d}x \; = \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} f(x)\,\text {d}x+\underset{{\mathcal {I}} \phantom {c}}{\int _{b}^{c}} f(x)\,\text {d}x \end{aligned}$$

(12)

To show that the ${\mathcal {I}}$-integral of a non-negative function f over [a; b] coincides with the Riemann integral if f is both ${\mathcal {I}}$-integrable and piecewise-continuous on [a; b], we recall that the Riemann integral is equivalent to the Darboux integral. The Darboux integral is defined as the supremum $L_f$ of the lower Darboux sums $L_{f,P}$ and the infimum $U_f$ of the upper Darboux sums $U_{f,P}$ over all subdivisions $P=(c_0\ldots c_n)$ of the interval [a; b] whenever $L_f$ and $U_f$ are equal. Formally:

Open image in new window

where

$$\begin{aligned}&L_f = \sup \limits _P\ L_{f,P} = \sup \limits _{P}\ \sum _{i=0}^{n-1} (c_{i+1}-c_i) \inf \limits _{\scriptscriptstyle x\in [c_i;\,c_{i+1}]}\ f(x)\quad \text {and}\quad U_f = \inf \limits _P\ U_{f,P}\\&\quad \quad =\inf \limits _{P}\ \sum _{i=0}^{n-1} (c_{i+1}-c_i) \sup \limits _{\scriptscriptstyle x\in [c_i;\,c_{i+1}]}\ f(x) \end{aligned}$$

Suppose f is non-negative, continuous, and ${\mathcal {I}}$-integrable on [a; b]. For any subdivision $P = (c_0\ldots c_n)$ of the interval [a; b], we can use (11) and (12) to split up the ${\mathcal {I}}$-integral over [a; b]:

$$\begin{aligned} \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} f(x)\,\text {d}x = \sum _{i=0}^{n-1} \underset{{\mathcal {I}} \phantom {c_{i+1}}}{\int _{c_i}^{c_{i+1}}} f(x)\,\text {d}x \end{aligned}$$

Using this together with the monotonicity and constant-interval property of ${\mathcal {I}}$, we have:

$$\begin{aligned} L_{f,P}&\mathop {=}\limits ^{\text {def}} \sum _{i=0}^{n-1} (c_{i+1} - c_i) \inf \limits _{\scriptscriptstyle x\in [c_i;\,c_{i+1}]} f(x) \mathop {=}\limits ^{(9)} \sum _{i=0}^{n-1} \underset{{\mathcal {I}} \phantom {c_{i+1}}}{\int _{c_i}^{c_{i+1}}} \inf \limits _{\scriptscriptstyle x\in [c_i;\,c_{i+1}]} f(x)\, \text {d}x \mathop {\le }\limits ^{(11)} \underbrace{\sum _{i=0}^{n-1} \underset{{\mathcal {I}} \phantom {c_{i+1}}}{\int _{c_i}^{c_{i+1}}} f(x)\,\text {d}x}_{=\,\underset{{\mathcal {I}}}{\int _a^b} f(x) \,\text {d}x} \mathop {\le }\limits ^{(11)}\\&\le \sum _{i=0}^{n-1} \underset{{\mathcal {I}} \phantom {c_{i+1}}}{\int _{c_i}^{c_{i+1}}} \sup \limits _{\scriptscriptstyle x\in [c_i;\,c_{i+1}]} f(x) \mathop {=}\limits ^{(9)} \sum _{i=0}^{n-1} (c_{i+1}-c_i) \sup \limits _{\scriptscriptstyle x\in [c_i;\,c_{i+1}]} f(x) \mathop {=}\limits ^{\text {def}} U_{f,P} \end{aligned}$$

Therefore, the ${\mathcal {I}}$-integral lies between all lower and upper Darboux sums. Since f is continuous, f is also Darboux-integrable, and therefore the supremum of the lower Darboux sums and the infimum of the upper Darboux sums are the same. Since the ${\mathcal {I}}$-integral lies inbetween, we have:

$$\begin{aligned} \underset{\scriptscriptstyle \text {Darboux}}{\int _a^b} f(x)\, \text {d}x \mathop {=}\limits ^{\text {def}} L_f = U_f = \sum _{i=0}^{n-1} \underset{{\mathcal {I}} \phantom {c_{i+1}}}{\int _{c_i}^{c_{i+1}}} f(x)\,\text {d}x = \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} f(x)\, \text {d}x. \end{aligned}$$

We have therefore shown that ${\mathcal {I}}$ coincides with the Riemann integral on all continuous non-negative functions. Since we can split the ${\mathcal {I}}$-integral of a piecewise-continuous function into a sum of ${\mathcal {I}}$-integrals of continuous functions, this extends to all piecewise-continuous non-negative functions.

However, this result does not extend to more general notions of integrals and integrability: For example, let us consider the following integral ${\mathcal {I}}$: A function is ${\mathcal {I}}$-integrable if it is a constant function or if it is $[\mathbb {Q}] = (\uplambda \,x.\ \text {if}\ x\in \mathbb {Q}\ \text {then}\ 1\ \text {else}\ 0)$, the indicator function of the rational numbers. The value of the integral is defined as

$$\begin{aligned} \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} c\, \text {d}x {:}{=} c\cdot (b - a) \quad \quad \text {and}\quad \quad \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} [\mathbb {Q}]\,\text {d}x {:}{=} b - a \end{aligned}$$

Then ${\mathcal {I}}$ fulfils all four properties, but unlike the Lebesgue/Bochner/Henstock–Kurzweil integral, the ${\mathcal {I}}$-integral of $[\mathbb {Q}]$ is non-zero on all non-empty intervals.

References

1.
Akra, M., Bazzi, L.: On the solution of linear recurrence equations. Comput. Optim. Appl. 10(2), 195–210 (1998). doi: 10.1023/A:1018373005182 MathSciNetCrossRefzbMATHGoogle Scholar
2.
Avigad, J., Donnelly, K.: Formalizing $O$ notation in Isabelle/HOL. In: Basin, D., Rusinowitch, M. (eds.) Automated Reasoning, Lecture Notes in Computer Science, pp. 357–371. Springer, Berlin (2004). doi: 10.1007/978-3-540-25984-8_27 Google Scholar
3.
Avigad, J., Hölzl, J., Serafin, L.: A formally verified proof of the central limit theorem. CoRR abs/1405.7012 (2014). Presented at the Isabelle Workshop 2014Google Scholar
4.
Ballarin, C.: Locales: a module system for mathematical theories. J. Autom. Reason. 52(2), 123–153 (2014). doi: 10.1007/s10817-013-9284-7 MathSciNetCrossRefzbMATHGoogle Scholar
5.
Bazzi, L., Mitter, S.K.: The solution of linear probabilistic recurrence relations. Algorithmica 36(1), 41–57 (2003). doi: 10.1007/s00453-002-1003-4 MathSciNetCrossRefzbMATHGoogle Scholar
6.
Boncelet Jr., C.G.: Block arithmetic coding for source compression. IEEE Trans. Inf. Theory 39(5), 1546–1554 (1993). doi: 10.1109/18.259639 CrossRefzbMATHGoogle Scholar
7.
Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 4th printing, 3rd edn. The MIT Press, Cambridge (2009)zbMATHGoogle Scholar
8.
Drmota, M., Szpankowski, W.: A Master theorem for discrete divide and conquer recurrences. J. ACM 60(3), 16:1–16:49 (2013). doi: 10.1145/2487241.2487242 MathSciNetCrossRefzbMATHGoogle Scholar
9.
Eberl, M.: The Akra–Bazzi Theorem and the Master Theorem. Archive of Formal Proofs (2015). http://www.isa-afp.org/entries/Akra_Bazzi.shtml, Formal proof development
10.
Eberl, M.: Landau Symbols. Archive of Formal Proofs (2015). http://www.isa-afp.org/entries/Landau_Symbols.shtml, Formal proof development
11.
Haftmann, F., Nipkow, T.: Code generation via higher-order rewrite systems. In: Blume, M., Kobayashi, N., Vidal, G. (eds.) Functional and Logic Programming. Lecture Notes in Computer Science, vol. 6009, pp. 103–117. Springer, Berlin (2010). doi: 10.1007/978-3-642-12251-4_9 CrossRefGoogle Scholar
12.
Hölzl, J.: Proving inequalities over reals with computation in Isabelle/HOL. In: Reis, G.D., Théry, L. (eds.) Proceedings of the ACM SIGSAM 2009 International Workshop on Programming Languages for Mechanized Mathematics Systems (PLMMS’09), pp. 38–45. Munich (2009)Google Scholar
13.
Krauss, A.: Automating Recursive Definitions and Termination Proofs in Higher-Order Logic. Ph.D. thesis, Technische Universität München, Institut für Informatik (2009). http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:91-diss-20090722-681651-1-1
14.
Leighton, T.: Notes on Better Master Theorems for Divide-and-Conquer Recurrences (1996). http://courses.csail.mit.edu/6.046/spring04/handouts/akrabazzi.pdf

Copyright information

Authors and Affiliations

Manuel Eberl
- 1
Email authorView author's OrcID profile

1.Fakultät für InformatikTechnische Universität MünchenGarchingGermany

Proving Divide and Conquer Complexities in Isabelle/HOL

Abstract

Keywords

Notes

Acknowledgments

References

Copyright information

Authors and Affiliations

Personalised recommendations

Cite article