Skip to main content

Proving Divide and Conquer Complexities in Isabelle/HOL

Abstract

The Akra–Bazzi method (Akra and Bazzi in Comput Optim Appl 10(2):195–210, 1998. doi:10.1023/A:1018373005182), a generalisation of the well-known Master Theorem, is a useful tool for analysing the complexity of Divide and Conquer algorithms. This work describes a formalisation of the Akra–Bazzi method (as generalised by Leighton in Notes on better Master theorems for divide-and-conquer recurrences, 1996. http://courses.csail.mit.edu/6.046/spring04/handouts/akrabazzi.pdf) in the interactive theorem prover Isabelle/HOL and the derivation of a generalised version of the Master Theorem from it. We also provide some automated proof methods that facilitate the application of this Master Theorem and allow mostly automatic verification of \(\varTheta \)-bounds for these Divide and Conquer recurrences. To our knowledge, this is the first formalisation of theorems for the analysis of such recurrences.

This is a preview of subscription content, access via your institution.

Fig. 1

Notes

  1. In Introduction to Algorithms [7], for example, only the ‘floor’ case is proven and the ‘ceiling’ case is stated to be analogous. The question of what happens when both floors and ceilings are used is not addressed.

  2. His conditions are slightly more restrictive; among other things, he requires them to hold for all \(x\ge 1\).

  3. Note that due to the other constraints on f and g, the condition that f(x) is positive for all sufficiently large x must hold if either g(x) is positive for all sufficiently large x or f(x) is positive in all the base cases, i. e. for \(x\in [x_0;x_1)\).

  4. The notation here becomes a bit informal. Terms like \(f(x) + O(g(x))\) stand for the set \(\{f(x) + h(x)\ |\ h(x) \in O(g(x))\}\) and all equality symbols are then essentially set inclusions, i. e. \(f(x) + O(\ldots ) = g(x) + O(\ldots )\) means that the left-hand side is a subset of the right-hand side.

  5. Note the implicit case distinction for \(p \ge 0\) and \(p < 0\): for \(p \ge 0\), we need to show \(b_i x - Hx \ln ^{-1-\varepsilon } x \le b_i x + h_i(x)\) , whereas for \(p < 0\) we need to show \(b_i x + Hx \ln ^{-1-\varepsilon } x \ge b_i x + h_i(x)\) since the negative exponent flips the inequality. This case distinction is not mentioned by Leighton.

  6. The Master Theorem presented in the book actually demands \(q = 0\), but Exercise 4.2–2 is a generalisation to \(q \ge 0\).

References

  1. Akra, M., Bazzi, L.: On the solution of linear recurrence equations. Comput. Optim. Appl. 10(2), 195–210 (1998). doi:10.1023/A:1018373005182

    MathSciNet  Article  MATH  Google Scholar 

  2. Avigad, J., Donnelly, K.: Formalizing \(O\) notation in Isabelle/HOL. In: Basin, D., Rusinowitch, M. (eds.) Automated Reasoning, Lecture Notes in Computer Science, pp. 357–371. Springer, Berlin (2004). doi:10.1007/978-3-540-25984-8_27

    Google Scholar 

  3. Avigad, J., Hölzl, J., Serafin, L.: A formally verified proof of the central limit theorem. CoRR abs/1405.7012 (2014). Presented at the Isabelle Workshop 2014

  4. Ballarin, C.: Locales: a module system for mathematical theories. J. Autom. Reason. 52(2), 123–153 (2014). doi:10.1007/s10817-013-9284-7

    MathSciNet  Article  MATH  Google Scholar 

  5. Bazzi, L., Mitter, S.K.: The solution of linear probabilistic recurrence relations. Algorithmica 36(1), 41–57 (2003). doi:10.1007/s00453-002-1003-4

    MathSciNet  Article  MATH  Google Scholar 

  6. Boncelet Jr., C.G.: Block arithmetic coding for source compression. IEEE Trans. Inf. Theory 39(5), 1546–1554 (1993). doi:10.1109/18.259639

    Article  MATH  Google Scholar 

  7. Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 4th printing, 3rd edn. The MIT Press, Cambridge (2009)

    MATH  Google Scholar 

  8. Drmota, M., Szpankowski, W.: A Master theorem for discrete divide and conquer recurrences. J. ACM 60(3), 16:1–16:49 (2013). doi:10.1145/2487241.2487242

    MathSciNet  Article  MATH  Google Scholar 

  9. Eberl, M.: The Akra–Bazzi Theorem and the Master Theorem. Archive of Formal Proofs (2015). http://www.isa-afp.org/entries/Akra_Bazzi.shtml, Formal proof development

  10. Eberl, M.: Landau Symbols. Archive of Formal Proofs (2015). http://www.isa-afp.org/entries/Landau_Symbols.shtml, Formal proof development

  11. Haftmann, F., Nipkow, T.: Code generation via higher-order rewrite systems. In: Blume, M., Kobayashi, N., Vidal, G. (eds.) Functional and Logic Programming. Lecture Notes in Computer Science, vol. 6009, pp. 103–117. Springer, Berlin (2010). doi:10.1007/978-3-642-12251-4_9

    Chapter  Google Scholar 

  12. Hölzl, J.: Proving inequalities over reals with computation in Isabelle/HOL. In: Reis, G.D., Théry, L. (eds.) Proceedings of the ACM SIGSAM 2009 International Workshop on Programming Languages for Mechanized Mathematics Systems (PLMMS’09), pp. 38–45. Munich (2009)

  13. Krauss, A.: Automating Recursive Definitions and Termination Proofs in Higher-Order Logic. Ph.D. thesis, Technische Universität München, Institut für Informatik (2009). http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:91-diss-20090722-681651-1-1

  14. Leighton, T.: Notes on Better Master Theorems for Divide-and-Conquer Recurrences (1996). http://courses.csail.mit.edu/6.046/spring04/handouts/akrabazzi.pdf

Download references

Acknowledgments

Tobias Nipkow and Johannes Hölzl commented on an early draft of this work. Louay Bazzi made a very helpful suggestion that allowed us to fix the missing case in Leighton’s proof. We also thank the anonymous reviewers for their helpful suggestions and insightful questions.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Manuel Eberl.

Additional information

This work was supported by DFG RTG 1480 (PUMA).

Appendix: The Class of Admissible Integrals

Appendix: The Class of Admissible Integrals

We will now provide a proof of our claim from Sect. 3.3 that any admissible integral must coincide with the Riemann integral for all piecewise-continuous functions. First of all, we shall state again what admissibility means formally: Consider an integral notion \({\mathcal {I}}\). Formally, \({\mathcal {I}}\) consists of

  • A functional \({\mathcal {I}}: \mathbb {R}^ \mathbb {R}\times \mathbb {R}^2 \rightarrow \mathbb {R}\) that maps a function f and interval bounds a and b to the \({\mathcal {I}}\)-integral of f from a to b, denoted by

    $$\begin{aligned} \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} f(x)\,\text {d}x. \end{aligned}$$
  • A set \({\mathcal {I}} \subseteq \mathbb {R}^\mathbb {R}\times \mathbb {R}^2\), the set of \({\mathcal {I}}\)-integrable functions. We say that f is \({\mathcal {I}}\)-integrable on [ab] if \((f,a,b) \in {\mathcal {I}}\).

For the remainder of this section, we will always indicate for every use of the integral sign which notion of integration is meant by writing the name of the integral underneath the integral as we did above.

Now assume that \({\mathcal {I}}\) fulfils the following four properties (which are the same that were stated in Sect. 3.3, but more formally):

$$\begin{aligned}&\forall a,b,c\in \mathbb {R}. \quad a \le b \ \wedge \ c \ge 0\ \longrightarrow \ ((\uplambda \, x.\ c), a, b) \in {\mathcal {I}} \ \wedge \ \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} c \text {d}x = c\cdot (a - b) \end{aligned}$$
(9)
$$\begin{aligned}&\forall a,b,a',b'\in \mathbb {R}.\ \forall f\in \mathbb {R}^\mathbb {R}. \quad a\le a' \le b' \le b \ \wedge \ (f,a,b)\in {\mathcal {I}}\ \longrightarrow \ (f,a',b')\in {\mathcal {I}} \end{aligned}$$
(10)
$$\begin{aligned}&\forall a,b\in \mathbb {R}.\ \forall f,g\in \mathbb {R}^\mathbb {R}. \quad (f,a,b)\in {\mathcal {I}} \ \wedge \ (g,a,b)\in {\mathcal {I}}\ \wedge \ (\forall x\in [a;b].\ f(x)\le g(x))\nonumber \\&\qquad \ \longrightarrow \ \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} f(x)\,\text {d}x \le \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} g(x)\,\text {d}x \end{aligned}$$
(11)
$$\begin{aligned}&\forall a,b,c\in \mathbb {R}.\ \forall f\in \mathbb {R}^\mathbb {R}.\quad a \le b \le c \ \wedge \ (f,a,c)\in {\mathcal {I}}\nonumber \\&\qquad \ \longrightarrow \ \underset{{\mathcal {I}} \phantom {c}}{\int _{a}^{c}} f(x)\,\text {d}x \; = \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} f(x)\,\text {d}x+\underset{{\mathcal {I}} \phantom {c}}{\int _{b}^{c}} f(x)\,\text {d}x \end{aligned}$$
(12)

To show that the \({\mathcal {I}}\)-integral of a non-negative function f over [ab] coincides with the Riemann integral if f is both \({\mathcal {I}}\)-integrable and piecewise-continuous on [ab], we recall that the Riemann integral is equivalent to the Darboux integral. The Darboux integral is defined as the supremum \(L_f\) of the lower Darboux sums \(L_{f,P}\) and the infimum \(U_f\) of the upper Darboux sums \(U_{f,P}\) over all subdivisions \(P=(c_0\ldots c_n)\) of the interval [ab] whenever \(L_f\) and \(U_f\) are equal. Formally:

where

$$\begin{aligned}&L_f = \sup \limits _P\ L_{f,P} = \sup \limits _{P}\ \sum _{i=0}^{n-1} (c_{i+1}-c_i) \inf \limits _{\scriptscriptstyle x\in [c_i;\,c_{i+1}]}\ f(x)\quad \text {and}\quad U_f = \inf \limits _P\ U_{f,P}\\&\quad \quad =\inf \limits _{P}\ \sum _{i=0}^{n-1} (c_{i+1}-c_i) \sup \limits _{\scriptscriptstyle x\in [c_i;\,c_{i+1}]}\ f(x) \end{aligned}$$

Suppose f is non-negative, continuous, and \({\mathcal {I}}\)-integrable on [ab]. For any subdivision \(P = (c_0\ldots c_n)\) of the interval [ab], we can use (11) and (12) to split up the \({\mathcal {I}}\)-integral over [ab]:

$$\begin{aligned} \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} f(x)\,\text {d}x = \sum _{i=0}^{n-1} \underset{{\mathcal {I}} \phantom {c_{i+1}}}{\int _{c_i}^{c_{i+1}}} f(x)\,\text {d}x \end{aligned}$$

Using this together with the monotonicity and constant-interval property of \({\mathcal {I}}\), we have:

$$\begin{aligned} L_{f,P}&\mathop {=}\limits ^{\text {def}} \sum _{i=0}^{n-1} (c_{i+1} - c_i) \inf \limits _{\scriptscriptstyle x\in [c_i;\,c_{i+1}]} f(x) \mathop {=}\limits ^{(9)} \sum _{i=0}^{n-1} \underset{{\mathcal {I}} \phantom {c_{i+1}}}{\int _{c_i}^{c_{i+1}}} \inf \limits _{\scriptscriptstyle x\in [c_i;\,c_{i+1}]} f(x)\, \text {d}x \mathop {\le }\limits ^{(11)} \underbrace{\sum _{i=0}^{n-1} \underset{{\mathcal {I}} \phantom {c_{i+1}}}{\int _{c_i}^{c_{i+1}}} f(x)\,\text {d}x}_{=\,\underset{{\mathcal {I}}}{\int _a^b} f(x) \,\text {d}x} \mathop {\le }\limits ^{(11)}\\&\le \sum _{i=0}^{n-1} \underset{{\mathcal {I}} \phantom {c_{i+1}}}{\int _{c_i}^{c_{i+1}}} \sup \limits _{\scriptscriptstyle x\in [c_i;\,c_{i+1}]} f(x) \mathop {=}\limits ^{(9)} \sum _{i=0}^{n-1} (c_{i+1}-c_i) \sup \limits _{\scriptscriptstyle x\in [c_i;\,c_{i+1}]} f(x) \mathop {=}\limits ^{\text {def}} U_{f,P} \end{aligned}$$

Therefore, the \({\mathcal {I}}\)-integral lies between all lower and upper Darboux sums. Since f is continuous, f is also Darboux-integrable, and therefore the supremum of the lower Darboux sums and the infimum of the upper Darboux sums are the same. Since the \({\mathcal {I}}\)-integral lies inbetween, we have:

$$\begin{aligned} \underset{\scriptscriptstyle \text {Darboux}}{\int _a^b} f(x)\, \text {d}x \mathop {=}\limits ^{\text {def}} L_f = U_f = \sum _{i=0}^{n-1} \underset{{\mathcal {I}} \phantom {c_{i+1}}}{\int _{c_i}^{c_{i+1}}} f(x)\,\text {d}x = \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} f(x)\, \text {d}x. \end{aligned}$$

We have therefore shown that \({\mathcal {I}}\) coincides with the Riemann integral on all continuous non-negative functions. Since we can split the \({\mathcal {I}}\)-integral of a piecewise-continuous function into a sum of \({\mathcal {I}}\)-integrals of continuous functions, this extends to all piecewise-continuous non-negative functions.

However, this result does not extend to more general notions of integrals and integrability: For example, let us consider the following integral \({\mathcal {I}}\): A function is \({\mathcal {I}}\)-integrable if it is a constant function or if it is \([\mathbb {Q}] = (\uplambda \,x.\ \text {if}\ x\in \mathbb {Q}\ \text {then}\ 1\ \text {else}\ 0)\), the indicator function of the rational numbers. The value of the integral is defined as

$$\begin{aligned} \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} c\, \text {d}x {:}{=} c\cdot (b - a) \quad \quad \text {and}\quad \quad \underset{{\mathcal {I}} \phantom {b}}{\int _{a}^{b}} [\mathbb {Q}]\,\text {d}x {:}{=} b - a \end{aligned}$$

Then \({\mathcal {I}}\) fulfils all four properties, but unlike the Lebesgue/Bochner/Henstock–Kurzweil integral, the \({\mathcal {I}}\)-integral of \([\mathbb {Q}]\) is non-zero on all non-empty intervals.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Eberl, M. Proving Divide and Conquer Complexities in Isabelle/HOL. J Autom Reasoning 58, 483–508 (2017). https://doi.org/10.1007/s10817-016-9378-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10817-016-9378-0

Keywords

  • Isabelle/HOL
  • Master Theorem
  • Akra–Bazzi
  • Divide and Conquer algorithms
  • Recurrences
  • Complexity
  • Landau symbols