Dynamic Programming via Static Incrementalization

  • Yanhong A. Liu
  • Scott D. Stoller
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1576)

Abstract

Dynamic programming is an important algorithm design technique. It is used for solving problems whose solutions involve recursively solving subproblems that share subsubproblems. While a straightforward recursive program solves common subsubproblems repeatedly and often takes exponential time, a dynamic programming algorithm solves every subsubproblem just once, saves the result, reuses it when the subsubproblem is encountered again, and takes polynomial time. This paper describes a systematic method for transforming programs written as straightforward recursions into programs that use dynamic programming. The method extends the original program to cache all possibly computed values, incrementalizes the extended program with respect to an input increment to use and maintain all cached results, prunes out cached results that are not used in the incremental computation, and uses the resulting incremental program to form an optimized new program. Incrementalization statically exploits semantics of both control structures and data structures and maintains as invariants equalities characterizing cached results. The principle underlying incrementalization is general for achieving drastic program speedups. Compared with previous methods that perform memoization or tabulation, the method based on incrementalization is more powerful and systematic. It has been implemented and applied to numerous problems and succeeded on all of them.

References

  1. 1.
    M. Abadi, B. Lampson, and J.-J. Lévy. Analysis and caching of dependencies. In Proceedings of the 1996 ACM SIGPLAN International Conference on Functional Programming. ACM, New York, May 1996.Google Scholar
  2. 2.
    A. V. Aho, J. E. Hopcroft, and J. D. Ullman. The Design and Analysis of Computer Algorithms. Addison-Wesley, Reading, Mass., 1974.MATHGoogle Scholar
  3. 3.
    F. L. Bauer, B. Möller, H. Partsch, and P. Pepper. Formal program construction by transformations — Computer-aided, intuition-guided programming. IEEE Trans. Softw. Eng., 15(2):165–180, Feb. 1989.MATHCrossRefGoogle Scholar
  4. 4.
    R. E. Bellman. Dynamic Programming. Princeton University Press, Princeton, New Jersey, 1957.Google Scholar
  5. 5.
    R. S. Bird. Tabulation techniques for recursive programs. ACM Comput. Surv., 12(4):403–417, Dec. 1980.MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    R. S. Bird. The promotion and accumulation strategies in transformational programming. ACM Trans. Program. Lang. Syst., 6(4):487–504, Oct. 1984.MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    R. S. Bird and O. de Moor. From dynamic programming to greedy algorithms. In B. Möller, H. Partsch, and S. Schuman, editors, Formal Program Development, volume 755 of Lecture Notes in Computer Science, pages 43–61. Springer-Verlag, Berlin, 1993.Google Scholar
  8. 8.
    E. A. Boiten. Improving recursive functions by inverting the order of evaluation. Sci. Comput. Program., 18(2):139–179, Apr. 1992.MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    R. M. Burstall and J. Darlington. A transformation system for developing recursive programs. J. ACM, 24(1):44–67, Jan. 1977.MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    W.-N. Chin. Towards an automated tupling strategy. In Proceedings of the ACM SIGPLAN Symposium on Partial Evaluation and Semantics-Based Program Manipulation, pages 119–132. ACM, New York, June 1993.CrossRefGoogle Scholar
  11. 11.
    W.-N. Chin and M. Hagiya. A bounds inference method for vector-based memorization. In ICFP 1997 [23], pages 176–187.Google Scholar
  12. 12.
    W.-N. Chin and S.-C. Khoo. Tupling functions with multiple recursion parameters. In P. Cousot, M. Falaschi, G. Filé, and A. Rauzy, editors, Proceedings of the 3rd International Workshop on Static Analysis, volume 724 of Lecture Notes in Computer Science, pages 124–140. Springer-Verlag, Berlin, Sept. 1993.Google Scholar
  13. 13.
    N. H. Cohen. Eliminating redundant recursive calls. ACM Trans. Program. Lang. Syst., 5(3):265–299, July 1983.MATHCrossRefGoogle Scholar
  14. 14.
    T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Introduction to Algorithms. The MIT Press/McGraw-Hill, 1990.Google Scholar
  15. 15.
    S. Curtis. Dynamic programming: A different perspective. In R. Bird and L. Meertens, editors, Algorithmic Languages and Calculi, pages 1–23. Chapman & Hall, London, U.K., 1997.Google Scholar
  16. 16.
    O. de Moor. A generic program for sequential decision processes. In M. Hermenegildo and D. S. Swierstra, editors, Programming Languages: Implementations, Logics, and Programs, volume 982 of Lecture Notes in Computer Science, pages 1–23. Springer-Verlag, Berlin, 1995.CrossRefGoogle Scholar
  17. 17.
    O. de Moor and J. Gibbons. Bridging the algorithm gap: A linear-time functional program for paragraph formatting. Technical Report CMS-TR-97-03, School of Computing and Mathematical Sciences, Oxford Brookes University, July 1997.Google Scholar
  18. 18.
    J. Field and T. Teitelbaum. Incremental reduction in the lambda calculus. In Proceedings of the 1990 ACM Conference on LISP and Functional Programming, pages 307–322. ACM, New York, June 1990.CrossRefGoogle Scholar
  19. 19.
    D. P. Friedman, D. S. Wise, and M. Wand. Recursive programming through table look-up. In Proceedings of the 1976 ACM Symposium on Symbolic and Algebraic Computation, pages 85–89. ACM, New York, 1976.CrossRefGoogle Scholar
  20. 20.
    Y. Futamura and K. Nogi. Generalized partial evaluation. In B. Bjø rner, A. P. Ershov, and N. D. Jones, editors, Partial Evaluation and Mixed Computation, pages 133–151. North-Holland, Amsterdam, 1988.Google Scholar
  21. 21.
    Z. Hu, H. Iwasaki, M. Takeichi, and A. Takano. Tupling calculation eliminates multiple data traversals. In ICFP 1997 [23], pages 164–175.Google Scholar
  22. 22.
    J. Hughes. Lazy memo-functions. In Proceedings of the 2nd Conference on Functional Programming Languages and Computer Architecture, volume 201 of Lecture Notes in Computer Science, pages 129–146. Springer-Verlag, Berlin, Sept. 1985.Google Scholar
  23. 23.
    Proceedings of the 1997 ACM SIGPLAN International Conference on Functional Programming. ACM, New York, June 1997.Google Scholar
  24. 24.
    R. M. Keller and M. R. Sleep. Applicative caching. ACM Trans. Program. Lang. Syst., 8(1):88–108, Jan. 1986.MATHCrossRefGoogle Scholar
  25. 25.
    H. Khoshnevisan. Efficient memo-table management strategies. Acta Informatica, 28(1):43–81, 1990.MATHMathSciNetCrossRefGoogle Scholar
  26. 26.
    Y. A. Liu. CACHET: An interactive, incremental-attribution-based program transformation system for deriving incremental programs. In Proceedings of the 10th Knowledge-Based Software Engineering Conference, pages 19–26. IEEE CS Press, Los Alamitos, Calif., Nov. 1995.CrossRefGoogle Scholar
  27. 27.
    Y. A. Liu. Principled strength reduction. In R. Bird and L. Meertens, editors, Algorithmic Languages and Calculi, pages 357–381. Chapman & Hall, London, U.K., 1997.Google Scholar
  28. 28.
    Y. A. Liu. Dependence analysis for recursive data. In Proceedings of the IEEE 1998 International Conference on Computer Languages, pages 206–215. IEEE CS Press, Los Alamitos, Calif., May 1998.Google Scholar
  29. 29.
    Y. A. Liu and G. Gómez. Automatic accurate time-bound analysis for high-level languages. In Proceedings of the ACM SIGPLAN 1998 Workshop on Languages, Compilers, and Tools for Embedded Systems, volume 1474 of Lecture Notes in Computer Science, pages 31–40. Springer-Verlag, June 1998.Google Scholar
  30. 30.
    Y. A. Liu and S. D. Stoller. Loop optimization for aggregate array computations. In Proceedings of the IEEE 1998 International Conference on Computer Languages, pages 262–271. IEEE CS Press, Los Alamitos, Calif., May 1998.Google Scholar
  31. 31.
    Y. A. Liu, S. D. Stoller, and T. Teitelbaum. Discovering auxiliary information for incremental computation. In Conference Record of the 23rd Annual ACM Symposium on Principles of Programming Languages, pages 157–170. ACM, New York, Jan. 1996.Google Scholar
  32. 32.
    Y. A. Liu, S. D. Stoller, and T. Teitelbaum. Static caching for incremental computation. ACM Trans. Program. Lang. Syst., 20(3):546–585, May 1998.CrossRefGoogle Scholar
  33. 33.
    Y. A. Liu and T. Teitelbaum. Systematic derivation of incremental programs. Sci. Comput. Program., 24(1):1–39, Feb. 1995.MATHCrossRefGoogle Scholar
  34. 34.
    D. Michie. “memo” functions and machine learning. Nature, 218:19–22, Apr. 1968.CrossRefGoogle Scholar
  35. 35.
    D. J. Mostow and D. Cohen. Automating program speedup by deciding what to cache. In Proceedings of the 9th International Joint Conference on Artificial Intelligence, pages 165–172. Morgan Kaufmann Publishers, San Francisco, Calif., Aug. 1985.Google Scholar
  36. 36.
    R. Paige. Programming with invariants. IEEE Software, pages 56–69, Jan. 1986.Google Scholar
  37. 37.
    R. Paige. Symbolic finite differencing — Part I. In Proceedings of the 3rd European Symposium on Programming, volume 432 of Lecture Notes in Computer Science, pages 36–56. Springer-Verlag, Berlin, May 1990.Google Scholar
  38. 38.
    R. Paige and S. Koenig. Finite differencing of computable expressions. ACM Trans. Program. Lang. Syst., 4(3):402–454, July 1982.MATHCrossRefGoogle Scholar
  39. 39.
    H. A. Partsch. Specification and Transformation of Programs — A Formal Approach to Software Development. Springer-Verlag, Berlin, 1990.MATHGoogle Scholar
  40. 40.
    A. Pettorossi. A powerful strategy for deriving efficient programs by transformation. In Conference Record of the 1984 ACM Symposium on LISP and Functional Programming. ACM, New York, Aug. 1984.Google Scholar
  41. 41.
    A. Pettorossi and M. Proietti. Rules and strategies for transforming functional and logic programs. ACM Comput. Surv., 28(2):360–414, June 1996.CrossRefGoogle Scholar
  42. 42.
    A. Pettorossi and M. Proietti. Program derivation via list introduction. In R. Bird and L. Meertens, editors, Algorithmic Languages and Calculi. Chapman & Hall, London, U.K., 1997.Google Scholar
  43. 43.
    W. Pugh. An improved cache replacement strategy for function caching. In Proceedings of the 1988 ACM Conference on LISP and Functional Programming, pages 269–276. ACM, New York, July 1988.CrossRefGoogle Scholar
  44. 44.
    W. Pugh. The Omega Test: A fast and practical integer programming algorithm for dependence analysis. Commun. ACM, 31(8), Aug. 1992.Google Scholar
  45. 45.
    W. Pugh and T. Teitelbaum. Incremental computation via function caching. In Conference Record of the 16th Annual ACM Symposium on Principles of Programming Languages, pages 315–328. ACM, New York, Jan. 1989.Google Scholar
  46. 46.
    P. W. Purdom and C. A. Brown. The Analysis of Algorithms. Holt, Rinehart and Winston, 1985.Google Scholar
  47. 47.
    T. Reps and T. Teitelbaum. The Synthesizer Generator: A System for Constructing Language-Based Editors. Springer-Verlag, New York, 1988.Google Scholar
  48. 48.
    W. L. Scherlis. Program improvement by internal specialization. In Conference Record of the 8th Annual ACM Symposium on Principles of Programming Languages, pages 41–49. ACM, New York, Jan. 1981.Google Scholar
  49. 49.
    D. R. Smith. KIDS: A semiautomatic program development system. IEEE Trans. Softw. Eng., 16(9):1024–1043, Sept. 1990.CrossRefGoogle Scholar
  50. 50.
    D. R. Smith. Structure and design of problem reduction generators. In B. Möller, editor, Constructing Programs from Specifications, pages 91–124. North-Holland, Amsterdam, 1991.Google Scholar
  51. 51.
    M. Sniedovich. Dynamic Programming. Marcel Dekker, New York, 1992.MATHGoogle Scholar
  52. 52.
    B. Wegbreit. Goal-directed program transformation. IEEE Trans. Softw. Eng., SE-2(2):69–80, June 1976.MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1999

Authors and Affiliations

  • Yanhong A. Liu
    • 1
  • Scott D. Stoller
    • 1
  1. 1.Computer Science DepartmentIndiana UniversityBloomington

Personalised recommendations