Higher-Order and Symbolic Computation

, Volume 16, Issue 1–2, pp 37–62 | Cite as

Dynamic Programming via Static Incrementalization

  • Yanhong A. Liu
  • Scott D. Stoller
Article

Abstract

Dynamic programming is an important algorithm design technique. It is used for problems whose solutions involve recursively solving subproblems that share subsubproblems. While a straightforward recursive program solves common subsubproblems repeatedly, a dynamic programming algorithm solves every subsubproblem just once, saves the result, and reuses it when the subsubproblem is encountered again. This can reduce the time complexity from exponential to polynomial. This paper describes a systematic method for transforming programs written as straightforward recursions into programs that use dynamic programming. The method extends the original program to cache all possibly computed values, incrementalizes the extended program with respect to an input increment to use and maintain all cached results, prunes out cached results that are not used in the incremental computation, and uses the resulting incremental program to form an optimized new program. Incrementalization statically exploits semantics of both control structures and data structures and maintains as invariants equalities characterizing cached results. It provides the basis of a general method for achieving drastic program speedups. Compared with previous methods that perform memoization or tabulation, the method based on incrementalization is more powerful and systematic. It has been implemented in a prototype system CACHET and applied to numerous problems and succeeded on all of them.

caching dependence analysis dynamic programming incremental computation incremental update incrementalization memoization program optimization program transformation pruning tabulation static analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abadi, M., Lampson, B., and Lévy, J.-J. Analysis and caching of dependencies. In Proceedings of the 1996 ACM SIGPLAN International Conference on Functional Programming, ACM, New York, 1996, pp. 83–91.Google Scholar
  2. 2.
    Aho, A.V., Hopcroft, J.E., and Ullman, J.D. The Design and Analysis of Computer Algorithms. Addison-Wesley, Reading, MA, 1974.Google Scholar
  3. 3.
    Bauer, F.L., Möller, B., Partsch, H., and Pepper, P. Formal program construction by transformations—Computer-aided, intuition-guided programming. IEEE Trans. Softw. Eng., 15(2) (1989) 165–180.Google Scholar
  4. 4.
    Bellman, R.E. Dynamic Programming. Princeton University Press, Princeton, New Jersey, 1957.Google Scholar
  5. 5.
    Bird, R.S. Tabulation techniques for recursive programs. ACM Comput. Surv., 12(4) (1980) 403–417.Google Scholar
  6. 6.
    Bird, R.S. The promotion and accumulation strategies in transformational programming. ACM Trans. Program. Lang. Syst., 6(4) (1984) 487–504.Google Scholar
  7. 7.
    Bird, R.S. and de Moor, O. From dynamic programming to greedy algorithms. In Formal Program Development, B. Möller, H. Partsch, and S. Schuman (Eds.). Vol. 755 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 1993, pp. 43–61.Google Scholar
  8. 8.
    Boiten, E.A. Improving recursive functions by inverting the order of evaluation. Sci. Comput. Program., 18(2) (1992) 139–179.Google Scholar
  9. 9.
    Burstall, R.M. and Darlington, J. A transformation system for developing recursive programs. J. ACM, 24(1) (1977) 44–67.Google Scholar
  10. 10.
    Cai, J. and Paige, R. Program derivation by fixed point computation. Sci. Comput. Program., 11 (1988/89) 197–261.Google Scholar
  11. 11.
    Chin, W.-N. Towards an automated tupling strategy. In Proceedings of the ACM SIGPLAN Symposium on Partial Evaluation and Semantics-Based Program Manipulation, ACM, New York, 1993, pp. 119–132.Google Scholar
  12. 12.
    Chin, W.-N. and Hagiya, M. A bounds inference method for vector-based memoization. In ICFP 1997 [26], pp. 176–187.Google Scholar
  13. 13.
    Chin W.-N. and Khoo, S.-C. Tupling functions with multiple recursion parameters. In Proceedings of the 3rd International Workshop on Static Analysis, P. Cousot, M. Falaschi, G. Filè, and A. Rauzy (Eds.). Vol. 724 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, Sept. 1993, pp. 124–140.Google Scholar
  14. 14.
    Cohen, N.H. Eliminating redundant recursive calls. ACM Trans. Program. Lang. Syst., 5(3) (1983) 265–299.Google Scholar
  15. 15.
    Cormen, T.H., Leiserson, C.E., and Rivest, R.L. Introduction to Algorithms. The MIT Press/McGraw-Hill, 1990.Google Scholar
  16. 16.
    Curtis, S. Dynamic programming: A different perspective. In Algorithmic Languages and Calculi, R. Bird and L. Meertens (Eds.). Chapman & Hall, London, UK, 1997, pp. 1–23.Google Scholar
  17. 17.
    de Moor, O. A generic program for sequential decision processes. In Programming Languages: Implementations, Logics, and Programs, M. Hermenegildo and D.S. Swierstra (Eds.). Vol. 982 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 1995, pp. 1–23.Google Scholar
  18. 18.
    de Moor, O. and Gibbons, J. Bridging the algorithm gap: A linear-time functional program for paragraph formatting. Technical Report CMS-TR-97-03, School of Computing and Mathematical Sciences, Oxford Brookes University, July 1997.Google Scholar
  19. 19.
    Field, J. and Teitelbaum, T. Incremental reduction in the lambda calculus. In Proceedings of the 1990 ACM Conference on LISP and Functional Programming, ACM, New York, 1990, pp. 307–322.Google Scholar
  20. 20.
    Friedman, D.P., Wise, D.S., and Wand, M. Recursive programming through table look-up. In Proceedings of the 1976 ACM Symposium on Symbolic and Algebraic Computation, ACM, New York, 1976, pp. 85–89.Google Scholar
  21. 21.
    Futamura, Y. and Nogi, K. Generalized partial evaluation. In Partial Evaluation and Mixed Computation, B. Bjørner, A.P. Ershov, and N.D. Jones (Eds.). North-Holland, Amsterdam, 1988, pp. 133–151.Google Scholar
  22. 22.
    Hu, T.C. and Shing, M.T. Computation of matrix chain products. Part i. SIAM J. Comput., 11(2) (1982) 362–373.Google Scholar
  23. 23.
    Hu, T.C. and Shing, M.T. Computation of matrix chain products. Part ii. SIAM J. Comput., 13(2) (1984) 228–251.Google Scholar
  24. 24.
    Hu, Z., Iwasaki, H., Takeichi, M., and Takano, A. Tupling calculation eliminates multiple data traversals. In ICFP 1997 [26], pp. 164–175.Google Scholar
  25. 25.
    Hughes, J. Lazy memo-functions. In Proceedings of the 2nd Conference on Functional Programming Languages and Computer Architecture, volume 201 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 1985, pp. 129–146.Google Scholar
  26. 26.
    Proceedings of the 1997 ACM SIGPLAN International Conference on Functional Programming, ACM, New York, 1997.Google Scholar
  27. 27.
    Keller, R.M. and Sleep, M.R. Applicative caching. ACM Trans. Program. Lang. Syst., 8(1) (1986) 88–108.Google Scholar
  28. 28.
    Khoshnevisan, H. Efficient memo-table management strategies. Acta Informatica, 28(1) (1990) 43–81.Google Scholar
  29. 29.
    Liu Y.A.CACHET:An interactive, incremental-attribution-based program transformation system for deriving incremental programs. In Proceedings of the 10th IEEE Knowledge-Based Software Engineering Conference, IEEE CS Press, Los Alamitos, CA, 1995, pp. 19–26.Google Scholar
  30. 30.
    Liu, Y.A. Principled strength reduction. In Algorithmic Languages and Calculi, R. Bird and L. Meertens (Eds.). Chapman & Hall, London, UK, 1997, pp. 357–381.Google Scholar
  31. 31.
    Liu, Y.A. Dependence analysis for recursive data. In Proceedings of the IEEE 1998 International Conference on Computer Languages, IEEE CS Press, Los Alamitos, CA, 1998, pp. 206–215.Google Scholar
  32. 32.
    Liu, Y.A. and Gómez, G. Automatic accurate cost-bound analysis for high-level languages. IEEE Transctions on Computers, 50(12) (2001) 1295–1309.Google Scholar
  33. 33.
    Liu, Y.A., Li, N., and Stoller, S.D. Solving regular tree grammar based constraints. In Proceedings of the 8th International Static Analysis Symposium, volume 2126 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 2001, pp. 213–233.Google Scholar
  34. 34.
    Liu, Y.A. and Stoller, S.D. Loop optimization for aggregate array computations. In Proceedings of the IEEE 1998 International Conference on Computer Languages, IEEE CS Press, Los Alamitos, CA, 1998, pp. 262–271.Google Scholar
  35. 35.
    Liu, Y.A. and Stoller, S.D. Eliminating dead code on recursive data. In Proceedings of the 6th International Static Analysis Symposium, volume 1694 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 1999, pp. 211–231.Google Scholar
  36. 36.
    Liu, Y.A. and Stoller, S.D. From recursion to iteration: What are the optimizations? In Proceedings of the ACM SIGPLAN 2000 Workshop on Partial Evaluation and Semantics-Based Program Manipulation, ACM, New York, 2000, pp. 73–82.Google Scholar
  37. 37.
    Liu, Y.A., Stoller, S.D., and Teitelbaum, T. Discovering auxiliary information for incremental computation. In Conference Record of the 23rd Annual ACM Symposium on Principles of Programming Languages, ACM, New York, 1996, pp. 157–170.Google Scholar
  38. 38.
    Liu, Y.A., Stoller, S.D., and Teitelbaum, T. Static caching for incremental computation. ACM Trans. Program. Lang. Syst., 20(3) (1998) 546–585.Google Scholar
  39. 39.
    Liu, Y.A. and Teitelbaum, T. Systematic derivation of incremental programs. Sci. Comput. Program., 24(1) (1995) 1–39.Google Scholar
  40. 40.
    Michie, D. “memo” functions and machine learning. Nature, 218 (1968) 19–22.Google Scholar
  41. 41.
    Mostow, D.J. and Cohen, D. Automating program speedup by deciding what to cache. In Proceedings of the 9th International Joint Conference on Artificial Intelligence, Morgan Kaufmann Publishers, San Francisco, CA, Aug. 1985, pp. 165–172.Google Scholar
  42. 42.
    Paige, R. Programming with invariants. IEEE Software, 3(1) (1986) 56–69.Google Scholar
  43. 43.
    Paige, R. Symbolic finite differencing—Part I. In Proceedings of the 3rd European Symposium on Programming, N.D. Jones (Ed.). Vol. 432 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, 1990, pp. 36–56.Google Scholar
  44. 44.
    Paige, R. and Koenig, S. Finite differencing of computable expressions. ACM Trans. Program. Lang. Syst., 4(3) (1982) 402–454.Google Scholar
  45. 45.
    Partsch, H.A. Specification and Transformation of Programs—A Formal Approach to Software Development. Springer-Verlag, Berlin, 1990.Google Scholar
  46. 46.
    Pettorossi, A. A powerful strategy for deriving efficient programs by transformation. In Conference Record of the 1984 ACM Symposium on LISP and Functional Programming. ACM, New York, 1984.Google Scholar
  47. 47.
    Pettorossi, A. and Proietti, M. Rules and strategies for transforming functional and logic programs. ACM Comput. Surv., 28(2) (1996) 360–414.Google Scholar
  48. 48.
    A. Pettorossi and M. Proietti. Program derivation via list introduction. In Algorithmic Languages and Calculi, R. Bird and L. Meertens (Eds.). Chapman & Hall, London, UK, 1997.Google Scholar
  49. 49.
    Pugh, W. An improved cache replacement strategy for function caching. In Proceedings of the 1988 ACM Conference on LISP and Functional Programming, ACM, New York, 1988, pp. 269–276.Google Scholar
  50. 50.
    Pugh, W. The Omega Test: A fast and practical integer programming algorithm for dependence analysis. Commun. ACM, 31(8) (1992) 102–114.Google Scholar
  51. 51.
    Pugh W. and Teitelbaum, T. Incremental computation via function caching. In Conference Record of the 16th Annual ACM Symposium on Principles of Programming Languages, ACM, New York, 1989, pp. 315–328.Google Scholar
  52. 52.
    Purdom, P.W. and Brown, C.A. The Analysis of Algorithms. Holt, Rinehart and Winston, 1985.Google Scholar
  53. 53.
    Reps, T. and Teitelbaum, T. The Synthesizer Generator: A System for Constructing Language-Based Editors. Springer-Verlag, New York, 1988.Google Scholar
  54. 54.
    Rosendahl, M. Automatic complexity analysis. In Proceedings of the 4th International Conference on Functional Programming Languages and Computer Architecture, ACM, New York, 1989, pp. 144–156.Google Scholar
  55. 55.
    Scherlis W.L. Program improvement by internal specialization. In Conference Record of the 8th Annual ACM Symposium on Principles of Programming Languages, ACM, New York, 1981, pp. 41–49.Google Scholar
  56. 56.
    Smith, D.R. KIDS: A semiautomatic program development system. IEEE Trans. Softw. Eng., 16(9) (1990) 1024–1043.Google Scholar
  57. 57.
    Smith, D.R. Structure and design of problem reduction generators. In Constructing Programs from Specifications, B. Möller (Ed.). North-Holland, Amsterdam, 1991, pp. 91–124.Google Scholar
  58. 58.
    Sniedovich, M. Dynamic Programming. Marcel Dekker, New York, 1992.Google Scholar
  59. 59.
    Unnikrishnan, L., Stoller, S.D., and Liu, Y.A. Automatic accurate stack space and heap space analysis for high-level languages. Technical Report TR 538, Computer Science Department, Indiana University, April 2000.Google Scholar
  60. 60.
    Unnikrishnan, L., Stoller, S.D., and Liu, Y.A. Automatic accurate live memory analysis for garbage-collected languages. In Proceedings of the ACM SIGPLAN 2001 Workshop on Languages, Compilers, and Tools for Embedded Systems, ACM, New York, 2001, pp. 102–111.Google Scholar
  61. 61.
    Wegbreit, B. Mechanical program analysis. Commun. ACM, 18(9) (1975) 528–538.Google Scholar
  62. 62.
    Wegbreit, B. Goal-directed program transformation. IEEE Trans. Softw. Eng., SE-2(2) (1976) 69–80.Google Scholar
  63. 63.
    Zhang, Y. and Liu, Y.A. Automating derivation of incremental programs. In Proceedings of the 1998 ACM SIGPLAN International Conference on Functional Programming, ACM, New York, 1998, p. 350.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Yanhong A. Liu
    • 1
  • Scott D. Stoller
    • 1
  1. 1.Computer Science DepartmentState University of New York at Stony BrookStony BrookUSA

Personalised recommendations