A formal framework for evaluating heuristic programs

  • Lenore Cowen
  • Joan Feigenbaum
  • Sampath Kannan
Session 16: Algorithms II
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1099)


We address the question of how one evaluates the usefulness of a heuristic program on a particular input. If theoretical tools do not allow us to decide for every instance whether a particular heuristic is fast enough, might we at least write a simple, fast companion program that makes this decision on some inputs of interest? We call such a companion program a timer for the heuristic. Timers are related to program checkers, as defined by Blum [3], in the following sense: Checkers are companion programs that check the correctness of the output produced by (unproven but bounded-time) programs on particular instances; timers, on the other hand, are companion programs that attempt to bound the running time on particular instances of correct programs whose running times have not been fully analyzed. This paper provides a family of definitions that formalize the notion of a timer and some preliminary results that demonstrate the utility of these definitions.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    L. Adleman and M. Huang, Recognizing primes in random polynomial time, in Proc. 19th Symposium on Theory of Computing, ACM, New York, 1987, pp. 462–469.Google Scholar
  2. 2.
    L. Babai, L. Fortnow, and C. Lund, Nondeterministic exponential time has two-prover interactive protocols, Computational Complexity, 1 (1991) 3–40.CrossRefGoogle Scholar
  3. 3.
    M. Blum, Program result checking: a new approach to making programs more reliable, in Proc. 20th International Colloquium on Automata, Languages, and Programming, Lecture Notes in Computer Science, vol. 700, Springer, Berlin, 1993, pp. 2–14. First appeared in preliminary form in report 88-009, International Computer Science Institute Technical, Berkeley CA, 1988.Google Scholar
  4. 4.
    L. Goldberg, Efficient Algorithms for Listing Combinatorial Structures, Cambridge University Press, Cambridge UK, 1993.Google Scholar
  5. 5.
    S. Goldwasser and S. Micali, Probabilistic encryption, J. Comput. and Sys. Scis., 28 (1984) 270–299.CrossRefGoogle Scholar
  6. 6.
    J. Grollman and A. Selman, Complexity measures for public-key cryptosystems, SIAM J. Comput., 17 (1988) 309–335.CrossRefGoogle Scholar
  7. 7.
    J. Gustedt and A. Steger, Testing hereditary properties efficiently on average, in Proc. of ORDAL '94.Google Scholar
  8. 8.
    J. Hastad, Pseudo-random generators under uniform assumptions, in Proc. 22nd Symposium on the Theory of Computing, ACM, New York, 1990, pp. 395–404.Google Scholar
  9. 9.
    R. Impagliazzo, L. Levin, and M. Luby, Pseudo-random generation from one-way functions, in Proc. 21st Symposium on the Theory of Computing, ACM, New York, 1989, pp. 12–24.Google Scholar
  10. 10.
    M. Jerrum and A. Sinclair, Approximating the permanent, SIAM J. Comput., 18 (1989) 1149–1178.CrossRefGoogle Scholar
  11. 11.
    D. Johnson, C. Papadimitriou, and M. Yannakakis, On generating all maximal independent sets, Inf. Proc. Letters, 27 (1988) 119–123.CrossRefGoogle Scholar
  12. 12.
    G. Kirchoff, Uber die Auflosung der Gleichungen, auf welche man bei der Untersuchung der linearen Verteilung galvanische Strome gefuhrt wird, Ann. Phys. Chem., 72 (1847) 497–508.Google Scholar
  13. 13.
    D. Knuth, Sorting and Searching, The Art of Computer Programming, vol. 3, Addison-Wesley, Reading, 1973.Google Scholar
  14. 14.
    K. Ko, T. Long, and D. Du, On one-way functions and polynomial-time isomorphisms, Theor. Comp. Sci., 47 (1986) 263–276.CrossRefGoogle Scholar
  15. 15.
    C. Lund, L. Fortnow, H. Karloff, and N. Nisan, Algebraic methods for interactive proof systems, J. ACM, 39 (1992) 859–868.CrossRefGoogle Scholar
  16. 16.
    J.R. Munkres, Topology: A First Course, Prentice-Hall, Englewood Cliffs, 1975.Google Scholar
  17. 17.
    R. Read and R. Tarjan, Bounds on backtrack algorithms for listing cycles, paths, and spanning trees, Networks, 5 (1975) 237–252.Google Scholar
  18. 18.
    A. Shamir, IP = PSPACE, J. ACM, 39 (1992) 869–877.CrossRefGoogle Scholar
  19. 19.
    L. Valiant, The complexity of computing the permanent, Theor. Comp. Sci., 8 (1979) 189–201.CrossRefGoogle Scholar
  20. 20.
    L. Valiant, Negative results on counting, in Proc. 4th Theoretical Computer Science GI Conference, Lecture Notes in Computer Science, vol. 67, 1979, Springer, Berlin, pp. 40–45.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1996

Authors and Affiliations

  • Lenore Cowen
    • 1
  • Joan Feigenbaum
    • 2
  • Sampath Kannan
    • 3
  1. 1.Dept. of Math. Sciences and Dept. of CSJohns Hopkins UniversityBaltimore
  2. 2.AT&T ResearchMurray Hill
  3. 3.Dept. of Computer and Information ScienceUniversity of PennsylvaniaPhiladelphia

Personalised recommendations