A formal framework for evaluating heuristic programs
We address the question of how one evaluates the usefulness of a heuristic program on a particular input. If theoretical tools do not allow us to decide for every instance whether a particular heuristic is fast enough, might we at least write a simple, fast companion program that makes this decision on some inputs of interest? We call such a companion program a timer for the heuristic. Timers are related to program checkers, as defined by Blum , in the following sense: Checkers are companion programs that check the correctness of the output produced by (unproven but bounded-time) programs on particular instances; timers, on the other hand, are companion programs that attempt to bound the running time on particular instances of correct programs whose running times have not been fully analyzed. This paper provides a family of definitions that formalize the notion of a timer and some preliminary results that demonstrate the utility of these definitions.
Unable to display preview. Download preview PDF.
- 1.L. Adleman and M. Huang, Recognizing primes in random polynomial time, in Proc. 19th Symposium on Theory of Computing, ACM, New York, 1987, pp. 462–469.Google Scholar
- 3.M. Blum, Program result checking: a new approach to making programs more reliable, in Proc. 20th International Colloquium on Automata, Languages, and Programming, Lecture Notes in Computer Science, vol. 700, Springer, Berlin, 1993, pp. 2–14. First appeared in preliminary form in report 88-009, International Computer Science Institute Technical, Berkeley CA, 1988.Google Scholar
- 4.L. Goldberg, Efficient Algorithms for Listing Combinatorial Structures, Cambridge University Press, Cambridge UK, 1993.Google Scholar
- 7.J. Gustedt and A. Steger, Testing hereditary properties efficiently on average, in Proc. of ORDAL '94.Google Scholar
- 8.J. Hastad, Pseudo-random generators under uniform assumptions, in Proc. 22nd Symposium on the Theory of Computing, ACM, New York, 1990, pp. 395–404.Google Scholar
- 9.R. Impagliazzo, L. Levin, and M. Luby, Pseudo-random generation from one-way functions, in Proc. 21st Symposium on the Theory of Computing, ACM, New York, 1989, pp. 12–24.Google Scholar
- 12.G. Kirchoff, Uber die Auflosung der Gleichungen, auf welche man bei der Untersuchung der linearen Verteilung galvanische Strome gefuhrt wird, Ann. Phys. Chem., 72 (1847) 497–508.Google Scholar
- 13.D. Knuth, Sorting and Searching, The Art of Computer Programming, vol. 3, Addison-Wesley, Reading, 1973.Google Scholar
- 16.J.R. Munkres, Topology: A First Course, Prentice-Hall, Englewood Cliffs, 1975.Google Scholar
- 17.R. Read and R. Tarjan, Bounds on backtrack algorithms for listing cycles, paths, and spanning trees, Networks, 5 (1975) 237–252.Google Scholar
- 20.L. Valiant, Negative results on counting, in Proc. 4th Theoretical Computer Science GI Conference, Lecture Notes in Computer Science, vol. 67, 1979, Springer, Berlin, pp. 40–45.Google Scholar