Skip to main content
SpringerLink
Log in

A formal framework for evaluating heuristic programs

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

We address the question of how one evaluates the usefulness of a heuristic program on a particular input. If theoretical tools do not allow us to decide for every instance whether a particular heuristic is fast enough, might we at least write a simple, fast companion program that makes this decision on some inputs of interest? We call such a companion program a timer for the heuristic. Timers are related to program checkers, as defined by Blum (1993), in the following sense: Checkers are companion programs that check the correctness of the output produced by (unproven but bounded‐time) programs on particular instances; timers, on the other hand, are companion programs that attempt to bound the running time on particular instances of correct programs whose running times have not been fully analyzed. This paper provides a family of definitions that formalize the notion of a timer and some preliminary results that demonstrate the utility of these definitions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. L. Adleman and M. Huang, Recognizing primes in random polynomial time, in: Proc. 19th Symposium on Theory of Computing(ACM, New York, 1987) pp. 462-469.

    Google Scholar 

  2. L. Babai, L. Fortnow and C. Lund, Nondeterministic exponential time has two-prover interactive protocols, Computational Complexity 1 (1991) 3-40.

    Article  MATH  MathSciNet  Google Scholar 

  3. M. Blum, Program result checking: a new approach to making programs more reliable, in: Proc. 20th International Colloquium on Automata, Languages, and Programming, Lecture Notes in Computer Science 700 (Springer, Berlin, 1993) pp. 2-14.

    Google Scholar 

  4. L. Goldberg, Efficient Algorithms for Listing Combinatorial Structures(Cambridge University Press, Cambridge, UK, 1993).

    Google Scholar 

  5. S. Goldwasser and S. Micali, Probabilistic encryption, Journal of Computer and System Sciences 28 (1984) 270-299.

    Article  MATH  MathSciNet  Google Scholar 

  6. J. Grollman and A. Selman, Complexity measures for public-key cryptosystems, SIAM Journal on Computing 17 (1988) 309-335.

    Article  MathSciNet  Google Scholar 

  7. J. Gustedt and A. Steger, Testing hereditary properties efficiently on average, in: Proc. of Orders, Algorithms and Applications, Lecture Notes in Computer Science 831 (Springer, Berlin, 1994) pp. 100-116.

    Google Scholar 

  8. J. Håstad, Pseudo-random generators under uniform assumptions, in: Proc. 22nd Symposium on the Theory of Computing(ACM, New York, 1990) pp. 395-404.

    Google Scholar 

  9. R. Impagliazzo, L. Levin and M. Luby, Pseudo-random generation from one-way functions, in: Proc. 21st Symposium on the Theory of Computing(ACM, New York, 1989) pp. 12-24.

    Google Scholar 

  10. M. Jerrum and A. Sinclair, Approximating the permanent, SIAM Journal on Computing 18 (1989) 1149-1178.

    Article  MATH  MathSciNet  Google Scholar 

  11. D. Johnson, C. Papadimitriou and M. Yannakakis, On generating all maximal independent sets, Information Processing Letters 27 (1988) 119-123.

    Article  MATH  MathSciNet  Google Scholar 

  12. G. Kirchoff, Uber die Auflosung der Gleichungen, auf welche man bei der Untersuchung der linearen Verteilung galvanische Strome gefuhrt wird, Ann. Phys. Chem. 72 (1847) 497-508.

    Google Scholar 

  13. D. Knuth, Sorting and Searching, The Art of Computer Programming, Vol. 3 (Addison-Wesley, Reading, MA, 1973).

    Google Scholar 

  14. K. Ko, T. Long and D. Du, On one-way functions and polynomial-time isomorphisms, Theoretical Computer Science 47 (1986) 263-276.

    Article  MATH  MathSciNet  Google Scholar 

  15. C. Lund, L. Fortnow, H. Karloff and N. Nisan, Algebraic methods for interactive proof systems, Journal of the ACM 39 (1992) 859-868.

    Article  MATH  MathSciNet  Google Scholar 

  16. M. Naor and M. Yung, Universal one-way hash functions and their cryptographic applications, in: Proc. 21st Symposium on the Theory of Computing(ACM, New York, 1989) pp. 33-43.

    Google Scholar 

  17. J.R. Munkres, Topology: A First Course(Prentice-Hall, Englewood Cliffs, NJ, 1975).

    Google Scholar 

  18. R. Read and R. Tarjan, Bounds on backtrack algorithms for listing cycles, paths, and spanning trees, Networks 5 (1975) 237-252.

    MATH  MathSciNet  Google Scholar 

  19. J. Rompel, One-way functions are necessary and sufficient for secure signatures, in: Proc. 22nd Symposium on the Theory of Computing(ACM, New York, 1990) pp. 387-394.

    Google Scholar 

  20. A. Shamir, IP = PSPACE, Journal of the ACM 39 (1992) 869-877.

    Article  MATH  MathSciNet  Google Scholar 

  21. L. Valiant, The complexity of computing the permanent, Theoretical Computer Science 8 (1979) 189-201.

    Article  MATH  MathSciNet  Google Scholar 

  22. L. Valiant, Negative results on counting, in: Proc. 4th Theoretical Computer Science GI Conference, Lecture Notes in Computer Science 67 (Springer, Berlin, 1979) pp. 40-45.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences and Department of Computer Science, Johns Hopkins University, Baltimore, MD, 21218, USA

    Lenore Cowen

  2. AT&T Labs – Research, 180 Park Avenue, Florham Park, NJ, 07932, USA

    Joan Feigenbaum

  3. Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA, 19104, USA

    Sampath Kannan

Authors
  1. Lenore Cowen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joan Feigenbaum
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sampath Kannan
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cowen, L., Feigenbaum, J. & Kannan, S. A formal framework for evaluating heuristic programs. Annals of Mathematics and Artificial Intelligence 22, 193–206 (1998). https://doi.org/10.1023/A:1018950418415

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1018950418415

Keywords

Search

Navigation