Advertisement

The TPTP Problem Library and Associated Infrastructure

The FOF and CNF Parts, v3.5.0
  • Geoff Sutcliffe
Article

Abstract

This paper describes the First-Order Form (FOF) and Clause Normal Form (CNF) parts of the TPTP problem library, and the associated infrastructure. TPTP v3.5.0 was the last release containing only FOF and CNF problems, and thus serves as the exemplar. This paper summarizes the history and development of the TPTP, describes the structure and contents of the TPTP, and gives an overview of TPTP related projects and tools.

Keywords

TPTP ATP system evaluation 

References

  1. 1.
    Andrews, P.B., Bishop, M., Issar, S., Nesmith, D., Pfenning, F., Xi, H.: TPS: a theorem-proving system for classical type theory. J. Autom. Reason. 16(3), 321–353 (1996)MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Barrett, C., de Moura, L., Stump, A.: SMT-COMP: satisfiability modulo theories competition. In: Etessami, K., Rajamani, S. (eds.) Proceedings of the 17th International Conference on Computer Aided Verification. Lecture Notes in Computer Science, no. 3576, pp. 20–23. Springer, New York (2005)Google Scholar
  3. 3.
    Benzmüller, C., Paulson, L., Theiss, F., Fietzke, A.: LEO-II—a cooperative automatic theorem prover for higher-order logic. In: Baumgartner, P., Armando, A., Gilles, D. (eds.) Proceedings of the 4th International Joint Conference on Automated Reasoning. Lecture Notes in Artificial Intelligence, no. 5195, pp. 162–170. Springer, New York (2008)Google Scholar
  4. 4.
    Benzmüller, C., Rabe, F., Sutcliffe, G.: THF0—the core TPTP language for classical higher-order logic. In: Baumgartner, P., Armando, A., Gilles, D. (eds.) Proceedings of the 4th International Joint Conference on Automated Reasoning. Lecture Notes in Artificial Intelligence, no. 5195, pp. 491–506. Springer, New York (2008)Google Scholar
  5. 5.
    Bonichon, R., Delahaye, D., Doligez, D.: Zenon: an extensible automated theorem prover producing checkable proofs. In: Dershowitz, N., Voronkov, A. (eds.) Proceedings of the 14th International Conference on Logic for Programming, Artificial Intelligence, and Reasoning. Lecture Notes in Artificial Intelligence, no. 4790, pp. 151–165 (2007)Google Scholar
  6. 6.
    Boyer, R., Lusk, E.L., McCune, W.W., Overbeek, R., Stickel, M.E., Wos, L.: Set theory in first-order logic: clauses for Godel’s axioms. J. Autom. Reason. 2(3), 287–327 (1986)MATHCrossRefGoogle Scholar
  7. 7.
    Comaromi, J.P., Beall, J., Matthews, W.E., New, G.R.: Dewey Decimal Classification and Relative Index, 20th edn. Forest, Albany (1989)Google Scholar
  8. 8.
    de Nivelle, H., Witkowski, P.: A small framework for proof checking. In: Schmidt, R., Konev, B., Schulz, S. (eds.) Proceedings of the Workshop on Practical Aspects of Automated Reasoning, 4th International Joint Conference on Automated Reasoning. CEUR Workshop Proceedings, no. 373, pp. 81–93 (2008)Google Scholar
  9. 9.
    Denney, E., Fischer, B., Schumann, J.: Using automated theorem provers to certify auto-generated aerospace software. In: Rusinowitch, M., Basin, D. (eds.) Proceedings of the 2nd International Joint Conference on Automated Reasoning. Lecture Notes in Artificial Intelligence, no. 3097, pp. 198–212 (2004)Google Scholar
  10. 10.
    Gebser, M., Liu, L., Namasivayam, G., Neumann, A., Schaub, T., Truszczynski, M.: The first answer set programming system competition. In: Baral, C., Bewka, G., Schlipf, J. (eds.) Proceedings of the 9th International Conference on Logic Programming and Nonmonotonic Reasoning. Lecture Notes in Artificial Intelligence, no. 4483, pp. 3–17. Springer, New York (2007)CrossRefGoogle Scholar
  11. 11.
    Gent, I., Walsh, T.: CSPLib: a benchmark library for constraints. In: Jaffar, J. (ed.) Proceedings of the 5th International Conference on the Principles and Practice of Constraint Programming. Lecture Notes in Computer Science, no. 1713, pp. 480–481. Springer, New York (1999)Google Scholar
  12. 12.
    Hoos, H., Stützle, T.: SATLIB: an online resource for research on SAT. In: Gent, I., van Maaren, H., Walsh, T. (eds.) Proceedings of the 3rd Workshop on the Satisfiability Problem (2001). http://www.satlib.org/
  13. 13.
    Kohlhase, M.: OMDoc—an open markup format for mathematical documents [version 1.2]. In: Lecture Notes in Artificial Intelligence, no. 4180. Springer, New York (2006)Google Scholar
  14. 14.
    Loechner, B., Hillenbrand, T.: A phytography of Waldmeister. J. AI Commun. 15(2/3), 127–133 (2002)MATHGoogle Scholar
  15. 15.
    Matuszek, C., Cabral, J., Witbrock, M., DeOliveira, J.: An introduction to the syntax and content of Cyc. In: Baralm C. (ed.) Proceedings of the 2006 AAAI Spring Symposium on Formalizing and Compiling Background Knowledge and Its Applications to Knowledge Representation and Question Answering, pp. 44–49 (2006)Google Scholar
  16. 16.
    McCharen, J.D., Overbeek, R.A., Wos, L.A.: Problems and experiments for and with automated theorem-proving programs. IEEE Trans. Comput. C-25(8), 773–782 (1976)CrossRefGoogle Scholar
  17. 17.
    McCune, W., Shumsky-Matlin, O.: Ivy: a preprocessor and proof checker for first-order logic. In: Kaufmann, M., Manolios, P., Strother Moore, J. (eds.) Computer-Aided Reasoning: ACL2 Case Studies. Advances in Formal Methods, no. 4, pp. 265–282. Kluwer Academic, Dordrecht (2000)Google Scholar
  18. 18.
    McCune, W.W.: Single axioms for groups and Abelian groups with various operations. J. Autom. Reason. 10(1), 1–13 (1993)MATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    McCune, W.W.: Otter 3.3 reference manual. Technical report ANL/MSC-TM-263, Argonne National Laboratory, Argonne (2003)Google Scholar
  20. 20.
    McCune, W.W., Wos, L.: Experiments in automated deduction with condensed detachment. In: Kapur, D. (ed.) Proceedings of the 11th International Conference on Automated Deduction. Lecture Notes in Artificial Intelligence, no. 607, pp. 209–223. Springer, New York (1992)Google Scholar
  21. 21.
    Meng, J., Quigley, C., Paulson, L.: Automation for interactive proof: first prototype. Inf. Comput. 204(10), 1575–1596 (2006)MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Newborn, M., Wang, Z.: Octopus: combining Learning and parallel search. J. Autom. Reason. 33(2), 171–218 (2004)MATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Nieuwenhuis, R.: The impact of CASC in the development of automated deduction systems. AI Commun. 15(2–3), 77–78 (2002)MathSciNetGoogle Scholar
  24. 24.
    Pease, A., Sutcliffe, G.: First order reasoning on a large ontology. In: Urban, J., Sutcliffe, G., Schulz, S. (eds.) Proceedings of the CADE-21 Workshop on Empirically Successful Automated Reasoning in Large Theories. CEUR Workshop Proceedings, no. 257, pp. 59–69 (2007)Google Scholar
  25. 25.
    Pelletier, F.J.: Seventy-five problems for testing automatic theorem provers. J. Autom. Reason. 2(2), 191–216 (1986)MATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    Pfenning, F., Schürmann, C.: System description: Twelf—a meta-logical framework for deductive systems. In: Ganzinger, H. (ed.) Proceedings of the 16th International Conference on Automated Deduction. Lecture Notes in Artificial Intelligence, no. 1632, pp. 202–206. Springer, New York (1999)CrossRefGoogle Scholar
  27. 27.
    Plaisted, D.A.: Non-Horn clause logic programming without contrapositives. J. Autom. Reason. 4(3), 287–325 (1988)MATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    Puzis, Y., Gao, Y., Sutcliffe, G.: Automated generation of interesting theorems. In: Sutcliffe, G., Goebel, R. (eds.) Proceedings of the 19th International FLAIRS Conference, pp. 49–54. AAAI, Menlo Park (2006)Google Scholar
  29. 29.
    Quaife, A.: Automated deduction in von Neumann-Bernays-Godel set theory. J. Autom. Reason. 8(1), 91–147 (1992)MATHCrossRefMathSciNetGoogle Scholar
  30. 30.
    Ramachandran, D., Reagan, P., Goolsbey, K.: First-orderized researchCyc: expressiveness and efficiency in a common sense knowledge base. In: Shvaiko, P. (ed.) Proceedings of the Workshop on Contexts and Ontologies: Theory, Practice and Applications (2005)Google Scholar
  31. 31.
    Ranise, S., Tinelli, C.: The SMT-LIB standard: version 1.2. Technical report, Department of Computer Science, The University of Iowa, Iowa City (2006)Google Scholar
  32. 32.
    Raths, T., Otten, J., Kreitz, C.: The ILTP problem library for intuitionistic logic—release v1.1. J. Autom. Reason. 38(1–2), 261–271 (2007)MATHCrossRefMathSciNetGoogle Scholar
  33. 33.
    Schulz, S.: Learning search control knowledge for equational theorem proving. In: Baader, F., Brewka, G., Eiter, T. (eds.) Proceedings of the Joint German/Austrian Conference on Artificial Intelligence. Lecture Notes in Artificial Intelligence, no. 2174, pp. 320–334. Springer, New York (2001)Google Scholar
  34. 34.
    Schulz, S.: E: a brainiac theorem prover. AI Commun. 15(2–3), 111–126 (2002)MATHGoogle Scholar
  35. 35.
    Schulz, S., Bonacina, M.-P.: On handling distinct objects in the superposition calculus. In: Konev, B., Schulz, S. (eds.) Proceedings of the 5th International Workshop on the Implementation of Logics, pp. 66–77 (2005)Google Scholar
  36. 36.
    Siekmann, J.H., Benzmüller, C., Brezhnev, V., Cheikhrouhou, L., Fiedler, A., Franke, A., Horacek, H., Kohlhase, M., Meier, A., Melis, E., Moschner, M., Normann, I., Pollet, M., Sorge, V., Ullrich, C., Wirth, C.P., Zimmer, J.: Proof development with OMEGA. In: Voronkov, A. (ed.) Proceedings of the 18th International Conference on Automated Deduction. Lecture Notes in Artificial Intelligence, no. 2392, pp. 143–148. Springer, New York (2002)Google Scholar
  37. 37.
    American Mathematical Society: Mathematical Subject Classification. American Mathematical Society, Providence (1992)Google Scholar
  38. 38.
    Stickel, M.E.: SNARK—SRI’s New Automated Reasoning Kit. http://www.ai.sri.com/~stickel/snark.html
  39. 39.
    Streeter, M., Golovin, D., Smith, S.F.: Combining multiple heuristics online. In: Holte, R.C., Howe, A. (eds.) Proceedings of the 22nd Conference on Artificial Intelligence, pp. 1197–1203. AAAI, Menlo Park (2007)Google Scholar
  40. 40.
    Suchanek, F., Kasneci, G., Weikum, G.: YAGO: a core of semantic knowledge. In: Patel-Schneider, P., Shenoy, P. (eds.) Proceedings of the 16th International World Wide Web Conference, pp. 697–706 (2007)Google Scholar
  41. 41.
    Suda, M., Sutcliffe, G., Wischnewsk, P., Lamotte-Schubert, M.: External sources of axioms in automated theorem proving. In: Mertsching, B. (ed.) Proceedings of the 32nd Annual Conference on Artificial Intelligence (2009)Google Scholar
  42. 42.
    Sutcliffe, G.: The TSTP Solution Library. http://www.TPTP.org/TSTP
  43. 43.
    Sutcliffe, G.: SystemOnTPTP. In: McAllester, D. (ed.) Proceedings of the 17th International Conference on Automated Deduction. Lecture Notes in Artificial Intelligence, no. 1831, pp. 406–410. Springer, New York (2000)CrossRefGoogle Scholar
  44. 44.
    Sutcliffe, G.: Semantic derivation verification. Int. J. Artif. Intell. Tools 15(6), 1053–1070 (2006)CrossRefGoogle Scholar
  45. 45.
    Sutcliffe, G.: TPTP, TSTP, CASC, etc. In: Diekert, V., Volkov, M., Voronkov, A. (eds.) Proceedings of the 2nd International Computer Science Symposium in Russia. Lecture Notes in Computer Science, no. 4649, pp. 7–23. Springer, New York (2007)Google Scholar
  46. 46.
    Sutcliffe, G.: The CADE-21 automated theorem proving system competition. AI Commun. 21(1), 71–82 (2008)MATHMathSciNetGoogle Scholar
  47. 47.
    Sutcliffe, G.: The SZS ontologies for automated reasoning software. In: Sutcliffe, G., Rudnicki, P., Schmidt, R., Konev, B., Schulz, S. (eds.) Proceedings of the LPAR Workshops: Knowledge Exchange: Automated Provers and Proof Assistants, and the 7th International Workshop on the Implementation of Logics. CEUR Workshop Proceedings, no. 418, pp. 38–49 (2008)Google Scholar
  48. 48.
    Sutcliffe, G.: The TPTP problem library and associated infrastructure. The FOF and CNF Parts, v3.5.0. J. Autom. Reason. (2009). doi: 10.1007/s10817-009-9143-8 Google Scholar
  49. 49.
    Sutcliffe, G., Fuchs, M., Suttner, C.: Progress in automated theorem proving, 1997–1999. In: Hoos, H., Stützle, T. (eds.) Proceedings of the IJCAI’01 Workshop on Empirical Methods in Artificial Intelligence, pp. 53–60 (2001)Google Scholar
  50. 50.
    Sutcliffe, G., Puzis, Y.: SRASS—a semantic relevance axiom selection system. In: Pfenning, F. (ed.) Proceedings of the 21st International Conference on Automated Deduction. Lecture Notes in Artificial Intelligence, no. 4603, pp. 295–310. Springer, New York (2007)CrossRefGoogle Scholar
  51. 51.
    Sutcliffe, G., Schulz, S., Claessen, K., Van Gelder, A.: Using the TPTP language for writing derivations and finite interpretations. In: Furbach, U., Shankar, N. (eds.) Proceedings of the 3rd International Joint Conference on Automated Reasoning. Lecture Notes in Artificial Intelligence, no. 4130, pp. 67–81 (2006)Google Scholar
  52. 52.
    Sutcliffe, G., Seyfang, D.: Smart selective competition parallelism ATP. In: Kumar, A., Russell, I. (eds.) Proceedings of the 12th International FLAIRS Conference, pp. 341–345. AAAI, Menlo Park (1999)Google Scholar
  53. 53.
    Sutcliffe, G., Suttner, C.: The state of CASC. AI Commun. 19(1), 35–48 (2006)MATHMathSciNetGoogle Scholar
  54. 54.
    Sutcliffe, G., Suttner, C., Pelletier, F.J.: The IJCAR ATP system competition. J. Autom. Reason. 28(3), 307–320 (2002)MATHCrossRefGoogle Scholar
  55. 55.
    Sutcliffe, G., Suttner, C.B.: The TPTP problem library: CNF release v1.2.1. J. Autom. Reason. 21(2), 177–203 (1998)MATHCrossRefMathSciNetGoogle Scholar
  56. 56.
    Sutcliffe, G., Suttner, C.B.: Evaluating general purpose automated theorem proving systems. Artif. Intell. 131(1–2), 39–54 (2001)MATHCrossRefMathSciNetGoogle Scholar
  57. 57.
    Sutcliffe, G., Zimmer, J., Schulz, S.: Communication formalisms for automated theorem proving tools. In: Sorge, V., Colton, S., Fisher, M., Gow, J. (eds.) Proceedings of the Workshop on Agents and Automated Reasoning, 18th International Joint Conference on Artificial Intelligence, pp. 52–57 (2003)Google Scholar
  58. 58.
    Tammet, T.: Gandalf. J. Autom. Reason. 18(2), 199–204 (1997)CrossRefGoogle Scholar
  59. 59.
    Trac, S., Puzis, Y., Sutcliffe, G.: An interactive derivation viewer. In: Autexier, S., Benzmüller, C. (eds.) Proceedings of the 7th Workshop on User Interfaces for Theorem Provers, 3rd International Joint Conference on Automated Reasoning. Electronic Notes in Theoretical Computer Science, vol. 174, pp. 109–123 (2006)Google Scholar
  60. 60.
    Trac, S., Sutcliffe, G., Pease, A.: Integration of the TPTPWorld into SigmaKEE. In: Schmidt, R., Konev, B., Schulz, S. (eds.) Proceedings of the Workshop on Practical Aspects of Automated Reasoning, 4th International Joint Conference on Automated Reasoning. CEUR Workshop Proceedings, no. 373, pp. 103–114 (2008)Google Scholar
  61. 61.
    Urban, J.: MPTP 0.2: design, implementation, and initial experiments. J. Autom. Reason. 37(1–2), 21–43 (2006)MATHGoogle Scholar
  62. 62.
    Urban, J., Sutcliffe, G., Pudlak, P., Vyskocil, J.: MaLARea SG1: machine learner for automated reasoning with semantic guidance. In: Baumgartner, P., Armando, A., Gilles, D. (eds.) Proceedings of the 4th International Joint Conference on Automated Reasoning. Lecture Notes in Artificial Intelligence, no. 5195, pp. 441–456. Springer, New York (2008)Google Scholar
  63. 63.
    Van Gelder, A., Sutcliffe, G.: Extending the TPTP language to higher-order logic with automated parser generation. In: Furbach, U., Shankar, N. (eds.) Proceedings of the 3rd International Joint Conference on Automated Reasoning. Lecture Notes in Artificial Intelligence, no. 4130, pp. 156–161. Springer, New York (2006)Google Scholar
  64. 64.
    Wolf, A., Letz, R.: Strategy parallelism in automated theorem proving. Int. J. Pattern Recogn. Artif. Intell. 13(2), 219–245 (1999)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of MiamiCoral GablesUSA

Personalised recommendations