Skip to main content
SpringerLink
Log in

The Role of Entropy in Guiding a Connection Prover

  • Conference paper
  • First Online:
Automated Reasoning with Analytic Tableaux and Related Methods (TABLEAUX 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12842))

Abstract

In this work we study how to learn good algorithms for selecting reasoning steps in theorem proving. We explore this in the connection tableau calculus implemented by leanCoP where the partial tableau provides a clean and compact notion of a state to which a limited number of inferences can be applied. We start by incorporating a state-of-the-art learning algorithm — a graph neural network (GNN) – into the plCoP theorem prover. Then we use it to observe the system’s behavior in a reinforcement learning setting, i.e., when learning inference guidance from successful Monte-Carlo tree searches on many problems. Despite its better pattern matching capability, the GNN initially performs worse than a simpler previously used learning algorithm. We observe that the simpler algorithm is less confident, i.e., its recommendations have higher entropy. This leads us to explore how the entropy of the inference selection implemented via the neural network influences the proof search. This is related to research in human decision-making under uncertainty, and in particular the probability matching theory. Our main result shows that a proper entropy regularization, i.e., training the GNN not to be overconfident, greatly improves plCoP ’s performance on a large mathematical corpus.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Assuming Disjunctive Normal Form.

  2. 2.

    In this sense, theorem proving can be considered as a meta learning task.

  3. 3.

    A discount factor of 0.99 is applied to positive rewards to favor shorter proofs.

  4. 4.

    This is motivated by the experiments with the ENIGMA-GNN system [18], where 8–10 layers produce better results than 5 layers.

  5. 5.

    The new extensions described here and the experimental configuration files are publicly available at plCoP ’s repository: https://github.com/zsoltzombori/plcop.

  6. 6.

    X and G stand for the probability distributions predicted by XGBoost and GNN, respectively.

References

  1. Alama, J., Heskes, T., Kühlwein, D., Tsivtsivadze, E., Urban, J.: Premise selection for mathematics by corpus analysis and kernel methods. J. Autom. Reason. 52(2), 191–213 (2014)

    Article  MathSciNet  Google Scholar 

  2. Alemi, A.A., Chollet, F., Een, N., Irving, G., Szegedy, C., Urban, J.: DeepMath - deep sequence models for premise selection. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS 2016, USA, pp. 2243–2251. Curran Associates Inc. (2016)

    Google Scholar 

  3. Anthony, T., Tian, Z., Barber, D.: Thinking fast and slow with deep learning and tree search. CoRR, abs/1705.08439 (2017)

    Google Scholar 

  4. Bansal, K., Loos, S.M., Rabe, M.N., Szegedy, C., Wilcox, S.: HOList: an environment for machine learning of higher order logic theorem proving. In: Chaudhuri, K., Salakhutdinov, R. (eds.) International Conference on Machine Learning, ICML 2019. Proceedings of Machine Learning Research, vol. 97, pp. 454–463. PMLR (2019)

    Google Scholar 

  5. Bibel, W.: A vision for automated deduction rooted in the connection method. In: Schmidt, R.A., Nalon, C. (eds.) TABLEAUX 2017. LNCS (LNAI), vol. 10501, pp. 3–21. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66902-1_1

    Chapter  MATH  Google Scholar 

  6. Blaauwbroek, L., Urban, J., Geuvers, H.: Tactic learning and proving for the Coq proof assistant. In: Albert, E., Kovács, L. (eds.) LPAR 2020: 23rd International Conference on Logic for Programming, Artificial Intelligence and Reasoning. EPiC Series in Computing, vol. 73, pp. 138–150. EasyChair (2020)

    Google Scholar 

  7. Browne, C., et al.: A survey of Monte Carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 4, 1–43 (2012)

    Article  Google Scholar 

  8. Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016, New York, NY, USA, pp. 785–794. ACM (2016)

    Google Scholar 

  9. Chvalovský, K., Jakubův, J., Suda, M., Urban, J.: ENIGMA-NG: efficient neural and gradient-boosted inference guidance for E. In: Fontaine, P. (ed.) CADE 2019. LNCS (LNAI), vol. 11716, pp. 197–215. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29436-6_12

    Chapter  Google Scholar 

  10. The Coq Proof Assistant. http://coq.inria.fr

  11. Crouse, M., et al.: A deep reinforcement learning approach to first-order logic theorem proving. Artificial Intelligence (2019). arXiv

    Google Scholar 

  12. Färber, M., Kaliszyk, C., Urban, J.: Machine learning guidance for connection tableaux. J. Autom. Reason. 65(2), 287–320 (2021)

    Article  MathSciNet  Google Scholar 

  13. Gauthier, T., Kaliszyk, C., Urban, J.: TacticToe: learning to reason with HOL4 tactics. In: Eiter, T., Sands, D. (eds.) LPAR-21. 21st International Conference on Logic for Programming, Artificial Intelligence and Reasoning. EPiC Series in Computing, vol. 46, pp. 125–143. EasyChair (2017)

    Google Scholar 

  14. Gauthier, T., Kaliszyk, C., Urban, J., Kumar, R., Norrish, M.: TacticToe: learning to prove with tactics. J. Autom. Reason. 65(2), 257–286 (2021)

    Article  MathSciNet  Google Scholar 

  15. Gordon, M.J.C., Melham, T.F. (eds.): Introduction to HOL: A Theorem Proving Environment for Higher Order Logic. Cambridge University Press, Cambridge (1993)

    MATH  Google Scholar 

  16. Harrison, J.: HOL light: a tutorial introduction. In: Srivas, M., Camilleri, A. (eds.) FMCAD 1996. LNCS, vol. 1166, pp. 265–269. Springer, Heidelberg (1996). https://doi.org/10.1007/BFb0031814

    Chapter  Google Scholar 

  17. Huang, D., Dhariwal, P., Song, D., Sutskever, I.: Gamepad: a learning environment for theorem proving. In: 7th International Conference on Learning Representations, ICLR 2019. OpenReview.net (2019)

    Google Scholar 

  18. Jakubův, J., Chvalovský, K., Olšák, M., Piotrowski, B., Suda, M., Urban, J.: ENIGMA anonymous: symbol-independent inference guiding machine (system description). In: Peltier, N., Sofronie-Stokkermans, V. (eds.) IJCAR 2020. LNCS (LNAI), vol. 12167, pp. 448–463. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-51054-1_29

    Chapter  Google Scholar 

  19. Jakubův, J., Urban, J.: ENIGMA: efficient learning-based inference guiding machine. In: Geuvers, H., England, M., Hasan, O., Rabe, F., Teschke, O. (eds.) CICM 2017. LNCS (LNAI), vol. 10383, pp. 292–302. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-62075-6_20

    Chapter  Google Scholar 

  20. Jakubuv, J., Urban, J.: Hammering Mizar by learning clause guidance. In: Harrison, J., O’Leary, J., Tolmach, A. (eds.) 10th International Conference on Interactive Theorem Proving, ITP 2019, Portland, OR, USA, 9–12 September 2019. LIPIcs, vol. 141, pp. 34:1–34:8. Schloss Dagstuhl - Leibniz-Zentrum für Informatik (2019)

    Google Scholar 

  21. Kaliszyk, C., Urban, J.: M2K dataset. https://github.com/JUrban/deepmath/blob/master/M2k_list

  22. Kaliszyk, C., Urban, J.: Mizar40 dataset. https://github.com/JUrban/deepmath

  23. Kaliszyk, C., Urban, J.: FEMaLeCoP: fairly efficient machine learning connection prover. In: Davis, M., Fehnker, A., McIver, A., Voronkov, A. (eds.) LPAR 2015. LNCS, vol. 9450, pp. 88–96. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-48899-7_7

    Chapter  Google Scholar 

  24. Kaliszyk, C., Urban, J.: MizAR 40 for Mizar 40. J. Autom. Reason. 55(3), 245–256 (2015). https://doi.org/10.1007/s10817-015-9330-8

    Article  MathSciNet  MATH  Google Scholar 

  25. Kaliszyk, C., Urban, J., Michalewski, H., Olsák, M.: Reinforcement learning of theorem proving. In: NeurIPS, pp. 8836–8847 (2018)

    Google Scholar 

  26. Kaliszyk, C., Urban, J., Vyskočil, J.: Efficient semantic features for automated reasoning over large theories. In: Yang, Q., Wooldridge, M. (eds.) IJCAI 2015, pp. 3084–3090. AAAI Press (2015)

    Google Scholar 

  27. Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006). https://doi.org/10.1007/11871842_29

    Chapter  Google Scholar 

  28. Kovács, L., Voronkov, A.: First-order theorem proving and Vampire. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 1–35. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39799-8_1

    Chapter  Google Scholar 

  29. Loos, S.M., Irving, G., Szegedy, C., Kaliszyk, C.: Deep network guided proof search. In: 21st International Conference on Logic for Programming, Artificial Intelligence, and Reasoning (LPAR) (2017)

    Google Scholar 

  30. Mohamed, O.A., Muñoz, C., Tahar, S. (eds.): TPHOLs 2008. LNCS, vol. 5170. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-71067-7

    Book  Google Scholar 

  31. Nagashima, Y., He, Y.: PaMpeR: proof method recommendation system for Isabelle/HOL. In: Huchard, M., Kästner, C., Fraser, G. (eds.) Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE 2018, Montpellier, France, 3–7 September 2018, pp. 362–372. ACM (2018)

    Google Scholar 

  32. Olsák, M., Kaliszyk, C., Urban, J.: Property invariant embedding for automated reasoning. In: Giacomo, G.D., et al. (eds.) ECAI 2020–24th European Conference on Artificial Intelligence, Santiago de Compostela, Spain, 29 August-8 September 2020 - Including 10th Conference on Prestigious Applications of Artificial Intelligence (PAIS 2020). Frontiers in Artificial Intelligence and Applications, vol. 325, pp. 1395–1402. IOS Press (2020)

    Google Scholar 

  33. Otten, J.: leanCoP 2.0 and ileanCoP 1.2: high performance lean theorem proving in classical and intuitionistic logic (system descriptions). In: Armando, A., Baumgartner, P., Dowek, G. (eds.) IJCAR 2008. LNCS (LNAI), vol. 5195, pp. 283–291. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-71070-7_23

    Chapter  Google Scholar 

  34. Otten, J., Bibel, W.: leanCoP: lean connection-based theorem proving. J. Symb. Comput. 36, 139–161 (2003)

    Article  MathSciNet  Google Scholar 

  35. Rawson, M., Reger, G.: Automated theorem proving, fast and slow. EasyChair Preprint no. 4433. EasyChair (2020)

    Google Scholar 

  36. Ross, S., Gordon, G., Bagnell, D.: A reduction of imitation learning and structured prediction to no-regret online learning. In: Gordon, G., Dunson, D., Dudík, M. (eds.) Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research, Fort Lauderdale, FL, USA, 11–13 April 2011, vol. 15, pp. 627–635. PMLR (2011)

    Google Scholar 

  37. Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. Trans. Neur. Netw. 20(1), 61–80 (2009)

    Article  Google Scholar 

  38. Schulz, S.: E - a Brainiac theorem prover. AI Commun. 15(2–3), 111–126 (2002)

    MATH  Google Scholar 

  39. Schulz, S.: System description: E 1.8. In: McMillan, K., Middeldorp, A., Voronkov, A. (eds.) LPAR 2013. LNCS, vol. 8312, pp. 735–743. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-45221-5_49

    Chapter  Google Scholar 

  40. Silver, D., et al.: Mastering the game of go without human knowledge. Nature 550, 354 (2017)

    Article  Google Scholar 

  41. Slind, K., Norrish, M.: A brief overview of HOL4. In: Mohamed et al. [30], pp. 28–32

    Google Scholar 

  42. Suda, M.: New techniques that improve Enigma-style clause selection guidance. In: International Conference on Automated Deduction, CADE 2021 (2021)

    Google Scholar 

  43. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. The MIT Press, Cambridge (2018)

    MATH  Google Scholar 

  44. Urban, J.: ERC project AI4Reason final scientific report (2021). http://grid01.ciirc.cvut.cz/~mptp/ai4reason/PR_CORE_SCIENTIFIC_4.pdf

  45. Urban, J., Sutcliffe, G., Pudlák, P., Vyskočil, J.: MaLARea SG1 - machine learner for automated reasoning with semantic guidance. In: IJCAR, pp. 441–456 (2008)

    Google Scholar 

  46. Urban, J., Vyskočil, J., Štěpánek, P.: MaLeCoP machine learning connection prover. In: Brünnler, K., Metcalfe, G. (eds.) TABLEAUX 2011. LNCS (LNAI), vol. 6793, pp. 263–277. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22119-4_21

    Chapter  Google Scholar 

  47. Vulkan, N.: An economist’s perspective on probability matching. J. Econ. Surv. 14(1), 101–118 (2000)

    Article  Google Scholar 

  48. Wenzel, M., Paulson, L.C., Nipkow, T.: The Isabelle framework. In: Mohamed et al. [30], pp. 33–38

    Google Scholar 

  49. Williams, R.J., Peng, J.: Function optimization using connectionist reinforcement learning algorithms. Connect. Sci. 3(3), 241–268 (1991)

    Article  Google Scholar 

  50. Yang, K., Deng, J.: Learning to prove theorems via interacting with proof assistants. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning, ICML 2019, Long Beach, California, USA, 9–15 June 2019. Proceedings of Machine Learning Research, vol. 97, pp. 6984–6994. PMLR (2019)

    Google Scholar 

  51. Zombori, Z., Urban, J., Brown, C.E.: Prolog technology reinforcement learning prover. In: Peltier, N., Sofronie-Stokkermans, V. (eds.) IJCAR 2020. LNCS (LNAI), vol. 12167, pp. 489–507. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-51054-1_33

    Chapter  Google Scholar 

Download references

Acknowledgments

ZZ was supported by the European Union, co-financed by the European Social Fund (EFOP-3.6.3-VEKOP-16-2017-00002), the Hungarian National Excellence Grant 2018-1.2.1-NKP-00008 and by the Hungarian Ministry of Innovation and Technology NRDI Office within the framework of the Artificial Intelligence National Laboratory Program. JU was funded by the AI4REASON ERC Consolidator grant nr. 649043 and the European Regional Development Fund under the Czech project AI&Reasoning CZ.02.1.01/0.0/0.0/15_003/0000466. MO was supported by the ERC starting grant no. 714034 SMART. We thank the TABLEAUX’21 reviewers for their thoughtful reviews and comments.

Author information

Authors and Affiliations

  1. Alfréd Rényi Institute of Mathematics, Budapest, Hungary

    Zsolt Zombori

  2. Eötvös Loránd University, Budapest, Hungary

    Zsolt Zombori

  3. Czech Technical University in Prague, Prague, Czechia

    Josef Urban

  4. University of Innsbruck, Innsbruck, Austria

    Miroslav Olšák

Authors
  1. Zsolt Zombori
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Josef Urban
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Miroslav Olšák
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zsolt Zombori .

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    Anupam Das

  2. University of Genoa, Genoa, Italy

    Sara Negri

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zombori, Z., Urban, J., Olšák, M. (2021). The Role of Entropy in Guiding a Connection Prover. In: Das, A., Negri, S. (eds) Automated Reasoning with Analytic Tableaux and Related Methods. TABLEAUX 2021. Lecture Notes in Computer Science(), vol 12842. Springer, Cham. https://doi.org/10.1007/978-3-030-86059-2_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86059-2_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86058-5

  • Online ISBN: 978-3-030-86059-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation