Abstract
We give a general-purpose programming language in which programs can reason about their own knowledge. To specify what these intelligent programs know, we define a “program epistemic” logic, akin to a dynamic epistemic logic for programs. Our logic properties are complex, including programs introspecting into future state of affairs, i.e., reasoning now about facts that hold only after they and other threads will execute. To model aspects anchored in privacy, our logic is interpreted over partial observability of variables, thus capturing that each thread can “see” only a part of the global space of variables. We verify program-epistemic properties on such AI-centred programs. To this end, we give a sound translation of the validity of our program-epistemic logic into first-order validity, using a new weakest-precondition semantics and a book-keeping of variable assignment. We implement our translation and fully automate our verification method for well-established examples using SMT solvers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The post-image of a function f is denoted by \(f^*\), i.e., \(f^*(E) = \bigcup \{f(x) | x\in E\}\).
- 2.
The weakest precondition \(wp(P,\phi )\) is a predicate such that: for any precondition \(\psi \) from which the program P terminates and establishes \(\phi \), \(\psi \) implies \(wp(P,\phi )\).
- 3.
Inputs are Haskell files.
- 4.
When we write \(\{i,i+1\}\) and \(\{i-1,i\}\), we mean \(\{i,i+1 \bmod n\}\) and \(\{i-1 \bmod n,i \}\).
- 5.
The formula \(\langle \beta \rangle \alpha \) reads “after some announcement of \(\beta \), \(\alpha \) is the case”, i.e., \(\beta \) can be truthfully announced and its announcement makes \(\alpha \) true. Formally, \( (W, s) \models \langle \beta \rangle \alpha \text { iff } (W,s) \models \beta \text { and } (W _{|\beta },s) \models \alpha \).
References
Balliu, M., Dam, M., Guernic, G.L.: ENCoVer: symbolic exploration for information flow security. In: 25th IEEE Computer Security Foundations Symposium (CSF 2012), pp. 30–44. IEEE Computer Society (2012). https://doi.org/10.1109/CSF.2012.24
Baltag, A., Moss, L.S., Solecki, S.: The logic of public announcements, common knowledge, and private suspicions. In: Proceedings of the 7th Conference on Theoretical Aspects of Rationality and Knowledge (TARK 1998), pp. 43–56. Morgan Kaufmann Publishers Inc. (1998). https://doi.org/10.1007/978-3-319-20451-2_38
Barfuss, W., Mann, R.P.: Modeling the effects of environmental and perceptual uncertainty using deterministic reinforcement learning dynamics with partial observability. Phys. Rev. E 105, 034409 (2022). https://doi.org/10.1103/PhysRevE.105.034409
Belardinelli, F., Boureanu, I., Malvone, V., Rajaona, S.F.: Program semantics and a verification technique for knowledge-based multi-agent systems. arXiv preprint arXiv:2206.13841 (2022)
van Benthem, J., van Eijck, J., Gattinger, M., Su, K.: Symbolic model checking for dynamic epistemic logic. In: van der Hoek, W., Holliday, W.H., Wang, W. (eds.) LORI 2015. LNCS, vol. 9394, pp. 366–378. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-48561-3_30
Blackburn, P., van Benthem, J.F., Wolter, F.: Handbook of Modal Logic. Elsevier, Oxford (2006)
Blackburn, P., de Rijke, M., Venema, Y.: Modal Logic. Cambridge University Press, New York (2001)
Charrier, T., Herzig, A., Lorini, E., Maffre, F., Schwarzentruber, F.: Building epistemic logic from observations and public announcements. In: Principles of Knowledge Representation and Reasoning: Proceedings of the Fifteenth International Conference (KR 2016), pp. 268–277. AAAI Press (2016)
Chaum, D.: The dining cryptographers problem: unconditional sender and recipient untraceability. J. Cryptol. 1(1), 65–75 (1988). https://doi.org/10.1007/BF00206326
Cimatti, A., Gario, M., Tonetta, S.: A lazy approach to temporal epistemic logic model checking. In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems (AAMAS-38), pp. 1218–1226. IFAAMAS (2016)
Dijkstra, E.W.: A Discipline of Programming. Prentice-Hall, Englewood Cliffs (1976)
van Ditmarsch, H.P., van der Hoek, W., Kooi, B.P.: Dynamic epistemic logic with assignment. In: Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2005), pp. 141–148. Association for Computing Machinery (2005). https://doi.org/10.1145/1082473.1082495
van Ditmarsch, H.P., Hoek, W.V.D., Kooi, B.: Dynamic Epistemic Logic. Synthese Library, Springer, Dordrecht (2007). https://doi.org/10.1007/978-1-4020-5839-4
van Ditmarsch, H., Hartley, M.I., Kooi, B., Welton, J., Yeo, J.B.: Cheryl’s birthday. Electron. Proc. Theor. Comput. Sci. 251, 1–9 (2017). https://doi.org/10.4204/eptcs.251.1
van Eijck, J.: A demo of epistemic modelling. In: Interactive Logic: Selected Papers from the 7th Augustus de Morgan Workshop, p. 303 (2007)
Fagin, R., Halpern, J.Y., Moses, Y., Vardi, M.Y.: Knowledge-based programs. In: Symposium on Principles of Distributed Computing, pp. 153–163. ACM (1995). https://doi.org/10.1145/224964.224982
Fagin, R., Halpern, J.Y., Moses, Y., Vardi, M.Y.: Reasoning About Knowledge. MIT Press, Cambridge (1995)
Gammie, P., van der Meyden, R.: MCK: model checking the logic of knowledge. In: Alur, R., Peled, D.A. (eds.) CAV 2004. LNCS, vol. 3114, pp. 479–483. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-27813-9_41
Gorogiannis, N., Raimondi, F., Boureanu, I.: A novel symbolic approach to verifying epistemic properties of programs. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI 2017), pp. 206–212 (2017). https://doi.org/10.24963/ijcai.2017/30
Grossi, D., Herzig, A., van der Hoek, W., Moyzes, C.: Non-determinism and the dynamics of knowledge. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (2017). https://doi.org/10.24963/ijcai.2017/146
Grossi, D., van der Hoek, W., Moyzes, C., Wooldridge, M.: Program models and semi-public environments. J. Log. Comput. 29(7), 1071–1097 (2016). https://doi.org/10.1093/logcom/exv086
Harel, D.: Dynamic Logic, pp. 497–604. Springer, Dordrecht (1984). https://doi.org/10.1007/978-94-009-6259-0_10
Hintikka, J.: Knowledge and Belief. Cornell University Press, NY (1962)
Jena, M.D., Singhar, S.S., Mohanta, B.K., Ramasubbareddy, S.: Ensuring data privacy using machine learning for responsible data science. In: Satapathy, S.C., Zhang, Y.-D., Bhateja, V., Majhi, R. (eds.) Intelligent Data Engineering and Analytics. AISC, vol. 1177, pp. 507–514. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-5679-1_49
Kacprzak, M., Lomuscio, A., Niewiadomski, A., Penczek, W., Raimondi, F., Szreter, M.: Comparing BDD and SAT based techniques for model checking Chaum’s dining cryptographers protocol. Fund. Inform. 72(1–3), 215–234 (2006)
Kacprzak, M., et al.: VerICS 2007 - a model checker for knowledge and real-time. Fund. Inform. 85(1–4), 313–328 (2008)
Lehman, D.: Knowledge, common knowledge, and related puzzles. In: Proceedings of the Third Annual ACM Symposium on Principles of Distributed Computing, pp. 62–67. ACM (1984). https://doi.org/10.1145/800222.806736
Lomuscio, A., Qu, H., Raimondi, F.: MCMAS: an open-source model checker for the verification of multi-agent systems. Int. J. Softw. Tools Technol. Transf. 19(1), 9–30 (2015). https://doi.org/10.1007/s10009-015-0378-x
McIver, A.K.: The secret art of computer programming. In: Leucker, M., Morgan, C. (eds.) ICTAC 2009. LNCS, vol. 5684, pp. 61–78. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03466-4_3
Morgan, C.: Programming from Specifications, 2nd edn. Prentice Hall, Hoboken (1994)
Morgan, C.: The Shadow Knows: refinement of ignorance in sequential programs. In: Uustalu, T. (ed.) MPC 2006. LNCS, vol. 4014, pp. 359–378. Springer, Heidelberg (2006). https://doi.org/10.1007/11783596_21
Parikh, R., Ramanujam, R.: Distributed processes and the logic of knowledge. In: Parikh, R. (ed.) Logic of Programs 1985. LNCS, vol. 193, pp. 256–268. Springer, Heidelberg (1985). https://doi.org/10.1007/3-540-15648-8_21
Plaza, J.A.: Logics of public communications. In: Proceedings of the 4th International Symposium on Methodologies for Intelligent Systems (1989)
Pratt, V.R.: Semantical considerations on Floyd-Hoare logic. In: 17th Annual Symposium on Foundations of Computer Science, pp. 109–121. IEEE (1976). https://doi.org/10.1109/SFCS.1976.27
Rajaona, S.F.: An algebraic framework for reasoning about privacy. Ph.D. thesis, University of Stellenbosch, Stellenbosch (2016). http://hdl.handle.net/10019.1/106607
Wang, S.: Dynamic epistemic model checking with Yices (2016). https://github.com/airobert/DEL/blob/master/report.pdf. Accessed 28 June 2022
Wooldridge, M., Lomuscio, A.: A computationally grounded logic of visibility, perception, and knowledge. Log. J. IGPL 9(2), 257–272 (2001). https://doi.org/10.1093/jigpal/9.2.257
Acknowledgments
S. Rajaona and I. Boureanu were partly supported by the EPSRC project “AutoPaSS”, EP/S024565/1.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rajaona, F., Boureanu, I., Malvone, V., Belardinelli, F. (2023). Program Semantics and Verification Technique for AI-Centred Programs. In: Chechik, M., Katoen, JP., Leucker, M. (eds) Formal Methods. FM 2023. Lecture Notes in Computer Science, vol 14000. Springer, Cham. https://doi.org/10.1007/978-3-031-27481-7_27
Download citation
DOI: https://doi.org/10.1007/978-3-031-27481-7_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-27480-0
Online ISBN: 978-3-031-27481-7
eBook Packages: Computer ScienceComputer Science (R0)