NFM 2017: NASA Formal Methods pp 99-114 | Cite as

On Learning Sparse Boolean Formulae for Explaining AI Decisions

  • Susmit Jha
  • Vasumathi Raman
  • Alessandro Pinto
  • Tuhin Sahai
  • Michael Francis
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10227)

Abstract

In this paper, we consider the problem of learning Boolean formulae from examples obtained by actively querying an oracle that can label these examplesz as either positive or negative. This problem has received attention in both machine learning as well as formal methods communities, and it has been shown to have exponential worst-case complexity in the general case as well as for many restrictions. In this paper, we focus on learning sparse Boolean formulae which depend on only a small (but unknown) subset of the overall vocabulary of atomic propositions. We propose an efficient algorithm to learn these sparse Boolean formulae with a given confidence. This assumption of sparsity is motivated by the problem of mining explanations for decisions made by artificially intelligent (AI) algorithms, where the explanation of individual decisions may depend on a small but unknown subset of all the inputs to the algorithm. We demonstrate the use of our algorithm in automatically generating explanations of these decisions. These explanations will make intelligent systems more understandable and accountable to human users, facilitate easier audits and provide diagnostic information in the case of failure. The proposed approach treats the AI algorithm as a black-box oracle; hence, it is broadly applicable and agnostic to the specific AI algorithm. We illustrate the practical effectiveness of our approach on a diverse set of case studies.

Keywords

Planning Algorithm Priority Queue Inductive Logic Programming Boolean Formula Path Planner 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

References

  1. 1.
    Abouzied, A., Angluin, D., Papadimitriou, C., Hellerstein, J.M., Silberschatz, A.: Learning and verifying quantified boolean queries by example. In: ACM Symposium on Principles of Database Systems, pp. 49–60. ACM (2013)Google Scholar
  2. 2.
    Angluin, D., Computational learning theory: survey and selected bibliography. In: ACM Symposium on Theory of Computing, pp. 351–369. ACM (1992)Google Scholar
  3. 3.
    Angluin, D., Kharitonov, M.: When won’t membership queries help? In: ACM Symposium on Theory of Computing, pp. 444–454. ACM (1991)Google Scholar
  4. 4.
    Bittner, B., Bozzano, M., Cimatti, A., Gario, M., Griggio, A.: Towards pareto-optimal parameter synthesis for monotonie cost functions. In: FMCAD, pp. 23–30, October 2014Google Scholar
  5. 5.
    Boigelot, B., Godefroid, P.: Automatic synthesis of specifications from the dynamic observation of reactive programs. In: Brinksma, E. (ed.) TACAS 1997. LNCS, vol. 1217, pp. 321–333. Springer, Heidelberg (1997). doi: 10.1007/BFb0035397 CrossRefGoogle Scholar
  6. 6.
    Botinčan, M., Babić, D., Sigma*: Symbolic learning of input-output specifications. In: POPL, pp. 443–456 (2013)Google Scholar
  7. 7.
    Cook, B., Kroening, D., Rümmer, P., Wintersteiger, C.M.: Ranking function synthesis for bit-vector relations. FMSD 43(1), 93–120 (2013)MATHGoogle Scholar
  8. 8.
    Ehrenfeucht, A., Haussler, D., Kearns, M., Valiant, L.: A general lower bound on the number of examples needed for learning. Inf. Comput. 82(3), 247–261 (1989)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Elizalde, F., Sucar, E., Noguez, J., Reyes, A.: Generating explanations based on Markov decision processes. In: Aguirre, A.H., Borja, R.M., Garciá, C.A.R. (eds.) MICAI 2009. LNCS (LNAI), vol. 5845, pp. 51–62. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-05258-3_5 CrossRefGoogle Scholar
  10. 10.
    Feng, C., Muggleton, S.: Towards inductive generalisation in higher order logic. In: 9th International Workshop on Machine learning, pp. 154–162 D (2014)Google Scholar
  11. 11.
    Godefroid, P., Taly, A.: Automated synthesis of symbolic instruction encodings from i/o samples. SIGPLAN Not. 47(6), 441–452 (2012)CrossRefGoogle Scholar
  12. 12.
    Goldsmith, J., Sloan, R.H., Szörényi, B., Turán, G.: Theory revision with queries: Horn, read-once, and parity formulas. Artif. Intell. 156(2), 139–176 (2004)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Gurfinkel, A., Belov, A., Marques-Silva, J.: Synthesizing safe bit-precise invariants. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014. LNCS, vol. 8413, pp. 93–108. Springer, Heidelberg (2014). doi: 10.1007/978-3-642-54862-8_7 CrossRefGoogle Scholar
  14. 14.
    Harbers, M., Meyer, J.-J., van den Bosch, K.: Explaining simulations through self explaining agents. J. Artif. Soc. Soc. Simul. 13, 10 (2010)Google Scholar
  15. 15.
    Hellerstein, L., Servedio, R.A.: On PAC learning algorithms for rich boolean function classes. Theoret. Comput. Sci. 384(1), 66–76 (2007)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Jha, S., Seshia, S.A.: A theory of formal synthesis via inductive learning. Acta Informatica, pp. 1–34 (2017)Google Scholar
  17. 17.
    Jha, S., A. Seshia, and A. Tiwari. Synthesis of optimal switching logic for hybrid systems. In: EMSOFT, pp. 107–116. ACM (2011)Google Scholar
  18. 18.
    Kearns, M., Li, M., Valiant, L.: Learning boolean formulas. J. ACM 41(6), 1298–1328 (1994)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Kearns, M., Valiant, L.: Cryptographic limitations on learning boolean formulae and finite automata. Journal of the ACM (JACM) 41(1), 67–95 (1994)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    LaValle, S.M.: Planning Algorithms. Cambridge University Press, Cambridge (2006)CrossRefMATHGoogle Scholar
  21. 21.
    Lecun, Y., Cortes, C.: The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/
  22. 22.
    Lee, J., Moray, N.: Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35(10), 1243–1270 (1992)CrossRefGoogle Scholar
  23. 23.
    Mansour, Y.: Learning boolean functions via the fourier transform. In: Theoretical Advances in Neural Computation and Learning, pp. 391–424 (1994)Google Scholar
  24. 24.
    Nau, D., Ghallab, M., Traverso, P.: Automated Planning: Theory and Practice. Morgan Kaufmann Publishers Inc., San Francisco (2004)MATHGoogle Scholar
  25. 25.
    Pitt, L., Valiant, L.G.: Computational limitations on learning from examples. J. ACM (JACM) 35(4), 965–984 (1988)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Raman, V.: Reactive switching protocols for multi-robot high-level tasks. In: IEEE/RSJ, pp. 336–341 (2014)Google Scholar
  27. 27.
    Raman, V., Lignos, C., Finucane, C., Lee, K.C.T., Marcus, M.P., Kress-Gazit, H.: Sorry Dave, I’m afraid i can’t do that: explaining unachievable robot tasks using natural language. In: Robotics: Science and Systems (2013)Google Scholar
  28. 28.
    Reynolds, A., Deters, M., Kuncak, V., Tinelli, C., Barrett, C.: Counterexample-guided quantifier instantiation for synthesis in SMT. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9207, pp. 198–216. Springer, Cham (2015). doi: 10.1007/978-3-319-21668-3_12 CrossRefGoogle Scholar
  29. 29.
    Ribeiro, M.T., Singh, S., Guestrin, C.: Why Should I Trust You?: Explaining the predictions of any classifier. In: KDD, pp. 1135–1144 (2016)Google Scholar
  30. 30.
    Russell, J., Cohn, R.: OODA Loop. Book on Demand, Norderstedt (2012)Google Scholar
  31. 31.
    Sankaranarayanan, S.: Automatic invariant generation for hybrid systems using ideal fixed points. In: HSCC, pp. 221–230 (2010)Google Scholar
  32. 32.
    Sankaranarayanan, S., Miller, C., Raghunathan, R., Ravanbakhsh, H., Fainekos, G.: A model-based approach to synthesizing insulin infusion pump usage parameters for diabetic patients. In: Annual Allerton Conference on Communication, Control, and Computing, pp. 1610–1617, October 2012Google Scholar
  33. 33.
    Sankaranarayanan, S., Sipma, H.B., Manna, Z.: Constructing invariants for hybrid systems. FMSD 32(1), 25–55 (2008)MATHGoogle Scholar
  34. 34.
    Štrumbelj, E., Kononenko, I.: Explaining prediction models and individual predictions with feature contributions. KIS 41(3), 647–665 (2014)Google Scholar
  35. 35.
    Urban, C., Gurfinkel, A., Kahsai, T.: Synthesizing ranking functions from bits and pieces. In: Chechik, M., Raskin, J.-F. (eds.) TACAS 2016. LNCS, vol. 9636, pp. 54–70. Springer, Heidelberg (2016). doi: 10.1007/978-3-662-49674-9_4 CrossRefGoogle Scholar
  36. 36.
    Yuan, C., Lim, H., Lu, T.-C.: Most relevant explanation in bayesian networks. J. Artif. Intell. Res. (JAIR) 42, 309–352 (2011)MathSciNetMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Susmit Jha
    • 1
  • Vasumathi Raman
    • 1
  • Alessandro Pinto
    • 1
  • Tuhin Sahai
    • 1
  • Michael Francis
    • 1
  1. 1.United Technologies Research CenterBerkeleyUSA

Personalised recommendations