A SAT-Based Approach to Learn Explainable Decision Sets

  • Alexey Ignatiev
  • Filipe Pereira
  • Nina Narodytska
  • Joao Marques-Silva
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10900)


The successes of machine learning in recent years have triggered a fast growing range of applications. In important settings, including safety critical applications and when transparency of decisions is paramount, accurate predictions do not suffice; one expects the machine learning model to also explain the predictions made, in forms understandable by human decision makers. Recent work proposed explainable models based on decision sets which can be viewed as unordered sets of rules, respecting some sort of rule non-overlap constraint. This paper investigates existing solutions for computing decision sets and identifies a number of drawbacks, related with rule overlap and succinctness of explanations, the accuracy of achieved results, but also the efficiency of proposed approaches. To address these drawbacks, the paper develops novel SAT-based solutions for learning decision sets. Experimental results on computing decision sets for representative datasets demonstrate that SAT enables solutions that are not only the most efficient, but also offer stronger guarantees in terms of rule non-overlap.


  1. 1.
    Agrawal, R., Srikant, R.: Fast algorithms for mining association rules in large databases. In: VLDB, pp. 487–499 (1994)Google Scholar
  2. 2.
    Angelino, E., Larus-Stone, N., Alabi, D., Seltzer, M., Rudin, C.: Learning certifiably optimal rule lists. In: KDD, pp. 35–44 (2017)Google Scholar
  3. 3.
    Bessiere, C., Hebrard, E., O’Sullivan, B.: Minimising decision tree size as combinatorial optimisation. In: CP, pp. 173–187 (2009)CrossRefGoogle Scholar
  4. 4.
    Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability. IOS Press, Amsterdam (2009)zbMATHGoogle Scholar
  5. 5.
    Cendrowska, J.: PRISM: an algorithm for inducing modular rules. Int. J. Man Mach. Stud. 27(4), 349–370 (1987)CrossRefGoogle Scholar
  6. 6.
    Clark, P., Boswell, R.: Rule induction with CN2: some recent improvements. In: Kodratoff, Y. (ed.) EWSL 1991. LNCS, vol. 482, pp. 151–163. Springer, Heidelberg (1991). Scholar
  7. 7.
    Clark, P., Niblett, T.: The CN2 induction algorithm. Mach. Learn. 3, 261–283 (1989)Google Scholar
  8. 8.
    Cohen, W.W.: Fast effective rule induction. In: ICML, pp. 115–123 (1995)CrossRefGoogle Scholar
  9. 9.
    DARPA: DARPA explainable Artificial Intelligence (XAI) program (2016).
  10. 10.
    Eén, N., Sörensson, N.: An extensible SAT-solver. In: SAT, pp. 502–518 (2003)CrossRefGoogle Scholar
  11. 11.
    EU Data Protection Regulation: Regulation (EU) 2016/679 of the European Parliament and of the Council (2016).
  12. 12.
    Feige, U., Mirrokni, V.S., Vondrák, J.: Maximizing non-monotone submodular functions. SIAM J. Comput. 40(4), 1133–1153 (2011)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Fürnkranz, J., Gamberger, D., Lavrac, N.: Foundations of Rule Learning. Springer, Heidelberg (2012). Scholar
  14. 14.
    Han, J., Kamber, M., Pei, J.: Data Mining: Concepts and Techniques, 3rd edn. Morgan Kaufmann, Burlington (2012)zbMATHGoogle Scholar
  15. 15.
    Hauser, J.R., Toubia, O., Evgeniou, T., Befurt, R., Dzyabura, D.: Disjunctions of conjunctions, cognitive simplicity, and consideration sets. J. Mark. Res. 47(3), 485–496 (2010)CrossRefGoogle Scholar
  16. 16.
    ICML, WHI Workshop: ICML Workshop on Human Interpretability in Machine Learning. Accessed Aug 2017
  17. 17.
    IJCAI, XAI Workshop: IJCAI Workshop on Explainable Artificial Intelligence (XAI). Accessed Aug 2017
  18. 18.
    Jordan, M.I., Mitchell, T.M.: Machine learning: trends, perspectives, and prospects. Science 349(6245), 255–260 (2015)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Kamath, A.P., Karmarkar, N., Ramakrishnan, K.G., Resende, M.G.C.: A continuous approach to inductive inference. Math. Program. 57, 215–238 (1992)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Kearns, M.J., Li, M., Valiant, L.G.: Learning boolean formulas. J. ACM 41(6), 1298–1328 (1994)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Lakkaraju, H., Bach, S.H., Leskovec, J.: Interpretable decision sets: a joint framework for description and prediction. In: KDD, pp. 1675–1684 (2016)Google Scholar
  22. 22.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)CrossRefGoogle Scholar
  23. 23.
    Marques-Silva, J., Argelich, J., Graça, A., Lynce, I.: Boolean lexicographic optimization: algorithms & applications. Ann. Math. Artif. Intell. 62(3–4), 317–343 (2011)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Marques-Silva, J., Heras, F., Janota, M., Previti, A., Belov, A.: On computing minimal correction subsets. In: IJCAI, pp. 615–622 (2013)Google Scholar
  25. 25.
    Mencía, C., Previti, A., Marques-Silva, J.: Literal-based MCS extraction. In: IJCAI, pp. 1973–1979 (2015)Google Scholar
  26. 26.
    Michalski, R.S.: On the quasi-minimal solution of the general covering problem. In: International Symposium on Information Processing, pp. 125–128 (1969)Google Scholar
  27. 27.
    Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)zbMATHGoogle Scholar
  28. 28.
    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529 (2015)CrossRefGoogle Scholar
  29. 29.
    Morgado, A., Ignatiev, A., Marques-Silva, J.: MSCG: robust core-guided MaxSAT solving. JSAT 9, 129–134 (2015)MathSciNetGoogle Scholar
  30. 30.
    NIPS, IML Symposium: NIPS Interpretable ML Symposium. Accessed Dec 2017
  31. 31.
    Olson, R.S., La Cava, W., Orzechowski, P., Urbanowicz, R.J., Moore, J.H.: PMLB: a large benchmark suite for machine learning evaluation and comparison. BioData Min. 10(1), 36 (2017)CrossRefGoogle Scholar
  32. 32.
    Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Quinlan, J.R.: Generating production rules from decision trees. In: IJCAI, pp. 304–307 (1987)Google Scholar
  34. 34.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kauffmann, Burlington (1993)Google Scholar
  35. 35.
    Rivest, R.L.: Learning decision lists. Mach. Learn. 2(3), 229–246 (1987)Google Scholar
  36. 36.
    Schaefer, M., Umans, C.: Completeness in the polynomial-time hierarchy: a compendium. SIGACT news 33(3), 32–49 (2002)CrossRefGoogle Scholar
  37. 37.
    Triantaphyllou, E.: Inference of a minimum size boolean function from examples by using a new efficient branch-and-bound approach. J. Global Optim. 5(1), 69–94 (1994)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Triantaphyllou, E.: Data Mining and Knowledge Discovery via Logic-Based Methods: Theory, Algorithms, and Applications. Springer, Heidelberg (2010). Scholar
  39. 39.
    Tseitin, G.S.: On the complexity of derivation in propositional calculus. In: Siekmann, J.H., Wrightson, G. (eds.) Automation of Reasoning. Symbolic Computation (Artificial Intelligence), pp. 466–483. Springer, Heidelberg (1983). Scholar
  40. 40.
    Umans, C., Villa, T., Sangiovanni-Vincentelli, A.L.: Complexity of two-level logic minimization. IEEE Trans. CAD Integr. Circuits Syst. 25(7), 1230–1246 (2006)CrossRefGoogle Scholar
  41. 41.
    Wang, T., Rudin, C., Doshi-Velez, F., Liu, Y., Klampfl, E., MacNeille, P.: Or’s of And’s for interpretable classification, with application to context-aware recommender systems. CoRR, abs/1504.07614 (2015)Google Scholar
  42. 42.
    Wang, T., Rudin, C., Doshi-Velez, F., Liu, Y., Klampfl, E., MacNeille, P.: A Bayesian framework for learning rule sets for interpretable classification. J. Mach. Learn. Res. 18, 1–37 (2017)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Alexey Ignatiev
    • 1
    • 3
  • Filipe Pereira
    • 1
  • Nina Narodytska
    • 2
  • Joao Marques-Silva
    • 1
  1. 1.LASIGE, Faculdade de CiênciasUniversidade de LisboaLisbonPortugal
  2. 2.VMWare ResearchPalo AltoUSA
  3. 3.ISDCT SB RASIrkutskRussia

Personalised recommendations