Rule Extraction with Guaranteed Fidelity

  • Ulf Johansson
  • Rikard König
  • Henrik Linusson
  • Tuve Löfström
  • Henrik Boström
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 437)


This paper extends the conformal prediction framework to rule extraction, making it possible to extract interpretable models from opaque models in a setting where either the infidelity or the error rate is bounded by a predefined significance level. Experimental results on 27 publicly available data sets show that all three setups evaluated produced valid and rather efficient conformal predictors. The implication is that augmenting rule extraction with conformal prediction allows extraction of models where test set errors or test sets infidelities are guaranteed to be lower than a chosen acceptable level. Clearly this is beneficial for both typical rule extraction scenarios, i.e., either when the purpose is to explain an existing opaque model, or when it is to build a predictive model that must be interpretable.


Rule extraction Conformal prediction Decision trees 


  1. 1.
    Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann (1993)Google Scholar
  2. 2.
    Vovk, V., Gammerman, A., Shafer, G.: Algorithmic Learning in a Random World. Springer-Verlag New York, Inc. (2005)Google Scholar
  3. 3.
    Papadopoulos, H.: Inductive conformal prediction: Theory and application to neural networks. Tools in Artificial Intelligence 18, 315–330 (2008)Google Scholar
  4. 4.
    Nguyen, K., Luo, Z.: Conformal prediction for indoor localisation with fingerprinting method. In: Iliadis, L., Maglogiannis, I., Papadopoulos, H., Karatzas, K., Sioutas, S. (eds.) AIAI 2012, Part II. IFIP AICT, vol. 382, pp. 214–223. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  5. 5.
    Makili, L., Vega, J., Dormido-Canto, S., Pastor, I., Murari, A.: Computationally efficient svm multi-class image recognition with confidence measures. Fusion Engineering and Design 86(6), 1213–1216 (2011)CrossRefGoogle Scholar
  6. 6.
    Andrews, R., Diederich, J., Tickle, A.B.: Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl.-Based Syst. 8(6), 373–389 (1995)CrossRefGoogle Scholar
  7. 7.
    Rudy, H.L., Lu, H., Setiono, R., Liu, H.: Neurorule: A connectionist approach to data mining, 478–489 (1995)Google Scholar
  8. 8.
    Fu, L.: Rule learning by searching on adapted nets. In: AAAI, pp. 590–595 (1991)Google Scholar
  9. 9.
    Craven, M.W., Shavlik, J.W.: Extracting tree-structured representations of trained networks. In: Advances in Neural Information Processing Systems, pp. 24–30. MIT Press (1996)Google Scholar
  10. 10.
    Huysmans, J., Baesens, B., Vanthienen, J.: Using rule extraction to improve the comprehensibility of predictive models. FETEW Research Report KBI 0612, K. U. Leuven (2006)Google Scholar
  11. 11.
    Martens, D., Huysmans, J., Setiono, R., Vanthienen, J., Baesens, B.: Rule extraction from support vector machines: An overview of issues and application in credit scoring. In: Rule Extraction from Support Vector Machines, pp. 33–63 (2008)Google Scholar
  12. 12.
    Zhou, Z.H.: Rule extraction: using neural networks or for neural networks? J. Comput. Sci. Technol. 19(2), 249–253 (2004)CrossRefGoogle Scholar
  13. 13.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann (2005)Google Scholar
  14. 14.
    Asuncion, A., Newman, D.J.: UCI machine learning repository (2007)Google Scholar
  15. 15.
    Sayyad Shirabad, J., Menzies, T.: PROMISE Repository of Software Engineering Databases. School of Information Technology and Engineering, University of Ottawa, Canada (2005)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2014

Authors and Affiliations

  • Ulf Johansson
    • 1
  • Rikard König
    • 1
  • Henrik Linusson
    • 1
  • Tuve Löfström
    • 1
  • Henrik Boström
    • 2
  1. 1.School of Business and ITUniversity of BoråsSweden
  2. 2.Department of Systems and Computer SciencesStockholm UniversitySweden

Personalised recommendations