ITER: An Algorithm for Predictive Regression Rule Extraction

  • Johan Huysmans
  • Bart Baesens
  • Jan Vanthienen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4081)


Various benchmarking studies have shown that artificial neural networks and support vector machines have a superior performance when compared to more traditional machine learning techniques. The main resistance against these newer techniques is based on their lack of interpretability: it is difficult for the human analyst to understand the motivation behind these models’ decisions. Various rule extraction techniques have been proposed to overcome this opacity restriction. However, most of these extraction techniques are devised for classification and only few algorithms can deal with regression problems.

In this paper, we present ITER, a new algorithm for pedagogical regression rule extraction. Based on a trained ‘black box’ model, ITER is able to extract human-understandable regression rules. Experiments show that the extracted model performs well in comparison with CART regression trees and various other techniques.


Support Vector Machine Mean Absolute Error Rule Extraction Regression Rule Training Observation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baesens, B., Van Gestel, T., Viaene, S., Stepanova, M., Suykens, J., Vanthienen, J.: Benchmarking state of the art classification algorithms for credit scoring. Journal of the Operational Research Society 54(6), 627–635 (2003)MATHCrossRefGoogle Scholar
  2. 2.
    Barakat, N., Diederich, J.: Eclectic rule-extraction from support vector machines. International Journal of Computational Intelligence 2(1), 59–62 (2005)Google Scholar
  3. 3.
    Breiman, L., Friedman, J.H., Olsen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth and Brooks (1984)Google Scholar
  4. 4.
    Fung, G., Sandilya, S., Rao, R.B.: Rule extraction from linear support vector machines. In: 11th ACM SIGKDD international conference on Knowledge discovery in data mining, pp. 32–40 (2005)Google Scholar
  5. 5.
    Martens, D., Baesens, B., Van Gestel, T., Vanthienen, J.: Adding comprehensibility to support vector machines using rule extraction techniques. In: Credit Scoring and Credit Control IX (2005)Google Scholar
  6. 6.
    Núñez, H., Angulo, C., Català, A.: Rule extraction from support vector machines. In: European Symposium on Artificial Neural Networks (ESANN), pp. 107–112 (2002)Google Scholar
  7. 7.
    Quinlan, J.R.: Learning with Continuous Classes. In: 5th Australian Joint Conference on Artificial Intelligence, pp. 343–348 (1992)Google Scholar
  8. 8.
    Setiono, R., Leow, W.K., Zurada, J.M.: Extraction of rules from artificial neural networks for nonlinear regression. IEEE Transactions on Neural Networks 13(3), 564–577 (2002)CrossRefGoogle Scholar
  9. 9.
    Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific, Singapore (2002)MATHCrossRefGoogle Scholar
  10. 10.
    Viaene, S., Derrig, R., Baesens, B., Dedene, G.: A comparison of state-of-the-art classification techniques for expert automobile insurance fraud detection. Journal of Risk And Insurance (Special Issue on Fraud Detection) 69(3), 433–443 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Johan Huysmans
    • 1
  • Bart Baesens
    • 1
    • 2
  • Jan Vanthienen
    • 1
  1. 1.Dept. of Decision Sciences and Information ManagementK.U.LeuvenLeuvenBelgium
  2. 2.School of ManagementUniversity of SouthamptonSouthamptonUnited Kingdom

Personalised recommendations