A New Neural Data Analysis Approach Using Ensemble Neural Network Rule Extraction

  • Atsushi Hara
  • Yoichi Hayashi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7552)


In this paper, we propose the Ensemble-Recursive-Rule eXtraction (E-Re-RX) algorithm, which is a rule extraction method from ensemble neural networks. We demonstrate that the use of ensemble neural networks produces higher recognition accuracy than individual neural networks and the extracted rules are more comprehensible. E-Re-RX algorithm is an effective rule extraction algorithm for dealing with data sets that mix discrete and continuous attributes. In this algorithm, primary rules are generated as well as secondary rules to handleonlythoseinstances that do not satisfy the primary rules, and then these rules are integrated. We show that this reduces the complexity of using multiple neural networks. This method achieves extremely high recognition rates, even with multiclass problems.


Ensemble neural network rule extraction Re-Rx Algorithm Ensemble method Recursive neural network rule extraction 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Breiman, L.: Bagging predictors. Mach. Learn. 24, 123–140 (1996)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Freund, Y., Schapire, R.: Experiments with a new boosting algorithm. In: Proc. of the Thirteenth International Conference on Machine Learning, Bari, Italy, pp. 148–156 (1996)Google Scholar
  3. 3.
    Hayashi, Y., Setiono, R.: Combining neural network predictions for medical diagnosis. Comput. Biol. Med. 32, 237–246 (2002)CrossRefGoogle Scholar
  4. 4.
    Hartono, P.: Ensemble of linear experts as an interpretable piecewise linear classifier. ICIC Exp. Lett. 2, 295–303 (2008)Google Scholar
  5. 5.
    Setiono, R., Baesens, B., Mues, C.: A note on knowledge discovery using neural networks and its application to credit card screening. Eur. J. Oper. Res. 192, 326–332 (2009)zbMATHCrossRefGoogle Scholar
  6. 6.
    Setiono, R.: A penalty-function approach for pruning feedforward neural networks. Neural Comp. 9(1), 185–204 (1997)zbMATHCrossRefGoogle Scholar
  7. 7.
    Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press (1995)Google Scholar
  8. 8.
    Setiono, R., Baesens, B., Mues, C.: Recursive neural network rule extraction for data with mixed attributes. IEEE Trans. Neural Netw. 19, 299–307 (2008)CrossRefGoogle Scholar
  9. 9.
    Akhand, M.A.H., Murase, K.: Neural Network Ensembles. Lambert Academic Publishing (LAP) (2010)Google Scholar
  10. 10.
    Alpaydin, E.: Multiple Neural Networks and Weighted Voting. IEEE Trans. on Pattern Recognition 2, 29–32 (1992)Google Scholar
  11. 11.
    University of California, Irvine Machine Learning Repository,
  12. 12.
    Hara, A., Hayashi, Y.: Ensemble neural network rule extraction using Re-RX algorithm. In: Proc. of IJCNN 2012 (2012) (under review)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Atsushi Hara
    • 1
  • Yoichi Hayashi
    • 2
  1. 1.Fujitsu Social Science Laboratory Ltd.Nakahara-kuJapan
  2. 2.Department of Computer ScienceMeiji UniversityTama-kuJapan

Personalised recommendations