Skip to main content
SpringerLink
Log in

Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Artificial neural networks often achieve high classification accuracy rates, but they are considered as black boxes due to their lack of explanation capability. This paper proposes the new rule extraction algorithm RxREN to overcome this drawback. In pedagogical approach the proposed algorithm extracts the rules from trained neural networks for datasets with mixed mode attributes. The algorithm relies on reverse engineering technique to prune the insignificant input neurons and to discover the technological principles of each significant input neuron of neural network in classification. The novelty of this algorithm lies in the simplicity of the extracted rules and conditions in rule are involving both discrete and continuous mode of attributes. Experimentation using six different real datasets namely iris, wbc, hepatitis, pid, ionosphere and creditg show that the proposed algorithm is quite efficient in extracting smallest set of rules with high classification accuracy than those generated by other neural network rule extraction methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Han J, Kambler M (2001) DataMining: concepts and techniques. Morgan Kaufmann, San Francisco

    Google Scholar 

  2. Cohen S, Rokach L, Maimon O (2007) Decision-tree instance-space decomposition with grouped gain-ratio. Inf Sci 177(17): 3592–3612

    Article  Google Scholar 

  3. Yager RR (2006) An extension of the nave Bayesian classifier. Inf Sci 176(5): 577–588

    Article  MathSciNet  Google Scholar 

  4. Kaikhah K, Doddmeti S (2006) Discovering trends in large datasets using neural network. Appl Intel 29: 51–60

    Article  Google Scholar 

  5. Dam H, Abbass HA, Lokan C, Xin yao (2008) Neural based learning classifier systems. IEEE Trans Knowl Data Eng 20(1): 26–39

    Article  Google Scholar 

  6. Mantas CJ, Puche JM, Mantas JM (2006) Extraction of similarity based fuzzy rules from artificial neural networks. Int J Approx Reason 43: 202–221

    Article  MathSciNet  MATH  Google Scholar 

  7. Setiono R, Liu H (1995) Understanding neural networks via rule extraction. In: Proceedings of 14th international joint conference on artificial intelligence, pp 480–485

  8. Tsukimoto H (2000) Extracting rules from trained neural networks. IEEE Trans Neural Netw 11(2): 377–389

    Article  Google Scholar 

  9. Jacobson H (2005) Rule extraction from recurrent neural networks: a taxonomy and review. Neural Comput 17(6): 1223–1263

    Article  MathSciNet  Google Scholar 

  10. Nayak R (2009) Generating rules with predicates, terms and variables from the pruned neural networks. Neural Netw 22: 405–414

    Article  Google Scholar 

  11. Taylor BJ, Darrah MA (2005) Rule extraction as a formal method for the verification and validation of neural networks. IEEE Int Joint Conf Neural Netw 5: 2915–2920

    Article  Google Scholar 

  12. Tickle AB, Andrews R, Golea M, Diederich J (1998) The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks. IEEE Trans Neural Netw 6: 1057–1068

    Article  Google Scholar 

  13. Chauvin Y (1990) Generalization performance of overtrained backpropagation networks. In: Hlomeida LB, Wellekens CJ (eds) Proceedings of neural networks euroship workshop, pp 46–55

  14. Reed R (1993) Pruning algorithm-a survey. IEEE Trans Neural Netw 4: 740–747

    Article  Google Scholar 

  15. Zurada JM (2002) Introduction to artificial neural systems. Jaisco Publishing House, Mumbai

    Google Scholar 

  16. Setiono R, Baesens B, Mues C (2008) A note on knowledge discovery using neural networks and its application to credit card screening. Eur J Oper Res 192(1): 326–332

    Article  Google Scholar 

  17. Attik M,Bougrain L, Alexandra F (2005) Neural network topology optimization. In: Proceedings of ICANN’05, vol 3697, pp 53–58

  18. Hassibi B, Stork DG, Wolf GJ (1993) Optimal brain surgeon and general network pruning. In: Proceedings of IEEE ICNN 1, pp 293–299

  19. Le Cun Y, Denker JS, Solla SA (1990) Optimal brain damage. In: Touretzky DS (ed) Advances neural information processing systems. Morgan Kaufmann, San Mateo, pp 598–605

    Google Scholar 

  20. Engelbrecht AP (2001) A new pruning heuristic based on variance analysis of sensitivity information. IEEE Trans Neural Netw 12(6): 1386–1399

    Article  Google Scholar 

  21. Xing H-J, Hu B-G (2009) Two phase construction of multilayer perceptrons using information theory. IEEE Trans Neural Netw 20(4): 715–721

    Article  Google Scholar 

  22. Castellano G, Fanelli AM, Pelillo M (1997) An iterative pruning algorithm for feedforward neural networks. IEEE Trans Neural Netw 8(3): 519–530

    Article  Google Scholar 

  23. Zeng X, Yeung DS (2006) Hidden neuron pruning of multilayer perceptrons using a quantified sensitivity measure. Neurocomputing 69: 825–837

    Article  Google Scholar 

  24. Chikofsky EJ, Cross JH (1990) Reverse engineering and design recovery: a taxonomy. IEEE Softw 7(1): 13–17

    Article  Google Scholar 

  25. Nattapon P, Nittaya K, Kittisak K (2010) Database reverse engineering based on association rule mining. IJCSI Int J Comput Sci Issues 7(2): 10–14

    Google Scholar 

  26. Setiono R, Liu H (1996) Symbolic representation of neural networks. IEEE Comput 29(3): 71–77

    Article  Google Scholar 

  27. Setiono R (1997) Extracting rules from neural networks by pruning and hidden-unit splitting. Neural Comput 9(1): 205–225

    Article  MATH  Google Scholar 

  28. Gupta A, Park S, Lam SW (1999) Generalized analytic rule extraction for feedforward neural networks. IEEE Trans Knowl Data Eng 11(6): 985–991

    Article  Google Scholar 

  29. Etchells TA, Lisboa PJG (2006) Orthogonal search-based rule extraction (OSRE) for trained neural networks: a practical and efficient approach. IEEE Trans Knowl Data Eng 17(2): 374–384

    Google Scholar 

  30. Setiono R, Baesens B, Mues C (2008) Recursive neural network rule extraction for data with mixed attributes. IEEE Trans Knowl Data Eng 19(2): 299–307

    Google Scholar 

  31. Emad EW, Wunsch II DC (2007) Neural network explanation using inversion. Neural Netw 20: 78–93

    Article  MATH  Google Scholar 

  32. Andrews R, Diederich J, Tickle AB (1996) Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl Based Syst 8(6): 373–389

    Article  Google Scholar 

  33. Setiono R, Liu H (1997) Neurolinear: from neural networks to oblique decision rules. Neural Comput 17(1): 1–24

    Google Scholar 

  34. Kim D, Lee J (2000) Handling continuous-valued attributes in decision tree with neural network modeling. In: Proceedings of the 11th European conference on machine learning. Lecture notes in computer science, vol 1810, pp 211–219

  35. Koichi O, Yoichi H, Gong T, Rudy S (2008) Greedy rule generation from discrete data and its use in neural network rule extraction. Neural Netw 21: 1020–1028

    Article  Google Scholar 

  36. Saito K, Nakano R (1988) Medical diagnostic expert system based on pdp model. In: Proceedings of IEEE international conference on neural networks, vol 1, pp 255–262

  37. Sestito S, Dillon TS (1992) Automated knowledge acquisition of rules with continuously valued attributes. In: Proceedings of 12th international conference on expert systems and their applications, pp 645–656

  38. Thrun SB (1993) Extracting provably correct rules from artificial neural networks. Technical Report IAI-TR-93-5, Institut für informatik III, University of Bonn

  39. Pop E, Hayward R, Diederich J (1994) RULENEG: extracting rules from a trained ANN by stepwise negation. Tech Rep, QUT NRC

  40. Craven MW, Shavlik JW (1994) Using sampling and queries to extract rules from trained neural networks. In: Proceedings of the 11th international conference on machine learning, San Francisco, CA

  41. Filer R, Sethi I, Austin J (1997) A comparison between two rule extraction methods for continuous input data. In: Proceedings of neural information processing systems, rule extraction from trained artificial neural networks workshop, pp 38–45

  42. Hruschka ER, Ebecken NFF (2006) Extracting rules from multilayer perceptrons in classification problems: a clustering-based approach. Neurocomputing 70: 384–397

    Article  Google Scholar 

  43. Kahramanli H, Allahverdi N (2009) Rule extraction from trained adaptive neural networks using artificial immune systems. Expert Syst Appl 36: 1513–1522

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Applications, Sarah Tucker College, Tirunelveli, 627007, India

    M. Gethsiyal Augasta

  2. Department of Computer Science, V.H.N.S.N. College, Virudhunagar, 626001, India

    T. Kathirvalavakumar

Authors
  1. M. Gethsiyal Augasta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. Kathirvalavakumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to T. Kathirvalavakumar.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Augasta, M.G., Kathirvalavakumar, T. Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems. Neural Process Lett 35, 131–150 (2012). https://doi.org/10.1007/s11063-011-9207-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-011-9207-8

Keywords

Search

Navigation