Advertisement

New Generation Computing

, Volume 37, Issue 1, pp 67–96 | Cite as

Rule Extraction from Neural Network Using Input Data Ranges Recursively

  • Manomita ChakrabortyEmail author
  • Saroj Kumar Biswas
  • Biswajit Purkayastha
Research Paper
  • 52 Downloads

Abstract

Neural network is one of the best tools for data mining tasks due to its high accuracy. However, one of the drawbacks of neural network is its black box nature. This limitation makes neural network useless for many applications which require transparency in their decision-making process. Many algorithms have been proposed to overcome this drawback by extracting transparent rules from neural network, but still researchers are in search for algorithms that can generate more accurate and simple rules. Therefore, this paper proposes a rule extraction algorithm named Eclectic Rule Extraction from Neural Network Recursively (ERENNR), with the aim to generate simple and accurate rules. ERENNR algorithm extracts symbolic classification rules from a single-layer feed-forward neural network. The novelty of this algorithm lies in its procedure of analyzing the nodes of the network. It analyzes a hidden node based on data ranges of input attributes with respect to its output and analyzes an output node using logical combination of the outputs of hidden nodes with respect to output class. And finally it generates a rule set by proceeding in a backward direction starting from the output layer. For each rule in the set, it repeats the whole process of rule extraction if the rule satisfies certain criteria. The algorithm is validated with eleven benchmark datasets. Experimental results show that the generated rules are simple and accurate.

Keywords

Neural network Data mining Rule extraction Classification Re-RX algorithm RxREN algorithm 

References

  1. 1.
    Anbananthen, S.K., Sainarayanan, G., Chekima, A., Teo, J.: Data mining using pruned artificial neural network tree (ANNT). Inf. Commun. Technol. 1, 1350–1356 (2006)Google Scholar
  2. 2.
    Augasta, M.G., Kathirvalavakumar, T.: Rule extraction from neural networks—a comparative study. International Conference on Pattern Recognition. Informatics and Medical Engineering, Salem, Tamilnadu, 404–408 (2012)Google Scholar
  3. 3.
    Augusta, M.G., Kathirvalavakumar, T.: Reverse engineering the neural networks for rule extraction in classification problems. Neural Process. Lett. 35(2), 131–150 (2012)CrossRefGoogle Scholar
  4. 4.
    Biswas, S.K., Chakraborty, M., Singh, H.R., Devi, D., Purkayastha, B., Das, A.K.: Hybrid case-based reasoning system by cost-sensitive neural network for classification. Soft Comput. 24, 1–18 (2016)Google Scholar
  5. 5.
    Biswas, S.K., Chakraborty, M., Purkayastha, B., Thounaojam, D.M., Roy, P.: Rule extraction from training data using neural network. Int. J. Artif. Intell. Tool 26, 3 (2017)Google Scholar
  6. 6.
    Chakraborty, M., Biswas, S.K., Purkayastha, B.: Recursive rule extraction from NN using reverse engineering technique. New Gener. Comput. 36(2), 119–142 (2018)CrossRefGoogle Scholar
  7. 7.
    Craven, M., Shavlik, J.: Extracting tree-structured representations of trained network. Advances in Neural Information Processing Systems (NIPS). MIT Press, Cambridge 8, 24–30 (1996)Google Scholar
  8. 8.
    Etchells, T.A., Lisboa, P.J.G.: Orthogonal search-based rule extraction (OSRE) for trained neural networks: a practical and efficient approach. IEEE Trans. Neural Networks 17(2), 374–384 (2006)CrossRefGoogle Scholar
  9. 9.
    Fortuny, E.J., Martens, D.: Active learning-based pedagogical rule extraction. IEEE Trans. Neural Netw. Learn. Syst. 26(11), 2664–2677 (2015)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Hayashi, Y., Nakano, S., Fujisawa, S.: Use of the recursive-rule extraction algorithm with continuous attributes to improve diagnostic accuracy in thyroid disease. Inf. Med. Unlocked 1, 1–8 (2015)CrossRefGoogle Scholar
  11. 11.
    Hayashi, Y., Yukita, S.: Rule extraction using recursive-rule extraction algorithm with J48graft combined with sampling selection techniques for the diagnosis of type 2 diabetes mellitus in the Pima Indian dataset. Inf. Med. Unlocked 2, 92–104 (2016)CrossRefGoogle Scholar
  12. 12.
    Hayashi, Y.: Application of a rule extraction algorithm family based on the Re-RX algorithm to financial credit risk assessment from a Pareto optimal perspective. Oper. Res. Perspect. 3, 32–42 (2016)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Hailesilassie, T.: Ensemble neural network rule extraction using Re-RX algorithm. Int. J. Comput. Sci. Inf. Sec. 14, 7 (2016)Google Scholar
  14. 14.
    Hara, A., Hayashi, Y.: Ensemble neural network rule extraction using re-RX algorithm. Neural Networks (IJCNN), 1–6 (2012)Google Scholar
  15. 15.
    Iqbal, R.A.: Eclectic rule extraction from neural networks using aggregated decision trees. IEEE, 7th International Conference on Electrical & Computer Engineering (ICECE), 129–132 (2012)Google Scholar
  16. 16.
    Jivani, K., Ambasana, J., Kanani, S.: A survey on rule extraction approaches based techniques for data classification using neural network. Int. J. Futuristic Trends Eng. Technol. 1(1), 4–7 (2014)Google Scholar
  17. 17.
    Kumar, S.: Neural networks: a classroom approach, 2nd edn. Tata McGraw-Hill Education, New Delhi (2004)Google Scholar
  18. 18.
    Odajimaa, K., Hayashi, Y., Tianxia, G., Setiono, R.: Greedy rule generation from discrete data and its use in neural network rule extraction. Neural Netw. 21(7), 1020–1028 (2008)CrossRefzbMATHGoogle Scholar
  19. 19.
    Permanasari, A. E., Rambli, D.R.A., Dominic, P.D.D.: Forecasting of salmonellosis incidence in human using Artificial Neural Network (ANN). Computer and Automation Engineering (ICCAE), The 2nd International Conference, 1, 136–139 (2010)Google Scholar
  20. 20.
    Sestito, S., Dillon, T.: Automated knowledge acquisition of rules with continuously valued attributes. In: Proceedings of 12th International Conference on Expert Systems and their Applications, 645–656 (1992)Google Scholar
  21. 21.
    Setiono, R., Liu, H.: Symbolic representation of neural networks. IEEE Comput. 29(3), 71–77 (1996)CrossRefGoogle Scholar
  22. 22.
    Setiono, R.: Extracting rules from neural networks by pruning and hidden-unit splitting. Neural Comput. 9(1), 205–225 (1997)CrossRefzbMATHGoogle Scholar
  23. 23.
    Setiono, R., Liu, H.: NeuroLinear: from neural networks to oblique decision rules. Neurocomputing 17(1), 1–24 (1997)CrossRefGoogle Scholar
  24. 24.
    Setiono, R., Kheng, W.: FERNN: an algorithm for fast extraction of rules from neural networks. Appl. Intell. 12(1), 15–25 (2000)CrossRefGoogle Scholar
  25. 25.
    Setiono, R., Baesens, B., Mues, C.: Recursive neural network rule extraction for data with mixed attributes. IEEE Trans. Neural Networks 19(2), 299–307 (2008)CrossRefGoogle Scholar
  26. 26.
    Taha, I.A., Ghosh, J.: Symbolic interpretation of artificial neural networks. IEEE Trans. Knowl. Data Eng. 11(3), 448–463 (1999)CrossRefGoogle Scholar
  27. 27.
    Towel, G., Shavlik, J.: The extraction of refined rules from knowledge based neural networks. Mach. Learn. 13(1), 71–101 (1993)Google Scholar
  28. 28.
    Zhou, Z.H.: Rule extraction: using neural networks or for neural networks? J. Comput. Sci. Technol. 19(2), 249–253 (2004)MathSciNetCrossRefGoogle Scholar

Copyright information

© Ohmsha, Ltd. and Springer Japan KK, part of Springer Nature 2018

Authors and Affiliations

  • Manomita Chakraborty
    • 1
    Email author
  • Saroj Kumar Biswas
    • 1
  • Biswajit Purkayastha
    • 1
  1. 1.Computer Science and Engineering DepartmentNational Institute of Technology SilcharSilcharIndia

Personalised recommendations