Skip to main content

Advertisement

SpringerLink
Log in

Extracting Classification Rules from Artificial Neural Network Trained with Discretized Inputs

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Rule extraction from artificial neural networks remains important task in complex diseases such as diabetes and breast cancer where the rules should be accurate and comprehensible. The quality of rules is improved by the improvement of the network classification accuracy which is done by the discretization of input attributes. In this paper, we developed a rule extraction algorithm based on multiobjective genetic algorithms and association rules mining to extract highly accurate and comprehensible classification rules from ANN’s that have been trained using the discretization of the continuous attributes. The data pre-processing provides very good improvement of the ANN accuracy and consequently leads to improve the performance of the classification rules in terms of fidelity and coverage. The results show that our algorithm is very suitable for medical decision making, so an excellent average accuracy of 94.73 has been achieved for the Pima dataset and 99.36 for the breast cancer dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Boutorh A, Guessoum A (2016) Complex diseases SNP selection and classification by hybrid association rule mining and artificial neural network based evolutionary algorithms. Eng Appl Artif Intell 51:58–70

    Article  Google Scholar 

  2. Fu K, Cheng DW, Tu Y, Zhang L (2016) Credit card fraud detection using convolutional neural networks. In: Proceedings of 23rd international conference on neural information processing, pp 483–490. https://doi.org/10.1007/978-3-319-46675-0_53

  3. Ruz GA, Estévez PA (2005) Image segmentation using fuzzy min-max neural networks for wood defect detection. In: Pham DT, Eldukhri EE, Soroka AJ (eds) Intelligent production machines and systems-first I*PROMS virtual conference, pp 183–188

  4. Fernando H, Surgenor B (2017) An unsupervised artificial neural network versus a rule-based approach for fault detection and identification in an automated assembly machine. Robot Comput Integr Manuf 43:79–88

    Article  Google Scholar 

  5. Luukka P (2007) Similarity classifier using similarity measure derived from Yu’s norms in classification of medical datasets. Comput Biol Med 37:1133–1140

    Article  Google Scholar 

  6. Hayashi Y, Setiono R, Azcarraga A (2016) Neural network training and rule extraction with augmented discretized input. Neurocomputing 207:610–622

    Article  Google Scholar 

  7. Yang SH, Chen YP (2012) An evolutionary constructive and pruning algorithm for artificial neural networks and its prediction applications. Neurocomputing 86:140–149

    Article  Google Scholar 

  8. Towell G, Shavlik JW (1993) The extraction of refined rules from knowledge-based neural networks. Mach Learn 131:71–101

    Google Scholar 

  9. Taha I, Ghosh J (1999) Symbolic interpretation of artificial neural networks. IEEE Trans Knowl Data Eng 11(3):448–463

    Article  Google Scholar 

  10. Khan I, Kulkarni A (2013) Knowledge extraction from survey data using neural networks. Proc Comput Sci 23:433–438

    Article  Google Scholar 

  11. Yedjour D, Benyettou A (2018) Symbolic interpretation of artificial neural networks based on multiobjective genetic algorithms and association rules mining. Appl Soft Comput 72:177–188

    Article  Google Scholar 

  12. Markowska-Kaczmar U (2008) Evolutionary approaches to rule extraction from neural networks. Stud Comput Intell (SCI) 82:177–209

    Google Scholar 

  13. Yedjour D, Aek Benyettou, Yedjour H (2018) Symbolic interpretation of artificial neural networks using genetic algorithms. Turk J Electr Eng Comput Sci 26(5):2465–2475. https://doi.org/10.3906/elk-1801-75

    Article  Google Scholar 

  14. Hruschka ER, Ebecken NFF (2006) Extracting rules from multilayer perceptrons in classification problems: a clustering-based approach. Neurocomputing 70:384–397

    Article  Google Scholar 

  15. Craven M, Shavlik J (1996) Extracting tree-structured representations of trained networks. In: Touretzky DS, Mozer MC, Hasselmo M (eds) Advances in neural information processing systems, vol 8. MIT Press, pp 24–30

  16. Bondarenko A, Aleksejeva L, Jumutc V, Borisov A (2017) Classification tree extraction from trained artificial neural networks. Procedia Comput Sci 104:556–563

    Article  Google Scholar 

  17. Ahmadizar F, Soltanian K, Akhlaghian F, Tsoulos I (2015) Artificial neural network development by means of a novel combination of grammatical evolution and genetic algorithm. Eng Appl Artif Intell 39:1–13

    Article  Google Scholar 

  18. Moore JH, Hill DP (2015) Epistasis analysis using artificial intelligence, Epistasis. Springer, New York, pp 327–346

    Google Scholar 

  19. Markowska-Kaczmar U, Trelak W (2005) Fuzzy logic and evolutionary algorithm—two techniques in rule extraction from neural networks. Neurocomputing 63:359–379

    Article  Google Scholar 

  20. Augasta M, Kathirvalavakumar T (2012) Reverse engineering the neural networks for rule extraction in classification problems. Neural Process Lett 35(2):131–150

    Article  Google Scholar 

  21. Gonçalves LB, Bernardes MM, Vellasco R (2006) Inverted hierarchical neuro-fuzzy bsp system: a novel neuro-fuzzy model for pattern classification and rule extraction in databases. IEEE Trans Syst Man Cybern Part C Appl Rev 36(2):236–248

    Article  Google Scholar 

  22. Liu H, Setiono R (1995) Chi2: feature selection and discretization of numeric attributes. In: Proceedings of Seventh Int’l Conference on Tools with Artificial Intelligence, pp 388–391

  23. Fu X, Wang L (2001) Rule extraction by genetic algorithms based on a simplified RBF neural network. In: Proceedings congress on evolutionary computation, pp 753–758

  24. Markowska-Kaczmar U, Mularczyk K (2006) GA-based rule extraction from neural networks for approximation. In: Proceedings of the international multiconference on computer science and information technology, pp 141–148

  25. Shinde S, Kulkarni U (2016) Extracting classification rules from modified fuzzy min–max neural network for data with mixed attributes. Appl Soft Comput 40:364–378

    Article  Google Scholar 

  26. Hayashi Y, Yukita S (2016) Rule extraction using recursive-rule extraction algorithm with J48graft combined with sampling selection techniques for the diagnosis of type 2 diabetes mellitus in the Pima Indian dataset. Inform Med Unlocked 2:92–104

    Article  Google Scholar 

  27. Zilke JR, Mencía EL, Janssen F (2016) Deepred–rule extraction from deep neural networks. In: International conference on discovery science, Springer, pp 457–473. https://doi.org/10.1007/978-3-319-46307-0_29

  28. Bologna G, Hayashi Y (2018) A comparison study on rule extraction from neural network ensembles, boosted shallow trees, and SVMs. In: Applied computational intelligence and soft computing. https://doi.org/10.1155/2018/4084850

  29. Chakraborty M, Biswas SK, Purkayastha B (2019) Rule extraction from neural network using input data ranges recursively. New Gener. Comput. 37:67–96. https://doi.org/10.1007/s00354-018-0048-0

    Article  Google Scholar 

  30. Mahdavifar S, Ghorbani AA (2020) DeNNeS: deep embedded neural network expert system for detecting cyber attacks. Neural Comput Appl. https://doi.org/10.1007/s00521-020-04830-w

    Article  Google Scholar 

  31. Zitzler E, Thiele L (1999) Multiobjective evolutionary algorithms: a comparative case study and strength Pareto approach. IEEE Trans Evolut Comput 3(4):257–271

    Article  Google Scholar 

  32. Elarbi M, Bechikh S, Ben Said L, Datta R (2017) Multi-objective optimization: classical and evolutionary approaches. In: Recent advances in evolutionary multi-objective optimization. Springer, Berlin, pp 1–30

  33. Agrawal R, Imielinski T, Swami A (1993) Mining association rules between sets of items in large databases. In: Proceedings of ACM-SIMOD international conference on management of data, Washington, DC, pp 207–216

  34. Kabir MMJ, Xu S, Kang BH, Zhao Z (2015) A new evolutionary algorithm for extracting a reduced set of interesting association rules. Neural Inf Process. https://doi.org/10.1007/978-3-319-26535-3_16

    Article  Google Scholar 

  35. Luna JM, Romero JR, Ventura S (2014) On the adaptability of G3PARM to the extraction of rare association rules. Knowl Inf Syst 38(2):391–418

    Article  Google Scholar 

  36. Gadaras I, Mikhailov L (2009) An interpretable fuzzy rule-based classification methodology for medical diagnosis. Artif Intell Med 47:25–41

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Faculty of Mathematics and Computer Science, Université des Sciences et de la Technologie d’Oran Mohamed Boudiaf, USTO-MB, BP 1505, El M’naouer, 31000, Oran, Algeria

    Dounia Yedjour

Authors
  1. Dounia Yedjour
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dounia Yedjour.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yedjour, D. Extracting Classification Rules from Artificial Neural Network Trained with Discretized Inputs. Neural Process Lett 52, 2469–2491 (2020). https://doi.org/10.1007/s11063-020-10357-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-020-10357-x

Keywords

Search

Navigation