Skip to main content

Advertisement

SpringerLink
Log in

Probabilistic ensemble Fuzzy ARTMAP optimization using hierarchical parallel genetic algorithms

  • Advances in Intelligent Data Processing and Analysis
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this study, a comprehensive methodology for overcoming the design problem of the Fuzzy ARTMAP neural network is proposed. The issues addressed are the sequence of training data for supervised learning and optimum parameter tuning for parameters such as baseline vigilance. A genetic algorithm search heuristic was chosen to solve this multi-objective optimization problem. To further augment the ARTMAP’s pattern classification ability, multiple ARTMAPs were optimized via genetic algorithm and assembled into a classifier ensemble. An optimal ensemble was realized by the inter-classifier diversity of its constituents. This was achieved by mitigating convergence in the genetic algorithms by employing a hierarchical parallel architecture. The best-performing classifiers were then combined in an ensemble, using probabilistic voting for decision combination. This study also integrated the disparate methods to operate within a single framework, which is the proposed novel method for creating an optimum classifier ensemble configuration with minimum user intervention. The methodology was benchmarked using popular data sets from UCI machine learning repository.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Carpenter GA, Grossberg S, Marzukon N, Reynolds JH, Rosen DB (1992) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans Neural Netw 3(5):698–713

    Article  Google Scholar 

  2. Jin M, Li R, Xu Z, Zhao X (2014) Reliable fault diagnosis method using ensemble Fuzzy ARTMAP based on improved Bayesian belief method. Neurocomputing 133(10):309–316

    Article  Google Scholar 

  3. Mokhtar M, Howe J (2013) Comparing the online learning capabilities of Gaussian ARTMAP and Fuzzy ARTMAP for building energy management systems. Expert Syst Appl 40(15):6007–6018

    Article  Google Scholar 

  4. Tran VT, AlThobiani F, Ball A, Choi BK (2013) An application to transient current signal based induction motor fault diagnosis of Fourier–Bessel expansion and simplified fuzzy ARTMAP. Expert Syst Appl 40(13):5372–5384

    Article  Google Scholar 

  5. Liang Z, Xu B, Chi Z, Feng D (2012) Intelligent characterization and evaluation of yarn surface appearance using saliency map analysis, wavelet transform and fuzzy ARTMAP neural network. Expert Syst Appl 39(4):4201–4212

    Article  Google Scholar 

  6. Tan SC, Lim CP (2010) Evolutionary fuzzy artmap neural networks and their applications to fault detection and diagnosis. Neural Process Lett 31(3):219–242

    Article  Google Scholar 

  7. Carpenter GA, Gaddam SC (2010) Biased ART: a neural architecture that shifts attention toward previously disregarded features following an incorrect prediction. Neural Netw 23(3):435–451

    Article  Google Scholar 

  8. Dagher I, Georgiopoulos M, Heileman GL, Bebis G (1999) An ordering algorithm for pattern presentation in fuzzy ARTMAP that tends to improve generalization performance. IEEE Trans Neural Netw 10(4):768–778

    Article  Google Scholar 

  9. Al-Daraiseh A, Georgiopoulos M, Anagnostopoulos G, Wu AS, Mollaghasemi M (2006) GFAM: a genetic algorithm optimization of fuzzy ARTMAP. In: Proceedings of the 2006 IEEE International Conference on Fuzzy Systems, pp 315–322

  10. Granger E, Henniges P, Oliveira L, Sabourin R (2006) Particle swarm optimization of fuzzy ARTMAP parameters. In: Proceedings of the 2006 International Joint Conference on Neural Networks, pp 2060–2067

  11. Palaniappan R, Eswaran C (2009) Using genetic algorithm to select the presentation order of training patterns that improves simplified fuzzy ARTMAP classification performance. Appl Soft Comput 9(1):100–106

    Article  Google Scholar 

  12. Cervantes L, Lee J, Lee J (2007) Agent-based approach to distributed ensemble learning of Fuzzy ARTMAP classifiers. In: Jo GS et al (eds) Agent and multi-agent systems: technologies and applications. Springer, Berlin, pp 805–814

    Chapter  Google Scholar 

  13. de Medeiros Santos A, de Paula Canuto AM (2008) Using ARTMAP-based ensemble systems designed by three variants of boosting. In: Kurkova-Pohlova V et al (eds) Artificial neural networks—ICANN 2008. Springer, Berlin, pp 562–571

    Chapter  Google Scholar 

  14. Williamson JR (1996) Gaussian ARTMAP: a neural network for fast incremental learning of noisy multidimensional maps. Neural Netw 9(5):881–897

    Article  Google Scholar 

  15. Gómez-Sánchez E, Dimitriadis YA, Cano-Izquierdo JM, López-Coronado J (2002) \(\mu\)ARTMAP: use of mutual information for category reduction in fuzzy ARTMAP. IEEE Trans Neural Netw 13(1):58–69

    Article  Google Scholar 

  16. Kaylani A, Georgiopoulos M, Mollaghasemi M, Anagnostopoulos GC (2008) MO-GART: multiobjective genetic ART architectures. In: Proceedings of the 2008 IEEE Congress on Evolutionary Computation, pp 1425–1432

  17. Zhang Y, Ji H, Zhang W (2014) TPPFAM: use of threshold and posterior probability for category reduction in fuzzy ARTMAP. Neurocomputing 124:63–71

    Article  Google Scholar 

  18. Fernández C, Valle C, Saravia F, Allende H (2012) Behavior analysis of neural network ensemble algorithm on a virtual machine cluster. Neural Comput Appl 21(3):535–542

    Article  Google Scholar 

  19. Seera M, Lim CP, Nahavandi S, Loo CK (2014) Condition monitoring of induction motors: a review and an application of an ensemble of hybrid intelligent models. Expert Syst Appl 41(10):4891–4903

    Article  Google Scholar 

  20. Seera M, Lim CP (2014) A hybrid intelligent system for medical data classification. Expert Syst Appl 41(5):2239–2249

    Article  Google Scholar 

  21. Sesmero MP, Alonso-Weber JM, Gutiérrez G, Ledezma A, Sanchis A (2012) A new artificial neural network ensemble based on feature selection and class recoding. Neural Comput Appl 21(4):771–783

    Article  Google Scholar 

  22. Hu JJ, Goodman ED (2002) The hierarchical fair-competition (HFC) model for parallel evolutionary algorithms. In: Proceedings of the 2002 IEEE Congress on Evolutionary Computation, pp 49–54

  23. Zenobi G, Cunningham P (2001) Using diversity in preparing ensembles of classifiers based on different feature subsets to minimize generalization error. In: de Raedt L et al. (eds) Machine learning: ECML 2001, pp 576–587

  24. Yang J, Honavar V (1998) Feature subset selection using a genetic algorithm. In: Liu H et al. (eds) Feature extraction, construction and selection, pp 117–136

  25. Opitz DW (1999) Feature selection for ensembles. In: Proceedings of the 16th National Conference on Artificial Intelligence (AAAI-99), pp 379–384

  26. Cunningham P (2000) Overfitting and diversity in classification ensembles based on feature selection. In: Computer Science Technical Report TCD-CS-2000-07, Trinity College Dublin, pp 1–8

  27. Oliveira LS, Morita M, Sabourin R (2006) Feature selection for ensembles using the multi-objective optimization approach. In: Jin Y (ed) Multi-objective machine learning. Springer, Berlin, pp 49–74

    Chapter  Google Scholar 

  28. Canuto AMP, Vale KMO, Feitosa A (2011) A reinforcement-based mechanism to select features for classifiers in ensemble systems. Int J Comput Inf Syst Ind Manag Appl 3:324–335

    Google Scholar 

  29. Lin X, Yacoub S, Burns J, Simske S (2003) Performance analysis of pattern classifier combination by plurality voting. Pattern Recognit Lett 24(12):1959–1969

    Article  Google Scholar 

  30. Loo CK, Rao M (2005) Accurate and reliable diagnosis and classification using probabilistic ensemble simplified fuzzy ARTMAP. IEEE Trans Knowl Data Eng 17(11):1589–1593

    Article  Google Scholar 

  31. Radtke PV, Sabourin R, Wong T, et al. (2006) Classification system optimization with multi-objective genetic algorithms. In: Tenth International Workshop on Frontiers in Handwriting Recognition

  32. Ishibuchi H, Yamamoto T (2003) Evolutionary multiobjective optimization for generating an ensemble of fuzzy rule-based classifiers. In: Cantu-Paz E et al (eds) Genetic and evolutionary computation GECCO 2003. Springer, Berlin, pp 1077–1088

    Chapter  Google Scholar 

  33. Mohamed MA, Hegazy AEF, Badr AA (2011) Evolutionary fuzzy ARTMAP approach for breast cancer diagnosis. Int J Comput Sci Netw Secur 11(4):77–84

    Google Scholar 

  34. Qasem SN, Shamsuddin SM, Zain AM (2012) Multi-objective hybrid evolutionary algorithms for radial basis function neural network design. Knowledge-Based Syst 27:475–497

    Article  Google Scholar 

  35. Wu CH, Tzeng GH, Lin RH (2009) A Novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression. Expert Syst Appl 36:4725–4735

    Article  Google Scholar 

  36. Frank A, Asuncion A (2011) UCI machine learning repository. http://archive.ics.uci.edu/ml

  37. Dzeroski S, Zenko B (2002) Is combining classifiers better than selecting the best one? In: International Workshop and Conference on Machine Learning, pp 123–130

  38. Partalas I, Tsoumakas G, Katakis I, Vlahavas I (2006) Ensemble pruning using reinforcement learning. In: Advances in artificial intelligence, Springer, pp 301–310

  39. Zhang Y, Burer S, Street WN (2006) Ensemble pruning via semi-definite programming. J Mach Learn Res 7:1315–1338

    MATH  MathSciNet  Google Scholar 

  40. Ordonez FJ, Ledezma A, Sanchis A (2008) Genetic approach for optimizing ensembles of classifiers. In: Proceedings of the Twenty-First International Florida Artificial Intelligence Research Society Conference (FLAIRS) Conference, pp 89–94

  41. Wang SJ, Mathew A, Chen Y, Xi LF, Ma L, Lee J (2009) Empirical analysis of support vector machine ensemble classifiers. Expert Syst Appl 36(3):6466–6476

    Article  Google Scholar 

  42. Wilcoxon F (1945) Individual comparisons by ranking methods. Biometrics 1(6):80–83

    Article  Google Scholar 

  43. Dos Santos EM, Sabourin R, Maupin P (2009) Overfitting cautious selection of classifier ensembles with genetic algorithms. Inf Fus 10(2):150–162

    Article  Google Scholar 

  44. Lofstrom T, Johansson U, Bostrom H (2009) Ensemble member selection using multi-objective optimization. In: IEEE 2009 Symposium on Computational Intelligence and Data Mining, IEEE, pp 245–251

  45. Connolly JF, Granger E, Sabourin R (2012) Evolution of heterogeneous ensembles through dynamic particle swarm optimization for video-based face recognition. Pattern Recognit 45(7):2460–2477

    Article  Google Scholar 

  46. Liu Y, Yao X, Higuchi T (2000) Evolutionary ensembles with negative correlation learning. IEEE Trans Evol Comput 4(4):380–387

    Article  Google Scholar 

Download references

Acknowledgments

This study was supported by University of Malaya Research Grant CG008-2013 and (UMRG) Project RG059-11ICT.

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Information Technology, University Malaya, Kuala Lumpur, Malaysia

    Chu Kiong Loo, Wei Shiung Liew & Manjeevan Seera

  2. Faculty of Biomedical Engineering, University Malaya, Kuala Lumpur, Malaysia

    Einly Lim

Authors
  1. Chu Kiong Loo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Shiung Liew
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manjeevan Seera
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Einly Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chu Kiong Loo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Loo, C.K., Liew, W.S., Seera, M. et al. Probabilistic ensemble Fuzzy ARTMAP optimization using hierarchical parallel genetic algorithms. Neural Comput & Applic 26, 263–276 (2015). https://doi.org/10.1007/s00521-014-1632-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-014-1632-y

Keywords

Search

Navigation