Evolving Systems

, Volume 6, Issue 4, pp 269–292 | Cite as

Generalized smart evolving fuzzy systems

  • Edwin Lughofer
  • Carlos Cernuda
  • Stefan Kindermann
  • Mahardhika Pratama
Original Paper

Abstract

In this paper, we propose a new methodology for learning evolving fuzzy systems (EFS) from data streams in terms of on-line regression/system identification problems. It comes with enhanced dynamic complexity reduction steps, acting on model components and on the input structure and by employing generalized fuzzy rules in arbitrarily rotated position. It is thus termed as Gen-Smart-EFS (GS-EFS), short for generalized smart evolving fuzzy systems. Equipped with a new projection concept for high-dimensional kernels onto one-dimensional fuzzy sets, our approach is able to provide equivalent conventional TS fuzzy systems with axis-parallel rules, thus maintaining interpretability when inferring new query samples. The on-line complexity reduction on rule level integrates a new merging concept based on a combined adjacency–homogeneity relation between two clusters (rules). On input structure level, complexity reduction is motivated by a combined statistical-geometric concept and acts in a smooth and soft manner by incrementally adapting feature weights: features may get smoothly out-weighted over time (\(\rightarrow\)soft on-line dimension reduction) but also may become reactivated at a later stage. Out-weighted features will contribute little to the rule evolution criterion, which prevents the generation of unnecessary rules and reduces over-fitting due to curse of dimensionality. The criterion relies on a newly developed re-scaled Mahalanobis distance measure for assuring monotonicity between feature weights and distance values. Gen-Smart-EFS will be evaluated based on high-dimensional real-world data (streaming) sets and compared with other well-known (evolving) fuzzy systems approaches. The results show improved accuracy with lower rule base complexity as well as smaller rule length when using Gen-Smart-EFS.

Keywords

Data stream regression Generalized evolving fuzzy systems (GS-EFS) Rule merging Adjacency–homogeneity relation Soft and smooth on-line dimension reduction Re-scaled Mahalanobis distance measure 

References

  1. Abonyi J, Babuska R, Szeifert F (2002) Modified Gath-Geva fuzzy clustering for identification of Takagi–Sugeno fuzzy models. IEEE Trans Syst Man Cybern Part B 32(5):612–621CrossRefGoogle Scholar
  2. Angelov P (2010) Evolving Takagi–Sugeno fuzzy systems from streaming data, eTS+. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 21–50CrossRefGoogle Scholar
  3. Angelov P, Filev D (2004) An approach to online identification of Takagi-Sugeno fuzzy models. IEEE Trans Syst Man Cybern Part B Cybern 34(1):484–498CrossRefGoogle Scholar
  4. Angelov P, Filev D, Kasabov N (2010) Evolving intelligent systems—methodology and applications. Wiley, New YorkCrossRefGoogle Scholar
  5. Angelov P, Kordon A (2010) Evolving inferential sensors in the chemical process industry. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems—methodology and applications. Wiley, New York, pp 313–336CrossRefGoogle Scholar
  6. Angelov P, Lughofer E, Zhou X (2008) Evolving fuzzy classifiers using different model architectures. Fuzzy Sets Syst 159(23):3160–3182MATHMathSciNetCrossRefGoogle Scholar
  7. Angelov P, Sadeghi-Tehran P, Ramezani R (2011) An approach to automatic real-time novelty detection, object identification, and tracking in video streams based on recursive density estimation and evolving Takagi–Sugeno fuzzy systems. Int J Intell Syst 26(3):189–205MATHCrossRefGoogle Scholar
  8. Angelov P, Zhou XW (2006) Evolving fuzzy systems from data streams in real-time. In: 2006 international symposium on evolving fuzzy systems (EFS’06), Ambleside, pp 29–35 (2006)Google Scholar
  9. Babuska R (1998) Fuzzy modeling for control. Kluwer Academic Publishers, NorwellCrossRefGoogle Scholar
  10. Backer SD, Scheunders P (2001) Texture segmentation by frequency-sensitive elliptical competitive learning. Image Vis Comput 19(9–10):639–648CrossRefGoogle Scholar
  11. Bauer F, Lukas M (2011) Comparing parameter choice methods for regularization of ill-posed problems. Math Comput Simul 81(9):1795–1841MATHMathSciNetCrossRefGoogle Scholar
  12. Bhattacharyya A (1943) On a measure of divergence between two statistical populations defined by their probability distributions. Bull Calcutta Math Soc 35:99–109MATHMathSciNetGoogle Scholar
  13. Bifet A, Kirkby R (2011) Data stream mining—a practical approach. Tech. rep., Department of Computer Sciences, University of Waikato, JapanGoogle Scholar
  14. Bouchachia A, Mittermeir R (2006) Towards incremental fuzzy classifiers. Soft Comput 11(2):193–207CrossRefGoogle Scholar
  15. Casillas J, Cordon O, Herrera F, Magdalena L (2003) Interpretability issues in fuzzy modeling. Springer, BerlinMATHCrossRefGoogle Scholar
  16. Castro J, Delgado M (1996) Fuzzy systems with defuzzification are universal approximators. IEEE Trans Syst Man Cybern Part B Cybern 26(1):149–152CrossRefGoogle Scholar
  17. Chen H, Tino P, Yao X, Rodan A (2014) Learning in the model space for fault diagnosis. IEEE Trans Neural Netw Learn Syst 25(1):124–136CrossRefGoogle Scholar
  18. Cheng W, Juang C (2011) An incremental support vector machine-trained TS-type fuzzy system for online classification problems. Fuzzy Sets Syst 163(1):24–44MATHMathSciNetCrossRefGoogle Scholar
  19. Cohen L, Avrahami-Bakish G, Last M, Kandel A, Kipersztok O (2008) Real-time data mining of non-stationary data streams from sensor networks. Inf Fusion 9(3):344–353CrossRefGoogle Scholar
  20. Costa B, Angelov P, Guedes L (2015) Fully unsupervised fault detection and identification based on recursive density estimation and self-evolving cloud-based classifier. Neurocomputing 150(A):289–303Google Scholar
  21. Diehl C, Cauwenberghs G (2003) SVM incremental learning, adaptation and optimization. In: Proceedings of the international joint conference on neural networks, vol 4, pp 2685–2690, Boston (2003)Google Scholar
  22. Djouadi A, Snorrason O, Garber F (1990) The quality of training-sample estimates of the Bhattacharyya coefficient. IEEE Trans Pattern Anal Mach Intell 12(1):92–97CrossRefGoogle Scholar
  23. Efendic H, Re LD (2006) Automatic iterative fault diagnosis approach for complex systems. WSEAS Trans Syst 5(2):360–367Google Scholar
  24. Eitzinger C, Heidl W, Lughofer E, Raiser S, Smith J, Tahir M, Sannen D, van Brussel H (2010) Assessment of the influence of adaptive components in trainable surface inspection systems. Mach Vis Appl 21(5):613–626CrossRefGoogle Scholar
  25. Gama J (2010) Knowledge discovery from data streams. Chapman & Hall/CRC, Boca RatonMATHCrossRefGoogle Scholar
  26. Gray R (1984) Vector quantization. IEEE ASSP Mag 1(2):4–29CrossRefGoogle Scholar
  27. Hametner C, Jakubek S (2013) Local model network identification for online engine modelling. Inf Sci 220:210–225CrossRefGoogle Scholar
  28. Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference and prediction, 2nd edn. Springer, New YorkCrossRefGoogle Scholar
  29. Heidl W, Thumfart S, Lughofer E, Eitzinger C, Klement E (2013) Machine learning based analysis of gender differences in visual inspection decision making. Inf Sci 224:62–76MathSciNetCrossRefGoogle Scholar
  30. Hill T, Lewicki P (2007) Statistics: methods and applications. StatSoft, TulsaGoogle Scholar
  31. Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65–70MATHMathSciNetGoogle Scholar
  32. Huang G, Saratchandran P, Sundararajan N (2004) An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks. IEEE Trans Syst Man Cybern Part B Cybern 34(6):2284–2292CrossRefGoogle Scholar
  33. Iglesias J, Angelov P, Ledezma A, Sanchis A (2010) Evolving classification of agent’s behaviors: a general approach. Evol Syst 1(3):161–172CrossRefGoogle Scholar
  34. Jang JS (1993) ANFIS: adaptive-network-based fuzzy inference systems. IEEE Trans Syst Man Cybern 23(3):665–685CrossRefGoogle Scholar
  35. Jimenez L, Landgrebe D (1998) Supervised classification in high-dimensional space: geometrical, statistical, and asymptotical properties of multivariate data. IEEE Trans Syst Man Cybern Part C Rev Appl 28(1):39–54CrossRefGoogle Scholar
  36. Jin Y (2000) Fuzzy modelling of high dimensional systems: complexity reduction and interpretability improvement. IEEE Trans Fuzzy Syst 8(2):212–221CrossRefGoogle Scholar
  37. Klement E, Mesiar R, Pap E (2000) Triangular norms. Kluwer Academic Publishers, DordrechtMATHCrossRefGoogle Scholar
  38. Kohonen T (1995) Self-organizing maps, 2nd edn. Springer, BerlinCrossRefGoogle Scholar
  39. Komijani M, Lucas C, Araabi B, Kalhor A (2012) Introducing evolving Takagi–Sugeno method based on local least squares support vector machine models. Evol Syst 3(2):81–93CrossRefGoogle Scholar
  40. Krishnamoorthy K, Mathew T (2009) Statistical tolerance regions: theory, applications, and computation. Wiley, HobokenCrossRefGoogle Scholar
  41. Leite D, Ballini R, Costa P, Gomide F (2012) Evolving fuzzy granular modeling from nonstationary fuzzy data streams. Evol Syst 3(2):65–79CrossRefGoogle Scholar
  42. Leite D, Costa P, Gomide F (2012) Interval approach for evolving granular system modeling. In: Sayed-Mouchaweh M, Lughofer E (eds) Learning in non-stationary environments: methods and applications. Springer, New York, pp 271–300CrossRefGoogle Scholar
  43. Lemos A, Caminhas W, Gomide F (2011) Multivariable Gaussian evolving fuzzy modeling system. IEEE Trans Fuzzy Syst 19(1):91–104CrossRefGoogle Scholar
  44. Lemos A, Caminhas W, Gomide F (2013) Adaptive fault detection and diagnosis using an evolving fuzzy classifier. Inf Sci 220:64–85CrossRefGoogle Scholar
  45. Liang N, Huang G, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17:1411–1423CrossRefGoogle Scholar
  46. Lima E, Hell M, Ballini R, Gomide F (2010) Evolving fuzzy modeling using participatory learning. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 67–86CrossRefGoogle Scholar
  47. Lin Y, Chang JY, Lin CT (2013) Identification and prediction of dynamic systems using an interactively recurrent self-evolving fuzzy neural network. IEEE Trans Neural Netw Learn Syst 24(2):310–321CrossRefGoogle Scholar
  48. Lippmann R (1991) A critical overview of neural network pattern classifiers. In: Proceedings of the IEEE workshop neural networks and signal processing, pp 266–275Google Scholar
  49. Lughofer E (2008) Extensions of vector quantization for incremental clustering. Pattern Recognit 41(3):995–1011MATHCrossRefGoogle Scholar
  50. Lughofer E (2008) FLEXFIS: a robust incremental learning approach for evolving TS fuzzy models. IEEE Trans Fuzzy Syst 16(6):1393–1410CrossRefGoogle Scholar
  51. Lughofer E (2011) Evolving fuzzy systems—methodologies, advanced concepts and applications. Springer, BerlinMATHCrossRefGoogle Scholar
  52. Lughofer E (2011) On-line incremental feature weighting in evolving fuzzy classifiers. Fuzzy Sets Syst 163(1):1–23MATHMathSciNetCrossRefGoogle Scholar
  53. Lughofer E (2013) On-line assurance of interpretability criteria in evolving fuzzy systems—achievements, new concepts and open issues. Inf Sci 251:22–46CrossRefGoogle Scholar
  54. Lughofer E, Angelov P (2011) Handling drifts and shifts in on-line data streams with evolving fuzzy systems. Appl Soft Comput 11(2):2057–2068CrossRefGoogle Scholar
  55. Lughofer E, Bouchot JL, Shaker A (2011) On-line elimination of local redundancies in evolving fuzzy systems. Evol Syst 2(3):165–187CrossRefGoogle Scholar
  56. Lughofer E, Kindermann S (2010) SparseFIS: data-driven learning of fuzzy systems with sparsity constraints. IEEE Trans Fuzzy Syst 18(2):396–411Google Scholar
  57. Lughofer E, Smith JE, Caleb-Solly P, Tahir M, Eitzinger C, Sannen D, Nuttin M (2009) Human–machine interaction issues in quality control based on on-line image classification. IEEE Trans Syst Man Cybern Part A Syst Hum 39(5):960–971CrossRefGoogle Scholar
  58. Lughofer E, Trawinski B, Trawinski K, Kempa O, Lasota T (2011) On employing fuzzy modeling algorithms for the valuation of residential premises. Inf Sci 181(23):5123–5142CrossRefGoogle Scholar
  59. Macias-Hernandez J, Angelov P (2010) Applications of evolving intelligent systems to the oil and gas industry. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 401–421CrossRefGoogle Scholar
  60. Maciel L, Lemos A, Gomide F, Ballini R (2012) Evolving fuzzy systems for pricing fixed income options. Evol Syst 3(1):5–18CrossRefGoogle Scholar
  61. Mahalanobis PC (1936) On the generalised distance in statistics. Proc Natl Inst Sci India 2(1):49–55MATHMathSciNetGoogle Scholar
  62. Pang S, Ozawa S, Kasabov N (2005) Incremental linear discriminant analysis for classification of data streams. IEEE Trans Syst Men Cybern Part B Cybern 35(5):905–914CrossRefGoogle Scholar
  63. Pedrycz W, Gomide F (2007) Fuzzy systems engineering: toward human-centric computing. Wiley, HobokenCrossRefGoogle Scholar
  64. Pratama M, Anavatti S, Lughofer E (2014) GENEFIS: towards an effective localist network. IEEE Trans Fuzzy Syst 22(3):547–562CrossRefGoogle Scholar
  65. Qin S, Li W, Yue H (2000) Recursive PCA for adaptive process monitoring. J Process Control 10(5):471–486CrossRefGoogle Scholar
  66. Rao R (2013) A novel weighted euclidean distance-based approach. In: Decision making in manufacturing environment using graph theory and fuzzy multiple attribute decision making methods. Springer Series in Manufacturing, New York, pp 159–191Google Scholar
  67. Rong HJ, Sundararajan N, Huang GB, Saratchandran P (2006) Sequential adaptive fuzzy inference system (SAFIS) for nonlinear system identification and prediction. Fuzzy Sets Syst 157(9):1260–1275MATHMathSciNetCrossRefGoogle Scholar
  68. Rubio J (2010) Stability analysis for an on-line evolving neuro-fuzzy recurrent network. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 173–199CrossRefGoogle Scholar
  69. Sannen D, Nuttin M, Smith J, Tahir M, Lughofer E, Eitzinger C (2008) An interactive self-adaptive on-line image classification framework. In: Gasteratos A, Vincze M, Tsotsos J (eds) Proceedings of ICVS 2008, LNCS, vol 5008. Springer, Santorini Island, pp 173–180Google Scholar
  70. Sayed-Mouchaweh M, Lughofer E (2012) Learning in non-stationary environments: methods and applications. Springer, New YorkCrossRefGoogle Scholar
  71. Shilton A, Palaniswami M, Ralph D, Tsoi A (2005) Incremental training of support vector machines. IEEE Trans Neural Netw 16(1):114–131CrossRefGoogle Scholar
  72. Soleimani H, Lucas K, Araabi B (2010) Recursive gathgeva clustering as a basis for evolving neuro-fuzzy modeling. Evol Syst 1(1):59–71CrossRefGoogle Scholar
  73. Sun H, Wang S (2011) Measuring the component overlapping in the Gaussian mixture model. Data Min Knowl Discov 23:479–502MATHMathSciNetCrossRefGoogle Scholar
  74. Tabata K, Kudo MSM (2010) Data compression by volume prototypes for streaming data. Pattern Recognit 43(9):3162–3176MATHCrossRefGoogle Scholar
  75. Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modeling and control. IEEE Trans Syst Man Cybern 15(1):116–132MATHCrossRefGoogle Scholar
  76. Tung S, Quek C, Guan C (2013) eT2FIS: an evolving type-2 neural fuzzy inference system. Inf Sci 220:124–148CrossRefGoogle Scholar
  77. Wang N, Er M, Meng X (2009) A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks. Neurocomputing 72(16–18):3818–3829CrossRefGoogle Scholar
  78. Yager R, Filev D (1994) Approximate clustering via the mountain method. IEEE Trans Syst Man Cybern 24(8):1279–1284CrossRefGoogle Scholar
  79. Yager RR (1990) A model of participatory learning. IEEE Trans Syst Man Cybern 20(5):1229–1234MathSciNetCrossRefGoogle Scholar
  80. Zdsar A, Dovzan D, Skrjanc I (2014) Self-tuning of 2 DOF control based on evolving fuzzy model. Appl Soft Comput 19:403–418CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Edwin Lughofer
    • 1
  • Carlos Cernuda
    • 1
  • Stefan Kindermann
    • 2
  • Mahardhika Pratama
    • 3
  1. 1.Department of Knowledge-Based Mathematical SystemsJohannes Kepler University LinzLinzAustria
  2. 2.Industrial Mathematics InstituteJohannes Kepler University LinzLinzAustria
  3. 3.School of Engineering and Information TechnologyUniversity of New South WalesCanberraAustralia

Personalised recommendations