Skip to main content

Generalized smart evolving fuzzy systems

Abstract

In this paper, we propose a new methodology for learning evolving fuzzy systems (EFS) from data streams in terms of on-line regression/system identification problems. It comes with enhanced dynamic complexity reduction steps, acting on model components and on the input structure and by employing generalized fuzzy rules in arbitrarily rotated position. It is thus termed as Gen-Smart-EFS (GS-EFS), short for generalized smart evolving fuzzy systems. Equipped with a new projection concept for high-dimensional kernels onto one-dimensional fuzzy sets, our approach is able to provide equivalent conventional TS fuzzy systems with axis-parallel rules, thus maintaining interpretability when inferring new query samples. The on-line complexity reduction on rule level integrates a new merging concept based on a combined adjacency–homogeneity relation between two clusters (rules). On input structure level, complexity reduction is motivated by a combined statistical-geometric concept and acts in a smooth and soft manner by incrementally adapting feature weights: features may get smoothly out-weighted over time (\(\rightarrow\) soft on-line dimension reduction) but also may become reactivated at a later stage. Out-weighted features will contribute little to the rule evolution criterion, which prevents the generation of unnecessary rules and reduces over-fitting due to curse of dimensionality. The criterion relies on a newly developed re-scaled Mahalanobis distance measure for assuring monotonicity between feature weights and distance values. Gen-Smart-EFS will be evaluated based on high-dimensional real-world data (streaming) sets and compared with other well-known (evolving) fuzzy systems approaches. The results show improved accuracy with lower rule base complexity as well as smaller rule length when using Gen-Smart-EFS.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Notes

  1. 1.

    http://en.wikipedia.org/wiki/Evolving_intelligent_system.

  2. 2.

    http://archive.ics.uci.edu/ml/.

References

  1. Abonyi J, Babuska R, Szeifert F (2002) Modified Gath-Geva fuzzy clustering for identification of Takagi–Sugeno fuzzy models. IEEE Trans Syst Man Cybern Part B 32(5):612–621

    Article  Google Scholar 

  2. Angelov P (2010) Evolving Takagi–Sugeno fuzzy systems from streaming data, eTS+. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 21–50

    Chapter  Google Scholar 

  3. Angelov P, Filev D (2004) An approach to online identification of Takagi-Sugeno fuzzy models. IEEE Trans Syst Man Cybern Part B Cybern 34(1):484–498

    Article  Google Scholar 

  4. Angelov P, Filev D, Kasabov N (2010) Evolving intelligent systems—methodology and applications. Wiley, New York

    Book  Google Scholar 

  5. Angelov P, Kordon A (2010) Evolving inferential sensors in the chemical process industry. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems—methodology and applications. Wiley, New York, pp 313–336

    Chapter  Google Scholar 

  6. Angelov P, Lughofer E, Zhou X (2008) Evolving fuzzy classifiers using different model architectures. Fuzzy Sets Syst 159(23):3160–3182

    MATH  MathSciNet  Article  Google Scholar 

  7. Angelov P, Sadeghi-Tehran P, Ramezani R (2011) An approach to automatic real-time novelty detection, object identification, and tracking in video streams based on recursive density estimation and evolving Takagi–Sugeno fuzzy systems. Int J Intell Syst 26(3):189–205

    MATH  Article  Google Scholar 

  8. Angelov P, Zhou XW (2006) Evolving fuzzy systems from data streams in real-time. In: 2006 international symposium on evolving fuzzy systems (EFS’06), Ambleside, pp 29–35 (2006)

  9. Babuska R (1998) Fuzzy modeling for control. Kluwer Academic Publishers, Norwell

    Book  Google Scholar 

  10. Backer SD, Scheunders P (2001) Texture segmentation by frequency-sensitive elliptical competitive learning. Image Vis Comput 19(9–10):639–648

    Article  Google Scholar 

  11. Bauer F, Lukas M (2011) Comparing parameter choice methods for regularization of ill-posed problems. Math Comput Simul 81(9):1795–1841

    MATH  MathSciNet  Article  Google Scholar 

  12. Bhattacharyya A (1943) On a measure of divergence between two statistical populations defined by their probability distributions. Bull Calcutta Math Soc 35:99–109

    MATH  MathSciNet  Google Scholar 

  13. Bifet A, Kirkby R (2011) Data stream mining—a practical approach. Tech. rep., Department of Computer Sciences, University of Waikato, Japan

  14. Bouchachia A, Mittermeir R (2006) Towards incremental fuzzy classifiers. Soft Comput 11(2):193–207

    Article  Google Scholar 

  15. Casillas J, Cordon O, Herrera F, Magdalena L (2003) Interpretability issues in fuzzy modeling. Springer, Berlin

    MATH  Book  Google Scholar 

  16. Castro J, Delgado M (1996) Fuzzy systems with defuzzification are universal approximators. IEEE Trans Syst Man Cybern Part B Cybern 26(1):149–152

    Article  Google Scholar 

  17. Chen H, Tino P, Yao X, Rodan A (2014) Learning in the model space for fault diagnosis. IEEE Trans Neural Netw Learn Syst 25(1):124–136

    Article  Google Scholar 

  18. Cheng W, Juang C (2011) An incremental support vector machine-trained TS-type fuzzy system for online classification problems. Fuzzy Sets Syst 163(1):24–44

    MATH  MathSciNet  Article  Google Scholar 

  19. Cohen L, Avrahami-Bakish G, Last M, Kandel A, Kipersztok O (2008) Real-time data mining of non-stationary data streams from sensor networks. Inf Fusion 9(3):344–353

    Article  Google Scholar 

  20. Costa B, Angelov P, Guedes L (2015) Fully unsupervised fault detection and identification based on recursive density estimation and self-evolving cloud-based classifier. Neurocomputing 150(A):289–303

  21. Diehl C, Cauwenberghs G (2003) SVM incremental learning, adaptation and optimization. In: Proceedings of the international joint conference on neural networks, vol 4, pp 2685–2690, Boston (2003)

  22. Djouadi A, Snorrason O, Garber F (1990) The quality of training-sample estimates of the Bhattacharyya coefficient. IEEE Trans Pattern Anal Mach Intell 12(1):92–97

    Article  Google Scholar 

  23. Efendic H, Re LD (2006) Automatic iterative fault diagnosis approach for complex systems. WSEAS Trans Syst 5(2):360–367

    Google Scholar 

  24. Eitzinger C, Heidl W, Lughofer E, Raiser S, Smith J, Tahir M, Sannen D, van Brussel H (2010) Assessment of the influence of adaptive components in trainable surface inspection systems. Mach Vis Appl 21(5):613–626

    Article  Google Scholar 

  25. Gama J (2010) Knowledge discovery from data streams. Chapman & Hall/CRC, Boca Raton

    MATH  Book  Google Scholar 

  26. Gray R (1984) Vector quantization. IEEE ASSP Mag 1(2):4–29

    Article  Google Scholar 

  27. Hametner C, Jakubek S (2013) Local model network identification for online engine modelling. Inf Sci 220:210–225

    Article  Google Scholar 

  28. Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference and prediction, 2nd edn. Springer, New York

    Book  Google Scholar 

  29. Heidl W, Thumfart S, Lughofer E, Eitzinger C, Klement E (2013) Machine learning based analysis of gender differences in visual inspection decision making. Inf Sci 224:62–76

    MathSciNet  Article  Google Scholar 

  30. Hill T, Lewicki P (2007) Statistics: methods and applications. StatSoft, Tulsa

    Google Scholar 

  31. Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65–70

    MATH  MathSciNet  Google Scholar 

  32. Huang G, Saratchandran P, Sundararajan N (2004) An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks. IEEE Trans Syst Man Cybern Part B Cybern 34(6):2284–2292

    Article  Google Scholar 

  33. Iglesias J, Angelov P, Ledezma A, Sanchis A (2010) Evolving classification of agent’s behaviors: a general approach. Evol Syst 1(3):161–172

    Article  Google Scholar 

  34. Jang JS (1993) ANFIS: adaptive-network-based fuzzy inference systems. IEEE Trans Syst Man Cybern 23(3):665–685

    Article  Google Scholar 

  35. Jimenez L, Landgrebe D (1998) Supervised classification in high-dimensional space: geometrical, statistical, and asymptotical properties of multivariate data. IEEE Trans Syst Man Cybern Part C Rev Appl 28(1):39–54

    Article  Google Scholar 

  36. Jin Y (2000) Fuzzy modelling of high dimensional systems: complexity reduction and interpretability improvement. IEEE Trans Fuzzy Syst 8(2):212–221

    Article  Google Scholar 

  37. Klement E, Mesiar R, Pap E (2000) Triangular norms. Kluwer Academic Publishers, Dordrecht

    MATH  Book  Google Scholar 

  38. Kohonen T (1995) Self-organizing maps, 2nd edn. Springer, Berlin

    Book  Google Scholar 

  39. Komijani M, Lucas C, Araabi B, Kalhor A (2012) Introducing evolving Takagi–Sugeno method based on local least squares support vector machine models. Evol Syst 3(2):81–93

    Article  Google Scholar 

  40. Krishnamoorthy K, Mathew T (2009) Statistical tolerance regions: theory, applications, and computation. Wiley, Hoboken

    Book  Google Scholar 

  41. Leite D, Ballini R, Costa P, Gomide F (2012) Evolving fuzzy granular modeling from nonstationary fuzzy data streams. Evol Syst 3(2):65–79

    Article  Google Scholar 

  42. Leite D, Costa P, Gomide F (2012) Interval approach for evolving granular system modeling. In: Sayed-Mouchaweh M, Lughofer E (eds) Learning in non-stationary environments: methods and applications. Springer, New York, pp 271–300

    Chapter  Google Scholar 

  43. Lemos A, Caminhas W, Gomide F (2011) Multivariable Gaussian evolving fuzzy modeling system. IEEE Trans Fuzzy Syst 19(1):91–104

    Article  Google Scholar 

  44. Lemos A, Caminhas W, Gomide F (2013) Adaptive fault detection and diagnosis using an evolving fuzzy classifier. Inf Sci 220:64–85

    Article  Google Scholar 

  45. Liang N, Huang G, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17:1411–1423

    Article  Google Scholar 

  46. Lima E, Hell M, Ballini R, Gomide F (2010) Evolving fuzzy modeling using participatory learning. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 67–86

    Chapter  Google Scholar 

  47. Lin Y, Chang JY, Lin CT (2013) Identification and prediction of dynamic systems using an interactively recurrent self-evolving fuzzy neural network. IEEE Trans Neural Netw Learn Syst 24(2):310–321

    Article  Google Scholar 

  48. Lippmann R (1991) A critical overview of neural network pattern classifiers. In: Proceedings of the IEEE workshop neural networks and signal processing, pp 266–275

  49. Lughofer E (2008) Extensions of vector quantization for incremental clustering. Pattern Recognit 41(3):995–1011

    MATH  Article  Google Scholar 

  50. Lughofer E (2008) FLEXFIS: a robust incremental learning approach for evolving TS fuzzy models. IEEE Trans Fuzzy Syst 16(6):1393–1410

    Article  Google Scholar 

  51. Lughofer E (2011) Evolving fuzzy systems—methodologies, advanced concepts and applications. Springer, Berlin

    MATH  Book  Google Scholar 

  52. Lughofer E (2011) On-line incremental feature weighting in evolving fuzzy classifiers. Fuzzy Sets Syst 163(1):1–23

    MATH  MathSciNet  Article  Google Scholar 

  53. Lughofer E (2013) On-line assurance of interpretability criteria in evolving fuzzy systems—achievements, new concepts and open issues. Inf Sci 251:22–46

    Article  Google Scholar 

  54. Lughofer E, Angelov P (2011) Handling drifts and shifts in on-line data streams with evolving fuzzy systems. Appl Soft Comput 11(2):2057–2068

    Article  Google Scholar 

  55. Lughofer E, Bouchot JL, Shaker A (2011) On-line elimination of local redundancies in evolving fuzzy systems. Evol Syst 2(3):165–187

    Article  Google Scholar 

  56. Lughofer E, Kindermann S (2010) SparseFIS: data-driven learning of fuzzy systems with sparsity constraints. IEEE Trans Fuzzy Syst 18(2):396–411

    Google Scholar 

  57. Lughofer E, Smith JE, Caleb-Solly P, Tahir M, Eitzinger C, Sannen D, Nuttin M (2009) Human–machine interaction issues in quality control based on on-line image classification. IEEE Trans Syst Man Cybern Part A Syst Hum 39(5):960–971

    Article  Google Scholar 

  58. Lughofer E, Trawinski B, Trawinski K, Kempa O, Lasota T (2011) On employing fuzzy modeling algorithms for the valuation of residential premises. Inf Sci 181(23):5123–5142

    Article  Google Scholar 

  59. Macias-Hernandez J, Angelov P (2010) Applications of evolving intelligent systems to the oil and gas industry. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 401–421

    Chapter  Google Scholar 

  60. Maciel L, Lemos A, Gomide F, Ballini R (2012) Evolving fuzzy systems for pricing fixed income options. Evol Syst 3(1):5–18

    Article  Google Scholar 

  61. Mahalanobis PC (1936) On the generalised distance in statistics. Proc Natl Inst Sci India 2(1):49–55

    MATH  MathSciNet  Google Scholar 

  62. Pang S, Ozawa S, Kasabov N (2005) Incremental linear discriminant analysis for classification of data streams. IEEE Trans Syst Men Cybern Part B Cybern 35(5):905–914

    Article  Google Scholar 

  63. Pedrycz W, Gomide F (2007) Fuzzy systems engineering: toward human-centric computing. Wiley, Hoboken

    Book  Google Scholar 

  64. Pratama M, Anavatti S, Lughofer E (2014) GENEFIS: towards an effective localist network. IEEE Trans Fuzzy Syst 22(3):547–562

    Article  Google Scholar 

  65. Qin S, Li W, Yue H (2000) Recursive PCA for adaptive process monitoring. J Process Control 10(5):471–486

    Article  Google Scholar 

  66. Rao R (2013) A novel weighted euclidean distance-based approach. In: Decision making in manufacturing environment using graph theory and fuzzy multiple attribute decision making methods. Springer Series in Manufacturing, New York, pp 159–191

  67. Rong HJ, Sundararajan N, Huang GB, Saratchandran P (2006) Sequential adaptive fuzzy inference system (SAFIS) for nonlinear system identification and prediction. Fuzzy Sets Syst 157(9):1260–1275

    MATH  MathSciNet  Article  Google Scholar 

  68. Rubio J (2010) Stability analysis for an on-line evolving neuro-fuzzy recurrent network. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 173–199

    Chapter  Google Scholar 

  69. Sannen D, Nuttin M, Smith J, Tahir M, Lughofer E, Eitzinger C (2008) An interactive self-adaptive on-line image classification framework. In: Gasteratos A, Vincze M, Tsotsos J (eds) Proceedings of ICVS 2008, LNCS, vol 5008. Springer, Santorini Island, pp 173–180

    Google Scholar 

  70. Sayed-Mouchaweh M, Lughofer E (2012) Learning in non-stationary environments: methods and applications. Springer, New York

    Book  Google Scholar 

  71. Shilton A, Palaniswami M, Ralph D, Tsoi A (2005) Incremental training of support vector machines. IEEE Trans Neural Netw 16(1):114–131

    Article  Google Scholar 

  72. Soleimani H, Lucas K, Araabi B (2010) Recursive gathgeva clustering as a basis for evolving neuro-fuzzy modeling. Evol Syst 1(1):59–71

    Article  Google Scholar 

  73. Sun H, Wang S (2011) Measuring the component overlapping in the Gaussian mixture model. Data Min Knowl Discov 23:479–502

    MATH  MathSciNet  Article  Google Scholar 

  74. Tabata K, Kudo MSM (2010) Data compression by volume prototypes for streaming data. Pattern Recognit 43(9):3162–3176

    MATH  Article  Google Scholar 

  75. Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modeling and control. IEEE Trans Syst Man Cybern 15(1):116–132

    MATH  Article  Google Scholar 

  76. Tung S, Quek C, Guan C (2013) eT2FIS: an evolving type-2 neural fuzzy inference system. Inf Sci 220:124–148

    Article  Google Scholar 

  77. Wang N, Er M, Meng X (2009) A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks. Neurocomputing 72(16–18):3818–3829

    Article  Google Scholar 

  78. Yager R, Filev D (1994) Approximate clustering via the mountain method. IEEE Trans Syst Man Cybern 24(8):1279–1284

    Article  Google Scholar 

  79. Yager RR (1990) A model of participatory learning. IEEE Trans Syst Man Cybern 20(5):1229–1234

    MathSciNet  Article  Google Scholar 

  80. Zdsar A, Dovzan D, Skrjanc I (2014) Self-tuning of 2 DOF control based on evolving fuzzy model. Appl Soft Comput 19:403–418

    Article  Google Scholar 

Download references

Acknowledgments

The first author acknowledges the support of the Austrian COMET-K2 programme of the Linz Center of Mechatronics (LCM), funded by the Austrian federal government and the federal state of Upper Austria. This publication reflects only the authors’ views.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Edwin Lughofer.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lughofer, E., Cernuda, C., Kindermann, S. et al. Generalized smart evolving fuzzy systems. Evolving Systems 6, 269–292 (2015). https://doi.org/10.1007/s12530-015-9132-6

Download citation

Keywords

  • Data stream regression
  • Generalized evolving fuzzy systems (GS-EFS)
  • Rule merging
  • Adjacency–homogeneity relation
  • Soft and smooth on-line dimension reduction
  • Re-scaled Mahalanobis distance measure