Evolving Systems

, Volume 2, Issue 3, pp 165–187 | Cite as

On-line elimination of local redundancies in evolving fuzzy systems

Original Paper

Abstract

In this paper, we examine approaches for reducing the complexity of evolving fuzzy systems (EFSs) by eliminating local redundancies during training, evolving the models on on-line data streams. Thus, the complexity reduction steps should support fast incremental single-pass processing steps. In EFSs, such reduction steps are important due to several reasons: (1) originally distinct rules representing distinct local regions in the input/output data space may move together over time and get significantly over-lapping as data samples are filling up the gaps in-between these, (2) two or several fuzzy sets in the fuzzy partitions may become redundant because of projecting high-dimensional clusters onto the single axes, (3) they can be also seen as a first step towards a better readability and interpretability of fuzzy systems, as unnecessary information is discarded and the models are made more transparent. One technique is performing a new rule merging approach directly in the product cluster space using a novel concept for calculating the similarity degree between an updated rule and the remaining ones. Inconsistent rules elicited by comparing the similarity of two redundant rule antecedent parts with the similarity of their consequents are specifically handled in the merging procedure. The second one is operating directly in the fuzzy partition space, where redundant fuzzy sets are merged based on their joint α-cut levels. Redundancy is measured by a novel kernel-based similarity measure. The complexity reduction approaches are evaluated based on high-dimensional noisy real-world measurements and an artificially generated data stream containing 1.2 million samples. Based on this empirical comparison, it will be shown that the novel techniques are (1) fast enough in order to cope with on-line demands and (2) produce fuzzy systems with less structural components while at the same time achieving accuracies similar to EFS not integrating any reduction steps.

Keywords

Evolving fuzzy systems Redundancy elimination Similarity Complexity reduction On-line rule Fuzzy set merging 

References

  1. Abraham W, Robins A (2005) Memory retention—the synaptic stability versus plasticity dilemma. Trends Neurosci 28(2):73–78CrossRefGoogle Scholar
  2. Angelov P (2010) Evolving takagi-sugeno fuzzy systems from streaming data, eTS+. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 21–50Google Scholar
  3. Angelov P, Filev D (2005) Simpl_eTS: A simplified method for learning evolving Takagi-Sugeno fuzzy models. In: Proceedings of FUZZ-IEEE 2005. Reno, Nevada, USA, pp 1068–1073Google Scholar
  4. Angelov P, Kordon A (2010) Evolving inferential sensors in the chemical process industry. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 313–336Google Scholar
  5. Babuska R (1998) Fuzzy modeling for control. Kluwer, NorwellGoogle Scholar
  6. Baturone I, Moreno-Velo F, Gersnoviez A (2006) A cad approach to simplify fuzzy system descriptions. In: Proceedings of the 2006 IEEE international conference on fuzzy systems. Vancouver, pp 2392–2399Google Scholar
  7. Bifet A, Holmes G, Kirkby R, Pfahringer B (2010) MOA: massive online analysis. J Mach Learning Res 11:1601–1604Google Scholar
  8. Breiman L, Friedman J, Stone C, Olshen R (1993) Classification and regression trees. Chapman & Hall, Boca RatonGoogle Scholar
  9. Burger M, Haslinger J, Bodenhofer U, Engl HW (2002) Regularized data-driven construction of fuzzy controllers. J Inverse Ill Posed Probl 10(4):319–344MathSciNetMATHGoogle Scholar
  10. Casillas J, Cordon O, Herrera F, Magdalena L (2003) Interpretability issues in fuzzy modeling. Springer, BerlinGoogle Scholar
  11. Chen M, Linkens D (2004) Rule-base self-generation and simplification for data-driven fuzzy models. Fuzzy Sets Syst 142(2):243–265MathSciNetMATHCrossRefGoogle Scholar
  12. Destercke S, Guillaume S, Charnomordic B (2007) Building an interpretable fuzzy rule base from data using orthogonal least squares—application to a depollution problem. Fuzzy Sets Syst 158(18):2078–2094MathSciNetMATHCrossRefGoogle Scholar
  13. Domingos P, Hulten G (2000) Mining high-speed data streams. In: Proceedings of the sixth ACM SIGKDD international conference on knowledge discovery and data mining. Boston, MA, pp 71–80Google Scholar
  14. Espinosa J, Vandewalle J (2000) Constructing fuzzy models with linguistic intergrity from numerical data—AFRELI algorithm. IEEE Trans Fuzzy Syst 8(5):591–600CrossRefGoogle Scholar
  15. Groißböock W, Lughofer E, Klement E (2004) A comparison of variable selection methods with the main focus on orthogonalization. In: Lopéz-Díaz M, Gil M, Grzegorzewski P, Hryniewicz O, Lawry J (eds) Soft methodology and random information systems. Advances in soft computing. Springer, Berlin, pp 479–486Google Scholar
  16. Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference and prediction, 2nd edn. Springer, BerlinGoogle Scholar
  17. Hathaway R, Bezdek J (1993) Switching regression models and fuzzy clustering. IEEE Trans Fuzzy Syst 1(3):195–204CrossRefGoogle Scholar
  18. Jimenez F, Gomez-Skarmeta AF, Sanchez G, Roubos H, Babuska R (2003) Accurate, transparent and compact fuzzy models by multi-objective evolutionary algorithms. In: Casillas J, Cordón O, Herrera F, Magdalena L (eds) Interpretability issues in fuzzy modeling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 431–451Google Scholar
  19. Kasabov N (2007) Evolving connectionist systems: the knowledge engineering approach, 2nd edn. Springer, LondonGoogle Scholar
  20. Kasabov NK, Song Q (2002) DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction. IEEE Trans Fuzzy Syst 10(2):144–154CrossRefGoogle Scholar
  21. Klement E, Mesiar R, Pap E (2000) Triangular norms. Kluwer, DordrechtGoogle Scholar
  22. Kordon A, Smits G, Kalos A, Jordaan E (2003) Robust soft sensor development using genetic programming. In: Leardi R (ed) Nature-inspired methods in chemometrics, pp 69–108Google Scholar
  23. Kurzhanskiy AA, Varaiya P (2006) Ellipsoidal toolbox. Tech. rep. UCB/EECS-2006-46, EECS Department, University of California, Berkeley. http://code.google.com/p/ellipsoids
  24. Lima E, Hell M, Ballini R, Gomide F (2010) Evolving fuzzy modeling using participatory learning. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 67–86Google Scholar
  25. Lughofer E (2006) Process safety enhancements for data-driven evolving fuzzy models. In: Proceedings of 2nd symposium on evolving fuzzy systems. Lake District, UK, pp 42–48Google Scholar
  26. Lughofer E (2008a) Extensions of vector quantization for incremental clustering. Pattern Recogn 41(3):995–1011MATHCrossRefGoogle Scholar
  27. Lughofer E (2008b) FLEXFIS: a robust incremental learning approach for evolving TS fuzzy models. IEEE Trans Fuzzy Syst 16(6):1393–1410CrossRefGoogle Scholar
  28. Lughofer E (2010) Towards robust evolving fuzzy systems. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 87–126Google Scholar
  29. Lughofer E (2011) Evolving fuzzy systems-Methodologies, advanced concepts and applications. Springer, Berlin. ISBN: 978-3-642-18086-6Google Scholar
  30. Lughofer E, Angelov P (2011) Handling drifts and shifts in on-line data streams with evolving fuzzy systems. Applied Soft Computing 11(2):2057–2068CrossRefGoogle Scholar
  31. Lughofer E, Hüllermeier E, Klement E (2005) Improving the interpretability of data-driven evolving fuzzy systems. In: Proceedings of EUSFLAT 2005. Barcelona, Spain, pp 28–33Google Scholar
  32. Lughofer E, Macian V, Guardiola C, Klement E (2010) Data-driven design of takagi-sugeno fuzzy systems for predicting NOx emissions. In: Hüllermeier E, Kruse R, Hoffmann F (eds) Proc. of the 13th international conference on information processing and management of uncertainty, IPMU 2010, part II (applications), CCIS, vol 81. Springer, Dortmund, pp 1–10Google Scholar
  33. Mikut R, Mäkel J, Gröll L (2005) Interpretability issues in data-based learning of fuzzy systems. Fuzzy Sets Syst 150(2):179–197MATHCrossRefGoogle Scholar
  34. Moser B A similarity measure for images and volumetric data based on Hermann Weyl’s discrepancy. IEEE Trans Pattern Anal Mach Intell (2010). doi:10.1109/TPAMI.2009.50
  35. Nelles O (2001) Nonlinear system identification. Springer, BerlinGoogle Scholar
  36. Oliveira JVD (1999) Semantic constraints for membership function optimization. IEEE Trans Syst Man Cybern Part A Syst Hum 29(1):128–138CrossRefGoogle Scholar
  37. Pang S, Ozawa S, Kasabov N (2005) Incremental linear discriminant analysis for classification of data streams. IEEE Trans Syst Man Cybern Part B Cybern 35(5):905–914CrossRefGoogle Scholar
  38. Qin S, Li W, Yue H (2000) Recursive PCA for adaptive process monitoring. J Process Control 10:471–486CrossRefGoogle Scholar
  39. Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann, San FranciscoGoogle Scholar
  40. Ramos J, Dourado A (2006) Pruning for interpretability of large spanned eTS. In: Proceedings of the 2006 international symposium on evolving fuzzy systems. Ambleside, UK, pp 55–60Google Scholar
  41. Rong HJ, Sundararajan N, Huang GB, Saratchandran P (2006) Sequential adaptive fuzzy inference system (SAFIS) for nonlinear system identification and prediction. Fuzzy Sets Syst 157(9):1260–1275MathSciNetMATHCrossRefGoogle Scholar
  42. Rong HJ, Sundararajan N, Huang GB, Zhao GS (2011) Extended sequential adaptive fuzzy inference system for classification problems. Evol Syst (in press). doi:10.1007/s12530-010-9023-9
  43. Ros L, Sabater A, Thomas F (2002) An ellipsoidal calculus based on propagation and fusion. IEEE Trans Syst Man Cybern Part B Cybern 32(4):430–442CrossRefGoogle Scholar
  44. Setnes M (2003) Simplification and reduction of fuzzy rules. In: Casillas J, Cordón O, Herrera F, Magdalena L (eds) Interpretability issues in fuzzy modeling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 278–302Google Scholar
  45. Setnes M, Babuska R, Kaymak U, Lemke H (1998a) Similarity measures in fuzzy rule base simplification. IEEE Trans Syst Man Cybern Part B Cybern 28(3):376–386CrossRefGoogle Scholar
  46. Setnes M, Babuska R, Verbruggen H (1998b) Complexity reduction in fuzzy modeling. Math Comput Simul 46(5-6):509–518CrossRefGoogle Scholar
  47. Shaker A, Senge R, Hüllermeier E (2011) Evolving fuzzy pattern trees for binary classification on data streams. Inf Sci (in press)Google Scholar
  48. Stone M (1974) Cross-validatory choice and assessment of statistical predictions. J R Stat Soc 36:111–147MATHGoogle Scholar
  49. Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modeling and control. IEEE Trans Syst Man Cybern 15(1):116–132MATHGoogle Scholar
  50. Wang L (1992) Fuzzy systems are universal approximators. In: Proceedings of the 1st IEEE conf. fuzzy systems. San Diego, CA, pp 1163–1169Google Scholar
  51. Wang L, Mendel J (1992) Fuzzy basis functions, universal approximation and orthogonal least-squares learning. IEEE Trans Neural Netw 3(5):807–814CrossRefGoogle Scholar
  52. Yager RR (1990) A model of participatory learning. IEEE Trans Syst Man Cybern 20:1229–1234MathSciNetCrossRefGoogle Scholar
  53. Yen J, Wang L, Gillespie C (1998) Improving the interpretability of TSK fuzzy models by combining global learning and local learning. IEEE Trans Fuzzy Syst 6(4):530–537CrossRefGoogle Scholar
  54. Zhou S, Gan J (2008) Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy systems modelling. Fuzzy Seta Syst 159(23):3091–3131MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Edwin Lughofer
    • 1
  • Jean-Luc Bouchot
    • 1
  • Ammar Shaker
    • 2
  1. 1.Department of Knowledge-Based Mathematical SystemsJohannes Kepler University of LinzLinzAustria
  2. 2.Department of Mathematics and Computer SciencePhilipps-University MarburgMarburgGermany

Personalised recommendations