Skip to main content
SpringerLink
Log in

Top–Down Sparse Fuzzy Regression Modeling from Data with Improved Coverage

  • Published:
International Journal of Fuzzy Systems Aims and scope Submit manuscript

Abstract

We propose a new fuzzy modeling algorithm from data for regression problems. It acts in a top–down manner by allowing the user to specify an upper number of allowed rules in the rule base which is sparsed out with the usage of an iterative constrained numerical optimization procedure. It is based on the combination of the least squares error and the sum of rule weights over all rules to achieve minimal error with lowest possible number of significantly active rules. Two major novel concepts are integrated into the optimization process: the first respects a minimal coverage degree of the sample space in order to approach \(\epsilon \)-completeness of the rule base (an important interpretability criterion) and the second optimizes the positioning and ranges of influence of the rules, which is done synchronously to the optimization of the rule weights within an intervened, homogeneous procedure. Based on empirical results achieved for several high-dimensional (partially noisy) data sets, it can be shown that our advanced, intervened optimization yields fuzzy systems with a better coverage and a higher degree of \(\epsilon \)-completeness compared to the fuzzy models achieved by related data-driven fuzzy modeling methods. This is even achieved with a significantly lower or at least equal number of rules and with a similar model error on separate validation data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://archive.ics.uci.edu/ml/.

  2. http://eigentaste.berkeley.edu/dataset/.

  3. http://www.dcsc.tudelft.nl/~rbabuska/fmid-v40.zip.

  4. http://www.dcsc.tudelft.nl/~rbabuska/fmid-v40.zip.

References

  1. Akerkar, R., Sajja, P.: Knowledge-Based Systems. Jones & Bartlett Learning, Sudbury (2009)

    Google Scholar 

  2. Babuska, R.: Fuzzy Modeling for Control. Kluwer Academic Publishers, Norwell (1998)

    Book  Google Scholar 

  3. Castro, J., Delgado, M.: Fuzzy systems with defuzzification are universal approximators. IEEE Trans. Syst. Man Cybern. Part B: Cybern. 26(1), 149–152 (1996)

    Article  Google Scholar 

  4. Celikyilmaz, A., Türksen, I.: Modeling Uncertainty with Fuzzy Logic: With Recent Theory and Applications. Springer, Berlin (2009)

    Book  MATH  Google Scholar 

  5. Cernuda, C., Lughofer, E., Röder, T., Märzinger, W., Reischer, T., Pawliczek, M., Brandstetter, M.: Self-adaptive non-linear methods for improved multivariate calibration in chemical processes. Lenzing. Ber. 92, 12–32 (2015)

    Google Scholar 

  6. Chiu, S.: Fuzzy model identification based on cluster estimation. J. Intell. Fuzzy Syst. 2(3), 267–278 (1994)

    Google Scholar 

  7. Cohen, A., Dahmen, W., DeVore, R.: Compressed sensing and best \(k\)-term approximation. J. Am. Math. Soc. 22(1), 211–231 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  8. Cordon, O., Gomide, F., Herrera, F., Hoffmann, F., Magdalena, L.: Ten years of genetic fuzzy systems: current framework and new trends. Fuzzy Sets Syst. 141(1), 5–31 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  9. Daubechies, I., Defrise, M., Mol, C.D.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57(11), 1413–1457 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  10. Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall Series in Computational Mathematics, Englewood Cliffs (1983)

  11. Fletcher, R.: Practical Methods of Optimization. Wiley, New York (2000)

    Book  MATH  Google Scholar 

  12. Gacto, M., Alcala, R., Herrera, F.: Interpretability of linguistic fuzzy rule-based systems: an overview of interpretability measures. Inf. Sci. 181(20), 4340–4360 (2011)

    Article  Google Scholar 

  13. Gray, R.: Vector quantization. IEEE ASSP Mag. 1(2), 4–29 (1984)

    Article  Google Scholar 

  14. Gustafson, D., Kessel, W.: Fuzzy clustering with a fuzzy covariance matrix. In: Proceedings of the IEEE CDC Conference, pp. 761–766. San Diego, CA (1979)

  15. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference and Prediction, 2nd edn. Springer, New York, Berlin Heidelberg (2009)

    Book  MATH  Google Scholar 

  16. Hensel, A., Spittel, T.: Kraft- und Arbeitsbedarf bildsamer Formgebungsverfahren. VEB Deutscher Verlag für Grundstoffindustrie (1978)

  17. Iglesias, J., Angelov, P., Ledezma, A., Sanchis, A.: Evolving classification of agent’s behaviors: a general approach. Evol. Syst. 1(3), 161–172 (2010)

    Article  Google Scholar 

  18. Iglesias, J., Tiemblo, A., Ledezma, A., Sanchis, A.: Web news mining in an evolving framework. Inf. Fusion 28, 90–98 (2016)

    Article  Google Scholar 

  19. J. Casillas, F.H., Pereza, R., Jesus, M.D., Villar, P.: Special issue on genetic fuzzy systems and the interpretability-accuracy trade-off. Int. J. Approx. Reason. 44(1), 1–3 (2007)

    Article  MathSciNet  Google Scholar 

  20. Klement, E., Mesiar, R., Pap, E.: Triangular Norms. Kluwer Academic Publishers, Dordrecht, Norwell, New York, London (2000)

    Book  MATH  Google Scholar 

  21. Lughofer, E.: FLEXFIS: a robust incremental learning approach for evolving TS fuzzy models. IEEE Trans. Fuzzy Syst. 16(6), 1393–1410 (2008)

    Article  Google Scholar 

  22. Lughofer, E.: Evolving Fuzzy Systems—Methodologies, Advanced Concepts and Applications. Springer, Berlin Heidelberg (2011)

    Book  MATH  Google Scholar 

  23. Lughofer, E.: On-line assurance of interpretability criteria in evolving fuzzy systems—achievements, new concepts and open issues. Inf. Sci. 251, 22–46 (2013)

    Article  MathSciNet  Google Scholar 

  24. Lughofer, E., Cernuda, C., Kindermann, S., Pratama, M.: Generalized smart evolving fuzzy systems. Evol. Syst. 6(4), 269–292 (2015)

    Article  Google Scholar 

  25. Lughofer, E., Kindermann, S.: SparseFIS: data-driven learning of fuzzy systems with sparsity constraints. IEEE Trans. Fuzzy Syst. 18(2), 396–411 (2010)

    Google Scholar 

  26. Lughofer, E., Macian, V., Guardiola, C., Klement, E.: Identifying static and dynamic prediction models for nox emissions with evolving fuzzy systems. Appl. Soft Comput. 11(2), 2487–2500 (2011)

    Article  Google Scholar 

  27. Lughofer, E., Smith, J.E., Caleb-Solly, P., Tahir, M., Eitzinger, C., Sannen, D., Nuttin, M.: Human-machine interaction issues in quality control based on on-line image classification. IEEE Trans. Syst. Man Cybern. Part A Syst. Humans 39(5), 960–971 (2009)

    Article  Google Scholar 

  28. Lughofer, E., Trawinski, B., Trawinski, K., Kempa, O., Lasota, T.: On employing fuzzy modeling algorithms for the valuation of residential premises. Inf. Sci. 181(23), 5123–5142 (2011)

    Article  Google Scholar 

  29. Lughofer, E., Weigl, E., Heidl, W., Eitzinger, C., Radauer, T.: Integrating new classes on the fly in evolving fuzzy classifier designs and its application in visual inspection. Appl. Soft Comput. 35, 558–582 (2015)

    Article  Google Scholar 

  30. Marquardt, D.: An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11(2), 431–441 (1963)

    Article  MathSciNet  MATH  Google Scholar 

  31. Nelles, O.: Nonlinear Syst. Identif. Springer, Berlin (2001)

    Book  Google Scholar 

  32. Nguyen, H., Sugeno, M., Tong, R., Yager, R.: Theor. Asp. Fuzzy Control. Wiley, New York (1995)

    Google Scholar 

  33. Oliveira, J.V.D., Pedrycz, W.: Advances in Fuzzy Clustering and its Applications. Wiley, Hoboken (2007)

    Book  Google Scholar 

  34. Pal, N., Chakraborty, D.: Mountain and subtractive clustering method: improvement and generalizations. Int. J. Intell. Syst. 15(4), 329–341 (2000)

    Article  MATH  Google Scholar 

  35. Pedrycz, W., Gomide, F.: Fuzzy Systems Engineering: Toward Human-Centric Computing. Wiley, Hoboken (2007)

    Book  Google Scholar 

  36. Pratama, M., Anavatti, S., Angelov, P., Lughofer, E.: PANFIS: a novel incremental learning machine. IEEE Trans. Neural Netw. Learn. Syst. 25(1), 55–68 (2014)

    Article  Google Scholar 

  37. Pratama, M., Anavatti, S., Garret, M., Lughofer, E.: Online identification of complex multi-input-multi-output system based on generic evolving neuro-fuzzy inference system. In: Proceedings of the IEEE EAIS 2013 workshop (SSCI 2013 conference), pp. 106–113. Singapore (2013)

  38. Rong, H.J., Sundararajan, N., Huang, G.B., Saratchandran, P.: Sequential adaptive fuzzy inference system (SAFIS) for nonlinear system identification and prediction. Fuzzy Sets Syst. 157(9), 1260–1275 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  39. Schaffer, C.: Overfitting avoidance as bias. Mach. Learn. 10(2), 153–178 (1993)

    Google Scholar 

  40. Serdio, F., Lughofer, E., Pichler, K., Pichler, M., Buchegger, T., Efendic, H.: Fault detection in multi-sensor networks based on multivariate time-series models and orthogonal transformations. Inf. Fusion 20, 272–291 (2014)

    Article  Google Scholar 

  41. Siler, W., Buckley, J.: Fuzzy Expert Systems and Fuzzy Reasoning: Theory and Applications. Wiley, Chichester, West Sussex (2005)

    MATH  Google Scholar 

  42. Takagi, T., Sugeno, M.: Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst. Man Cybern. 15(1), 116–132 (1985)

    Article  MATH  Google Scholar 

  43. Tikhonov, A., Arsenin, V.: Solutions of Ill-Posed Problems. Winston & Sons, Washington (1977)

    MATH  Google Scholar 

  44. Vetterlein, T., Ciabattoni, A.: On the (fuzzy) logical content of cadiag-2. Fuzzy Sets and Syst. 161, 1941–1958 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  45. Vetterlein, T., Mandl, H., Adlassnig, K.P.: Fuzzy arden syntax: a fuzzy programming language for medicine. Artif. Intell. Med. 49, 1–10 (2010)

    Article  Google Scholar 

  46. Zhou, S., Gan, J.: Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy systems modelling. Fuzzy Sets Syst. 159(23), 3091–3131 (2008)

    Article  MathSciNet  Google Scholar 

  47. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. Royal Stat. Soc, Series B 301–320 (2005)

Download references

Acknowledgements

The first author acknowledges the support of the Austrian COMET-K2 programme of the Linz Center of Mechatronics (LCM), funded by the Austrian federal government and the federal state of Upper Austria, and the support of the COMET Project ’Heuristic Optimization in Production and Logistics’ (HOPL), #843532 funded by the Austrian Research Promotion Agency (FFG). This publication reflects only the authors’ views.

Author information

Authors and Affiliations

  1. Department of Knowledge-Based Mathematical Systems, Johannes Kepler University Linz, Linz, Austria

    Edwin Lughofer

  2. Industrial Mathematics Institute, Johannes Kepler University Linz, Linz, Austria

    Stefan Kindermann

  3. Latrobe University, Melbourne, Victoria, Australia

    Mahardhika Pratama

  4. Seccion de Estudios de Posgrado e Investigacion, Esime Azcapotzalco, Instituto Politecnico Nacional, Av. de las Granjas, Mexico City, Mexico

    Jose de Jesus Rubio

Authors
  1. Edwin Lughofer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefan Kindermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mahardhika Pratama
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jose de Jesus Rubio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Edwin Lughofer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lughofer, E., Kindermann, S., Pratama, M. et al. Top–Down Sparse Fuzzy Regression Modeling from Data with Improved Coverage. Int. J. Fuzzy Syst. 19, 1645–1658 (2017). https://doi.org/10.1007/s40815-016-0271-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40815-016-0271-0

Keywords

Search

Navigation