Skip to main content
SpringerLink
Log in

Local Linear Neuro-Fuzzy Models: Fundamentals

  • Chapter
Nonlinear System Identification

  • 3453 Accesses

Abstract

This chapter introduces the fundamental ideas of local linear neuro-fuzzy models. The concept is presented, and the wide variety of existing learning schemes is summarized. The equivalence to Takagi-Sugeno fuzzy systems is analyzed, and the constraints for a proper interpretation in terms of fuzzy logic are outlined. Then the problem of learning is subdivided into two parts: (i) parameters of the local models and (ii) structural parameters of the rule premises. For the relatively simple part (i), the local and global estimation approaches are detailed and compared. For the much more complex part (ii), an overview of proposed learning schemes is given, and a specific algorithm based on incremental axis-orthogonal tree construction is discussed in detail. It is called local linear model tree (LOLIMOT) and will be used extensively throughout this book. Its basic features and settings are treated in this chapter. LOLIMOT allows training a local linear neuro-fuzzy model from data deterministically and without adjusting many fiddle parameters as it is usually the case for most alternative neural network approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Strictly speaking, owing to the existence of an offset term, the LLMs are local affine not local linear. Nevertheless, “local linear models” is the standard terminology in the literature.

  2. 2.

    In this context, “local” refers to the effect of the parameters on the model; there is no relation to the expression “nonlinear local” optimization, where “local” refers to the parameter search space.

  3. 3.

    Note that equal data distribution is not always desirable; see Sect. 14.9.

  4. 4.

    This step is necessary because all validity functions are changed slightly by the division as a consequence of the common normalization denominator in (13.4).

  5. 5.

    Some slow-down effect can be observed because the loss function evaluations in (13.38) and Step 3 becomes more involved as the number of LLMs increases. However, for most applications, the computational demand is dominated by the parameter optimizations, and this slow-down effect can be neglected.

  6. 6.

    At some point the number of training data samples will not suffice to estimate the parameters. Note that because of the regularization effect of the local estimation, this point is far beyond M = 150 where the nominal number of parameters is equal to the number of data samples.

  7. 7.

    Both curves for σ n = 0 are, of course, identical since the training data is equal to the true process behavior.

References

  1. Babuška, R.: Fuzzy Modeling and Identification. Ph.D. thesis, Dept. of Control Engineering, Delft University of Technology, Delft, The Netherlands (1996)

    Google Scholar 

  2. Babuška, R., Setnes, M., Kaymak, U., van Naute Lemke, H.R.: Simplification of fuzzy rule bases. In: European Congress on Intelligent Techniques and Soft Computing (EUFIT), pp. 1115–1119, Aachen, Germany (1996)

    Google Scholar 

  3. Babuška, R., Verbruggen, H.B.: Comparing different methods for premise identification in Sugeno-Takagi models. In: European Congress on Intelligent Techniques and Soft Computing (EUFIT), pp. 1188–1192, Aachen, Germany (1994)

    Google Scholar 

  4. Babuška, R., Verbruggen, H.B.: An overview of fuzzy modeling for control. Control Eng. Pract. 4(11), 1593–1606 (1996)

    Article  Google Scholar 

  5. Babuška, R., Verbruggen, H.B.: Fuzzy set methods for local modelling and identification. In: Murray-Smith, R., Johansen, T.A. (eds.) Multiple Model Approaches to Modelling and Control, chapter 2, pp. 75–100. Taylor & Francis, London (1997)

    Google Scholar 

  6. Bernd, T., Kroll, A., Schwarz, H.: Approximation nichtlinearer dynamischer Prozesse mit optimierten funktionalen Fuzzy-Modellen. In: 7. Workshop “Fuzzy Control” des GMA-UA 1.4.2, Dortmund, Germany (1997)

    Google Scholar 

  7. Billings, S.A., Voon, W.S.F.: Piecewise linear identification of non-linear systems. Int. J. Control 46(1), 215–235 (1987)

    Article  MATH  Google Scholar 

  8. Breiman, L., Stone, C.J., Friedman, J.H., Olshen, R.: Classification and Regression Trees. Chapman & Hall, New York (1984)

    MATH  Google Scholar 

  9. Carpenter, G., Grossberg, S.: The ART of adaptive pattern recognition by a self-organizing neural network. IEEE Comput. 21(3), 77–88 (1988)

    Article  Google Scholar 

  10. Cleveland, W.S., Devlin, S.J., Grosse, E.: Regression by local fitting: methods, properties, and computational algorithms. J. Econ. 37, 87–114 (1996)

    MathSciNet  Google Scholar 

  11. Ernst, S.: Hinging hyperplane trees for approximation and identification. In: IEEE Conference on Decision and Control (CDC), pp. 1261–1277, Tampa, USA (1998)

    Google Scholar 

  12. Fischer, T., Nelles, O.: Merging strategy for local model networks based on the Lolimot algorithm. In: International Conference on Artificial Neural Networks, pp. 153–160. Springer (2014)

    Google Scholar 

  13. Friedman, J.H.: Multivariate adaptive regression splines (with discussion). Ann. Stat. 19(1), 1–141 (1991)

    MATH  Google Scholar 

  14. Halme, A., Visala, A., Zhang, X.-C.: Process modelling using the functional state approach. In: Murray-Smith, R., Johansen, T.A. (eds.) Multiple Model Approaches to Modelling and Control, chapter 4, pp. 121–144. Taylor & Francis, London (1997)

    Google Scholar 

  15. Hathaway, R.J., Bezdek, J.C.: Switching regression models and fuzzy clustering. IEEE Trans. Fuzzy Syst. 1(3), 195–204 (1993)

    Article  Google Scholar 

  16. Hilhorst, R.A.: Supervisory Control of Mode-Switch Processes. Ph.D. thesis, Electrical Engineering Department, University of Twente, Twente, The Netherlands (1992)

    Google Scholar 

  17. Hunt, K.J., Haas, R., Murray-Smith, R.: Extending the functional equivalence of radial basis functions networks and fuzzy inference systems. IEEE Trans. Neural Netw. 7(3), 776–781 (1996)

    Article  Google Scholar 

  18. Isaksson, A.J., Ljung, L., Strömberg, J.-E.: On recursive construction of trees as models of dynamic systems. In: IEEE International Conference on Decision and Control, pp. 1686–1687, Brighton, UK (1991)

    Google Scholar 

  19. Ishibuchi, H., Nozaki, K., Yamamoto, N., Tanaka, H.: Construction of fuzzy classification systems with rectangular fuzzy rules using genetic algorithms. Fuzzy Sets Syst. 65, 237–253 (1994)

    Article  MathSciNet  Google Scholar 

  20. Jacobs, R., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixture of local experts. Neural Comput. 3, 79–87 (1991)

    Article  Google Scholar 

  21. Jang, J.-S.R.: Neuro-Fuzzy Modeling: Architectures, Analyses, and Applications. Ph.D. thesis, EECS Department, Univ. of California at Berkeley, Berkeley, USA (1992)

    Google Scholar 

  22. Jang, J.-S.R.: ANFIS: adaptive-network-based fuzzy inference systems. IEEE Trans. Syst. Man Cybern. 23(3), 665–685 (1993)

    Article  Google Scholar 

  23. Jang, J.-S.R., Sun, C.T., Mizutani, E.: Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence. Prentice Hall, Englewood Cliffs (1997)

    Google Scholar 

  24. Johansen, T.A.: Operating Regime Based Process Modeling and Identification. Ph.D. thesis, Dept. of Engineering Cybernetics, Norwegian Institute of Technology, Trondheim, Norway (1994)

    Google Scholar 

  25. Johansen, T.A.: Identification of non-linear system structure and parameters using regime decomposition. Automatica 31(2), 321–326 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  26. Johansen, T.A., Murray-Smith, R.: The operating regime approach to nonlinear modelling and control. In: Murray-Smith, R., Johansen, T.A. (eds.) Multiple Model Approaches to Modelling and Control, chapter 1, pp. 3–72. Taylor & Francis, London (1997)

    Google Scholar 

  27. Jones, R.D., Coworkers: Nonlinear Adaptive Networks: A Little Theory, a Few Applications. Technical Report 91-273, Los Alamos National Lab., New Mexico (1991)

    Google Scholar 

  28. Jordan, M.I., Jacobs, R.A.: Hierarchical mixtures of experts and the EM algorithm. Neural Comput. 6(2), 181–214 (1994)

    Article  Google Scholar 

  29. Kortmann, P.: Fuzzy-Modelle zur Systemidentifikation. Reihe 8: Mess-, Steuerungs- und Regelungstechnik, Nr. 647. VDI-Verlag, Düsseldorf (1997)

    Google Scholar 

  30. Kroll, A.: Fuzzy-Systeme zur Modellierung und Regelung komplexer technischer Systeme. Reihe 8: Mess-, Steuerungs- und Regelungstechnik, Nr. 612. VDI-Verlag, Düsseldorf (1997)

    Google Scholar 

  31. Ljung, L.: System Identification: Theory for the User, 2nd edn. Prentice Hall, Englewood Cliffs (1999)

    Google Scholar 

  32. Meila, M., Jordan, M.I.: Markov mixtures of experts. In: Murray-Smith, R., Johansen, T.A. (eds.) Multiple Model Approaches to Modelling and Control, chapter 5, pp. 145–166. Taylor & Francis, London (1997)

    Google Scholar 

  33. Murray-Smith, R.: A Local Model Network Approach to Nonlinear Modeling. Ph.D. thesis, University of Strathclyde, Strathclyde, UK (1994)

    Google Scholar 

  34. Murray-Smith, R., Johansen, T.A.: Local learning in local model networks. In: IEE International Conference on Artificial Neural Networks, pp. 40–46 (1995)

    Google Scholar 

  35. Murray-Smith, R., Johansen, T.A.: Local learning in local model networks. In: Murray-Smith, R., Johansen, T.A. (eds.) Multiple Model Approaches to Modelling and Control, chapter 7, pp. 185–210. Taylor & Francis, London (1997)

    Google Scholar 

  36. Nakamori, Y., Ryoke, M.: Identification of fuzzy prediction models through hyperellipsoidal clustering. IEEE Trans. Syst. Man Cybern 24(8), 1153–1173 (1994)

    Article  Google Scholar 

  37. Narendra, K.S., Balakrishnan, J., Ciliz, M.K.: Adaptation and learning using multiple models, switching, and tuning. IEEE Control Syst. 15(3), 37–51 (1995)

    Article  Google Scholar 

  38. Nelles, O.: LOLIMOT – Lokale, lineare Modelle zur Identifikation nichtlinearer, dynamischer Systeme. Automatisierungstechnik (at) 45(4), 163–174 (1997)

    Article  Google Scholar 

  39. Nelles, O.: Nonlinear System Identification with Local Linear Neuro-Fuzzy Models. Ph.D. Thesis, TU Darmstadt, Automatisierungstechnik series. Shaker Verlag, Aachen, Germany (1999)

    Google Scholar 

  40. Nelles, O., Isermann, R.: Basis function networks for interpolation of local linear models. In: IEEE Conference on Decision and Control (CDC), pp. 470–475, Kobe, Japan (1996)

    Google Scholar 

  41. Omohundro, S.M.: Efficient algorithms with neural network behavior. J. Complex Syst. 1, 273–347 (1987)

    MathSciNet  MATH  Google Scholar 

  42. Omohundro, S.M.: Bumptrees for efficient function, constraint and classification learning. In: Lippmann, R.P., Moody, J.E., Touretzky, D.S. (eds.) Advances in Neural Information Processing Systems, pp. 693–699. Morgan Kaufmann, San Francisco (1991)

    Google Scholar 

  43. Park, M.-K., Ji, S.-H., Kim, E.-T., Park, M.: Identification of Takagi-Sugeno fuzzy models via clustering and hough transform. In: Hellendoorn, H., Driankov, D. (eds.) Fuzzy Model Identification: Selected Approaches, chapter 3, pp. 91–161. Springer, Berlin (1997)

    Google Scholar 

  44. Pickhardt, R.: Adaptive Regelung nach einem Multi-Modell-Verfahren. Reihe 8: Mess-, Steuerungs- und Regelungstechnik, Nr. 499. VDI-Verlag, Düsseldorf (1995)

    Google Scholar 

  45. Pickhardt, R.: Adaptive Regelung auf der Basis eines Multi-Modells bei einer Transportregelstrecke für Schüttgüter. Automatisierungstechnik 3, 113–120 (1997)

    Article  Google Scholar 

  46. Pottmann, M., Unbehauen, H., Seborg, D.E.: Application of a general multi-model approach for identification of highly nonlinear processes: a case study. Int. J. Control 57(1), 97–120 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  47. Quinlan, J.: C4.5 Programs for Machine Learning. Morgan Kaufmann Publishers, San Francisico (1993)

    Google Scholar 

  48. Reed, R.: Pruning algorithms: a survey. IEEE Trans. Neural Netw. 4(5), 740–747 (1993)

    Article  Google Scholar 

  49. Sanger, T.D.: A tree-structured adaptive network for function approximation in high-dimensional spaces. IEEE Trans. Neural Netw. 2(2), 285–293 (1991)

    Article  Google Scholar 

  50. Sanger, T.D.: A tree-structured algorithm for reducing computation in networks with separable basis functions. Neural Comput. 3(1), 67–78 (1991)

    Article  Google Scholar 

  51. Strokbro, K., Umberger, D.K., Hertz, J.A.: Exploiting neurons with localized receptive fields to learn chaos. J. Complex Syst. 4(3), 603–622 (1990)

    MATH  Google Scholar 

  52. Sugeno, M., Kang, G.T.: Structure identification of fuzzy model. Fuzzy Sets Syst. 28(1), 15–33 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  53. Sugeno, M., Tanaka, K.: Successive identification of a fuzzy model and its application to prediction of a complex system. Fuzzy Sets Syst. 42, 315–334 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  54. Takagi, T., Sugeno, M.: Fuzzy identification of systems and its application to modeling and control. IEEE Trans. Syst. Man Cybern. 15(1), 116–132 (1985)

    Article  MATH  Google Scholar 

  55. Tanaka, M., Ye, J., Tanino, T.: Identification of nonlinear systems using fuzzy logic and genetic algorithms. In: IFAC Symposium on System Identification, pp. 301–306, Copenhagen, Denmark (1994)

    Google Scholar 

  56. Titterington, D., Smith, A.F.M., Markov, U.E.: Statistical Analysis of Finite Mixture Distributions. Wiley, Chichester (1985)

    Google Scholar 

  57. Wang, L.-X.: Adaptive fuzzy systems and control. design and stability analysis. Prentice Hall, Englewood Cliffs (1994)

    Google Scholar 

  58. Yoshinari, Y., Pedrycz, W., Hirota, K.: Construction of fuzzy models through clustering techniques. Fuzzy Sets Syst. 54, 157–165 (1993)

    Article  MathSciNet  Google Scholar 

  59. Zhang, X.-C., Visala, A., Halme, A., Linko, P.: Functional state modelling approach for bioprocesses: local models for aerobic yeast growth processes. J. Process Control 4, 127–134 (1994)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Siegen, Netphen, Germany

    Oliver Nelles

Authors
  1. Oliver Nelles
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Cite this chapter

Nelles, O. (2020). Local Linear Neuro-Fuzzy Models: Fundamentals. In: Nonlinear System Identification. Springer, Cham. https://doi.org/10.1007/978-3-030-47439-3_13

Download citation

Publish with us

Policies and ethics

Search

Navigation