Applied Intelligence

, Volume 44, Issue 1, pp 149–165 | Cite as

Novel approaches for parameter estimation of local linear models for dynamical system identification

  • Guilherme A. BarretoEmail author
  • Luís Gustavo M. Souza


In this paper we introduce two novel techniques for local linear modeling of dynamical systems. As in the standard approach, we use vector quantization (VQ) algorithms, such as the Self-Organizing Map, to partition the joint input-output space into smaller regions. Then, to each neuron we associate a vector of parameters which must be suitably estimated. The first estimation technique uses the prototypes of the i-th neuron and its K nearest neighbors to build the corresponding local linear model. The second technique builds the i-th local linear model using the data vectors that are mapped into the regions comprised of the Voronoi cells of the i-th neuron and its K nearest neighbors. A comprehensive evaluation of the proposed techniques is carried out for the task of inverse identification of three benchmarking Single Input/Single Output (SISO) dynamical systems. Their performances are compared to those achieved by the Multilayer Perceptron and the Extreme Learning Machine networks. We also evaluate how robust are the proposed techniques with respect to the VQ algorithm used to partition the input-output space. The results show that proposed techniques consistently outperform standard approaches for all evaluated datasets.


Nonlinear system identification Global models Feedforward neural networks Local linear models Self-organizing maps Vector quantization 



The first author thanks CNPq for the financial support through the grant no. 309841/2012-7. The second author thanks FUNCAP and CAPES for the scholarships granted along the development of this research. Both authors thanks NUTEC (Fundação Núcleo de Tecnologia Industrial do Ceará) for providing the laboratory infrastructure for the execution of the computer experiments reported in this paper.


  1. 1.
    Abonyi J (2003) Fuzzy Model Identification for Control. BirkhäuserGoogle Scholar
  2. 2.
    Abonyi J, Nemeth S, Vincze C, Arva P (2003) Process analysis and product quality estimation by self-organizing maps with an application to polyethylene production. Comput Ind 52(3):221–234CrossRefGoogle Scholar
  3. 3.
    Ahalt S, Krishnamurthy A, Cheen P, Melton D (1990) Competitive learning algorithms for vector quantization. Neural Netw 3(3):277–290CrossRefGoogle Scholar
  4. 4.
    Andrášik A, Mészáros A, de Azevedo S (2004) On-line tuning of a neural PID controller based on plant hybrid modeling. Comput Chem Eng 28(8):1499–1509CrossRefGoogle Scholar
  5. 5.
    Azeem MF, Hanmandlu M, Ahmad N (2000) Generalization of adaptive neuro-fuzzy inference systems. IEEE Transactions on Neural Networks 11(6):1332–1346CrossRefGoogle Scholar
  6. 6.
    Babuška R, Verbruggen H (2003) Neuro-fuzzy methods for nonlinear system identification. Annu Rev Control 27:73–85CrossRefGoogle Scholar
  7. 7.
    Barreto GA, Aguayo L (2009). In: Príncipe JC, Miikkulainen R (eds) Time series clustering for anomaly detection using competitive neural networks. Springer, pp 28–36Google Scholar
  8. 8.
    Barreto GA, Araújo AFR (2004) Identification and control of dynamical systems using the self-organizing map. IEEE Transactions on Neural Networks 15(5):1244–1259CrossRefGoogle Scholar
  9. 9.
    Barreto GA, Araújo AFR (2004) Identification and control of dynamical systems using the self-organizing map. IEEE Transactions on Neural Networks 15(5):1244–1259CrossRefGoogle Scholar
  10. 10.
    Barreto GA, Souza LGM (2006) Adaptive filtering with the self-organizing maps: A performance comparison. Neural Netw 19(6):785–798CrossRefzbMATHGoogle Scholar
  11. 11.
    Barreto GA, Araújo AFR, Ritter HJ (2003) Self-organizing feature maps for modeling and control of robotic manipulators. J Intell Robot Syst 36(4):407–450CrossRefzbMATHGoogle Scholar
  12. 12.
    Berglund E, Sitte J (2006) The parameterless self-organizing map algorithm. IEEE Transactions on Neural Networks 17(2):305–316CrossRefGoogle Scholar
  13. 13.
    Billings SA, Voon WSF (1983) Structure detection and model validity tests in the identification of nonlinear systems. IEE Proceedings, Part D, Control Theory and Applications 130(4):193–199CrossRefzbMATHGoogle Scholar
  14. 14.
    Billings SA, Voon WSF (1986) Correlation based model validity tests for nonlinear models. Int J Control 44(1):235–244MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Billings SA, Zhu QM (1994) Nonlinear model validation using correlation tests. Int J Control 60(6):1107–1120MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Bittanti S, Piroddi L (1997) Nonlinear identification and control of a heat exchanger: A neural network approach. J Frankl Inst 334(1):135–153CrossRefGoogle Scholar
  17. 17.
    Chen JQ, Xi YG (1998) Nonlinear system modeling by competitive learning and adaptive fuzzy inference system. IEEE Trans Syst Man Cybern C 28(2):231–238CrossRefGoogle Scholar
  18. 18.
    Chen S, Billings SA, Grant PM (1990) Nonlinear system identification using neural networks. Int J Control 51:1191–1214MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Cho J, Principe J, Erdogmus D, Motter M (2006) Modeling and inverse controller design for an unmanned aerial vehicle based on the self-organizing map. IEEE Transactions on Neural Networks 17(2):445–460CrossRefGoogle Scholar
  20. 20.
    Cho J, Principe J, Erdogmus D, Motter M (2007) Quasi-sliding mode control strategy based on multiple linear models. Neurocomputing 70(4-6):962–974Google Scholar
  21. 21.
    Daosud W, Thitiyasook P, Arpornwichanop A, Kittisupakorn P, Hussain MA (2005) Neural network inverse model-based controller for the control of a steel pickling process. Comput Chem Eng 29(10):2110–2119CrossRefGoogle Scholar
  22. 22.
    Darken C, Moody J (1990) Fast adaptive k-means clustering: Some empirical results. In: Proceedings of the international joint conference on neural networks (IJCNN’90), vol 2, pp 233–238Google Scholar
  23. 23.
    Gan M, Peng H, Chen L (2012) A global-local optimization approach to parameter estimation of RBF-type models. Inf Sci 197:144–160CrossRefGoogle Scholar
  24. 24.
    Göppert J, Rosenstiel W (1993) Topology preserving interpolation in selforganizing maps. In: Proceedings of the NeuroNIMES’93, pp 425–434Google Scholar
  25. 25.
    Göppert J, Rosenstiel W (1995) Topological interpolation in SOM by affine transformations. In: Proceedings of the european symposium on artificial neural networks (ESANN’95), pp 15–20Google Scholar
  26. 26.
    Gregorčič G, Lightbody G (2007) Local model network identification with gaussian processes. IEEE Transactions on Neural Networks 18(5):1404–1423CrossRefGoogle Scholar
  27. 27.
    Gregorčič G, Lightbody G (2008) Nonlinear system identification: from multiple-model networks to gaussian processes. Eng Appl Artif Intell 21(7):1035–1055CrossRefGoogle Scholar
  28. 28.
    Hametner C, Jakubek S (2013) Local model network identification for online engine modelling. Inf Sci 220:210–225CrossRefGoogle Scholar
  29. 29.
    Huang G, Huang GB, Song S, You K (2015) Extreme learning machine: Theory and applications. Neural Netw 61(1):32–48CrossRefGoogle Scholar
  30. 30.
    Huang GB, Zhu QY, Ziew CK (2006) Extreme learning machine: Theory and applications. Neurocomputing 70(1–3):489–501CrossRefGoogle Scholar
  31. 31.
    Hunt KJ, Sbarbaro D, Zbikowski R, Gawthrop P (1992) Neural networks for control systems: a survey. Automatica 28(6):1083–1111MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Hussain MA (1996) Inverse model control strategies using neural networks: Analysis, simulation and on-line implementation. PhD thesis, University of LondonGoogle Scholar
  33. 33.
    Hussain MA, Kershenbaum LS (2000) Implementation of an inverse-model-based control strategy using neural networks on a partially simulated exothermic reactor. Chem Eng Res Des 78(2):299–311CrossRefGoogle Scholar
  34. 34.
    Kim E, Lee H, Park M, Park M (1998) A simply identified Sugeno-type fuzzy model via double clustering. Inf Sci 110(1–2):25–39CrossRefGoogle Scholar
  35. 35.
    Kohonen T (2013) Essentials of the self-organizing map. Neural Netw 37:52–65CrossRefGoogle Scholar
  36. 36.
    Kohonen TK, Oja E, Simula O, Visa A, Kangas J (1996) Engineering applications of the self-organizing map. Proc IEEE 84(10):1358–1384CrossRefGoogle Scholar
  37. 37.
    Li X, Yu W (2002) Dynamic system identification via recurrent multilayer perceptrons. Inf Sci 147 (1–4):45–63CrossRefzbMATHGoogle Scholar
  38. 38.
    Lightbody G, Irwin GW (1997) Nonlinear control structures based on embedded neural system models. IEEE Transactions on Neural Networks 8(3):553–567CrossRefGoogle Scholar
  39. 39.
    Lima CAM, Coelho ALV, Von Zuben FJ (2007) Hybridizing mixtures of experts with support vector machines: Investigation into nonlinear dynamic systems identification. Inf Sci 177(10):2049–2074CrossRefGoogle Scholar
  40. 40.
    Liu J, Djurdjanovic D (2008) Topology preservation and cooperative learning in identification of multiple model systems. IEEE Transactions on Neural Networks 19(12):2065–2072CrossRefGoogle Scholar
  41. 41.
    Ljung L (1999) System Identification: Theory for the user, 2nd edn. Prentice-Hall, Englewood CliffsGoogle Scholar
  42. 42.
    MacQueen J (1967) Some methods for classification and analysis of multivariate observations. In: Le Cam LM, Neyman J (eds) Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol 1, pp 281–297. University of California Press, BerkeleyGoogle Scholar
  43. 43.
    Mu C, Sun C, Yu X (2011) Internal model control based on a novel least square support vector machines for MIMO nonlinear discrete systems. Neural Comput Applic 20(8):1159–1166CrossRefGoogle Scholar
  44. 44.
    Murray-Smith R, Gollee H (1994) A constructive learning algorithm for local model networks. In: IEEE workshop on computer-intensive methods in control and signal processing, pp 21–29Google Scholar
  45. 45.
    Murray-Smith R, Hunt KJ (1995). In: Hunt KJ, Irwin GR, Warwick K (eds) Local model architectures for nonlinear modelling and control. Springer, Neural network engineering in dynamic control systems, pp 61–82Google Scholar
  46. 46.
    Narendra KS (1996) Neural networks for control theory and practice. Proc IEEE 84(10):1385–1406CrossRefGoogle Scholar
  47. 47.
    Narendra KS, Lewis FL (2001) Special issue on neural networks feedback control. Automatica 37(8)Google Scholar
  48. 48.
    Narendra KS, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Transactions on Neural Networks 1(1):4–27CrossRefGoogle Scholar
  49. 49.
    Norgaard M, Ravn O (2000) Neural Networks for Modelling and Control of Dynamic Systems. Springer-Verlag, Hansen LKCrossRefGoogle Scholar
  50. 50.
    Papadakis SE, Kaburlasos VG (2010) Piecewise-linear approximation of non-linear models based on probabilistically/possibilistically interpreted intervals numbers (INs). Inf Sci 180(24):5060–5076CrossRefzbMATHGoogle Scholar
  51. 51.
    Peng H, Nakano K, Shioya H (2007) A comprehensive review for industrial applicability of artificial neural networks. IEEE Trans Control Syst Technol 15(1):130–143CrossRefGoogle Scholar
  52. 52.
    Principe JC, Wang L, Motter MA (1998) Local dynamic modeling with self-organizing maps and applications to nonlinear system identification and control. Proc IEEE 86(11):2240–2258CrossRefGoogle Scholar
  53. 53.
    Rezaee B, Fazel Zarandi M (2010) Data-driven fuzzy modeling for TakagiSugenoKang fuzzy system. Inf Sci 180(2):241–255CrossRefGoogle Scholar
  54. 54.
    Rubio JJ (2009) SOFMLS: Online self-organizing fuzzy modified least-squares network. IEEE Trans Fuzzy Syst 17(6):1296–1309MathSciNetCrossRefGoogle Scholar
  55. 55.
    Shorten R, Murray-Smith R, Bjørgan R, Gollee H (1999) On the interpretation of local models in blended multiple model structures. Int J Control 72(7–8):620–628CrossRefzbMATHGoogle Scholar
  56. 56.
    Sjöberg J, Zhang Q, Ljung L, Benveniste A, Deylon B, Glorennec PY, Hjalmarsson H, Juditsky A (1995) Nonlinear black-box modeling in system identification: A unified overview. Automatica 31(12):1691–1724MathSciNetCrossRefzbMATHGoogle Scholar
  57. 57.
    Soong TT (2004) Fundamentals of Probability and Statistics for Engineers, 1st edn. Wiley, West SussexzbMATHGoogle Scholar
  58. 58.
    Souza LGM, Barreto GA (2010) On building local models for inverse system identification with vector quantization algorithms. Neurocomputing 73(10–12):1993–2005CrossRefGoogle Scholar
  59. 59.
    Takagi T, Sugeno M (1985) Fuzzy identification of systems and its application to modeling and control. IEEE Trans Syst Man Cybern 15(1):116–132CrossRefzbMATHGoogle Scholar
  60. 60.
    Teslić L, Hartmann B, Nelles O, Škrjanc I (2011) Nonlinear system identification by gustafson-kessel fuzzy clustering and supervised local model network learning for the drug absorption spectra process. IEEE Transactions on Neural Networks 22(12):1941–1951CrossRefGoogle Scholar
  61. 61.
    Vasuki A, Vanathi PT (2006) A review of vector quantization techniques. IEEE Potentials 25(4):39–47CrossRefGoogle Scholar
  62. 62.
    Walter J, Ritter H, Schulten K (1990) Non-linear prediction with self-organizing map. In: Proceedings of the IEEE international joint conference on neural networks (IJCNN’90), vol 1, pp 587–592Google Scholar
  63. 63.
    Yu W (2004) Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithm. Inf Sci 158:131–157CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Guilherme A. Barreto
    • 1
    Email author
  • Luís Gustavo M. Souza
    • 2
  1. 1.Department of Teleinformatics Engineering, Av. Mister Hull, S/N, Campus of Pici, Center of TechnologyFederal University of CearáFortalezaBrazil
  2. 2.Department of Electrical Engineering, Av. Ministro Petrônio Portela, S/N - Campus of Ininga, Center of TechnologyFederal University of PiauíTeresinaBrazil

Personalised recommendations