Neural Computing and Applications

, Volume 26, Issue 1, pp 171–186 | Cite as

Balanced simplicity–accuracy neural network model families for system identification

  • Hector M. Romero UgaldeEmail author
  • Jean-Claude Carmona
  • Juan Reyes-Reyes
  • Victor M. Alvarado
  • Christophe Corbier
Original Article


Nonlinear system identification tends to provide highly accurate models these last decades; however, the user remains interested in finding a good balance between high-accuracy models and moderate complexity. In this paper, four balanced accuracy–complexity identification model families are proposed. These models are derived, by selecting different combinations of activation functions in a dedicated neural network design presented in our previous work (Romero-Ugalde et al. in Neurocomputing 101:170–180. doi: 10.1016/j.neucom.2012.08.013, 2013). The neural network, based on a recurrent three-layer architecture, helps to reduce the number of parameters of the model after the training phase without any loss of estimation accuracy. Even if this reduction is achieved by a convenient choice of the activation functions and the initial conditions of the synaptic weights, it nevertheless leads to a wide range of models among the most encountered in the literature. To validate the proposed approach, three different systems are identified: The first one corresponds to the unavoidable Wiener–Hammerstein system proposed in SYSID2009 as a benchmark; the second system is a flexible robot arm; and the third system corresponds to an acoustic duct.


Nonlinear system identification Black box Neural networks Model reduction Estimation quality 

List of symbols

\(J_{u} \in R^{1\times n_b}\)

Input regressor vector

\(J_{\hat{y}} \in R^{1\times n_a}\)

Output regressor vector

\(n_a \in R^{1\times 1}\)

Number of pass outputs of the system

\(n_b \in R^{1\times 1}\)

Number of pass inputs of the system

\(X \in R^{1\times 1}\)

Synaptic weight

\(Z_{b} \in R^{1\times 1}\)

Synaptic weight

\(Z_{a} \in R^{1\times 1}\)

Synaptic weight

\(V_{b_i} \in R^{1\times 1}\)

Synaptic weight

\(V_{a_i} \in R^{1\times 1}\)

Synaptic weight

\(Z_h \in R^{1\times 1}\)

Synaptic weight

\(W_{b_{i}} \in R^{1\times n_b}\)

Synaptic weight

\(W_{a_i} \in R^{1\times n_a}\)

Synaptic weight

\(W_{B} \in R^{1\times n_b}\)

Synaptic weight

\(W_{A} \in R^{1\times n_a}\)

Synaptic weight

\(V_{B} \in R^{1\times 1}\)

Synaptic weight

\(V_{A} \in R^{1\times 1}\)

Synaptic weight

\(Z_H \in R^{1\times 1}\)

Synaptic weight

\(X^* \in R^{1\times 1}\)

Synaptic weight after training

\(Z_{b}^* \in R^{1\times 1}\)

Synaptic weight after training

\(Z_{a}^* \in R^{1\times 1}\)

Synaptic weight after training

\(V_{b_i}^* \in R^{1\times 1}\)

Synaptic weight after training

\(V_{a_i}^* \in R^{1\times 1}\)

Synaptic weight after training

\(Z_h^* \in R^{1\times 1}\)

Synaptic weight after training

\(W_{b_{i}}^* \in R^{1\times n_b}\)

Synaptic weight after training

\(W_{a_i}^* \in R^{1\times n_a}\)

Synaptic weight after training

\(W_{B}^* \in R^{1\times n_b}\)

Synaptic weight after training

\(W_{A}^* \in R^{1\times n_a}\)

Synaptic weight after training

\(V_{B}^* \in R^{1\times 1}\)

Synaptic weight after training

\(V_{A}^* \in R^{1\times 1}\)

Synaptic weight after training

\(Z_H^* \in R^{1\times 1}\)

Synaptic weight after training

\(\varphi _{1}\)

Activation function (linear or nonlinear)

\(\varphi _{2}\)

Activation function (linear or nonlinear)

\(\varphi _{3}\)

Activation function (linear or nonlinear)

\(nn \in R^{1\times 1}\)

Number of neurons


Simulation error

\(\mu _t\)

Mean value of the simulation error


Standard deviation of the error


Root mean square (RMS) of the error


Input of the neural network


Output of the neural network


  1. 1.
    Aadaleesan P, Miglan N, Sharma R, Saha P (2008) Nonlinear system identification using Wiener type Laguerre–Wavelet network model. Chem Eng Sci 63(15):3932–3941. doi: 10.1016/j.ces.2008.04.043 CrossRefGoogle Scholar
  2. 2.
    An SQ, Lu T, Ma Y (2010) Simple adaptive control for siso nonlinear systems using neural network based on genetic algorithm. In: Proceedings of the ninth international conference on machine learning and cybernetics IEEE, Qingdao, ChinaGoogle Scholar
  3. 3.
    Angelov P (2011) Fuzzily connected multimodel systems evolving autonomously from data streams. IEEE Trans Syst Man Cybern Part B Cybern 41(4):898–910. doi: 10.1109/TSMCB.2010.2098866 CrossRefGoogle Scholar
  4. 4.
    Bebis G, Georgiopoulos M (1994) Feed-forward neural networks: why network size is so important. IEEE Potentials 13(4):27–31CrossRefGoogle Scholar
  5. 5.
    Biao L, Qing-chun L, Zhen-hua J, Sheng-fang N (2009) System identification of locomotive diesel engines with autoregressive neural network. In: ICIEA, IEEE, Xi’an, China. doi: 10.1109/ICIEA.2009.5138836
  6. 6.
    Castañeda C, Loukianov A, Sanchez E, Castillo-Toledo B (2013) Real-time torque control using discrete-time recurrent high-order neural networks. Neural Comput Appl 22:1223–1232. doi: 10.1007/s00521-012-0890-9 CrossRefGoogle Scholar
  7. 7.
    Chen R (2011) Reducing network and computation complexities in neural based real-time scheduling scheme. Appl Math Comput 217(13):6379–6389. doi: 10.1016/j.amc.2011.01.014 CrossRefMathSciNetzbMATHGoogle Scholar
  8. 8.
    Cichocki A, Unbehauen R (1993) Neural networks for optimization and signal processing, 1st edn. John Wiley and Sons Ltd, Baffins Lane, Chichester, West Sussex PO19 IUD, EnglandGoogle Scholar
  9. 9.
    Coelho L, Wicthoff M (2009) Nonlinear identification using a b-spline neural network and chaotic immune approaches. Mech Syst Signal Process 23(8):2418–2434. doi: 10.1016/j.ymssp.2009.01.013 CrossRefGoogle Scholar
  10. 10.
    Curteanu S, Cartwright H (2011) Neural networks applied in chemistry. I. Determination of the optimal topology of multilayer perceptron neural networks. J Chemom 25:527–549. doi: 10.1002/cem.1401 CrossRefGoogle Scholar
  11. 11.
    de Jesus Rubio J (2014) Fuzzy slopes model of nonlinear systems with sparse data. Soft Comput. doi: 10.1007/s00500-014-1289-6 Google Scholar
  12. 12.
    de Jesus Rubio J (2014) Evolving intelligent algorithms for the modelling of brain and eye signals. Appl Soft Comput 14(part B):259–268. doi: 10.1016/j.asoc.2013.07.023 Google Scholar
  13. 13.
    Endisch C, Stolze P, Endisch P, Hackl C, Kennel R (2009) Levenberg–Marquardt-based obs algorithm using adaptive pruning interval for system identification with dynamic neural networks. In: International conference on systems, man, and cybernetics, IEEE, San Antonio, Texas, USAGoogle Scholar
  14. 14.
    Farivar F, Shoorehdeli MA, Teshnehlab M (2012) An interdisciplinary overview and intelligent control of human prosthetic eye movements system for the emotional support by a huggable pet-type robot from a biomechatronical viewpoint. J Frankl Inst 347(7):2243–2267. doi: 10.1016/j.jfranklin.2011.04.014 CrossRefMathSciNetGoogle Scholar
  15. 15.
    Ge H, Du W, Qian F, Liang Y (2009) Identification and control of nonlinear systems by a time-delay recurrent neural network. Neurocomputing 72:2857–2864. doi: 10.1016/j.neucom.2008.06.030 CrossRefGoogle Scholar
  16. 16.
    Ge H, Qian F, Liang Y, Du W, Wang L (2008) Identification and control of nonlinear systems by a dissimilation particle swarm optimization-based elman neural network. Nonlinear Anal Real World Appl 9(4):1345–1360. doi: 10.1016/j.nonrwa.2007.03.008 CrossRefMathSciNetzbMATHGoogle Scholar
  17. 17.
    Goh CK, Teoh EJ, Tan KC (2008) Hybrid multiobjective evolutionary design for artificial neural networks. IEEE Trans Neural Netw 19(9):1531–1548CrossRefGoogle Scholar
  18. 18.
    Han X, Xie W, Fu Z, Luo W (2011) Nonlinear systems identification using dynamic multi-time scale neural networks. Neurocomputing 74(17):3428–3439CrossRefGoogle Scholar
  19. 19.
    Hangos K, Bokor J, Szederknyi G (2004) Analysis and control of nonlinear process systems. Springer, BerlinzbMATHGoogle Scholar
  20. 20.
    Hsu CF (2009) Adaptive recurrent neural network control using a structure adaptation algorithm. Neural Comput Appl 18:115–125. doi: 10.1007/s00521-007-0164-0 CrossRefGoogle Scholar
  21. 21.
    Isermann R, Munchhof M (2011) Identification of dynamic systems. An introduction with applications. Springer, BerlinCrossRefGoogle Scholar
  22. 22.
    de Jesus Rubio J, Pérez Cruz JH (2014) Evolving intelligent system for the modelling of nonlinear systems with dead-zone input. Appl Soft Comput 14(Part B):289–304. doi: 10.1016/j.asoc.2013.03.018 CrossRefGoogle Scholar
  23. 23.
    Khalaj G, Yoozbashizadeh H, Khodabandeh A, Nazari A (2013) Artificial neural network to predict the effect of heat treatments on vickers microhardness of low-carbon nb microalloyed steels. Neural Comput Appl 22(5):879–888. doi: 10.1007/s00521-011-0779-z
  24. 24.
    Leite D, Costa P, Gomide F (2013) Evolving granular neural networks from fuzzy data streams. Neural Netw 38:1–16. doi: 10.1016/j.neunet.2012.10.006 CrossRefGoogle Scholar
  25. 25.
    Lemos A, Caminhas W, Gomide F (2011) Multivariable gaussian evolving fuzzy modeling system. IEEE Trans Fuzzy Syst 19(1):91–104. doi: 10.1109/TFUZZ.2010.2087381 CrossRefGoogle Scholar
  26. 26.
    Ljung L (1999) System identification theory for the user. PTR Prentice Hall, Upper Saddle River, NJ 07458Google Scholar
  27. 27.
    Loghmanian S, Jamaluddin H, Ahmad R, Yusof R, Khalid M (2012) Structure optimization of neural network for dynamic system modeling using multi-objective genetic algorithm. Neural Comput Appl 21(6):1281–1295. doi: 10.1007/s00521-011-0560-3
  28. 28.
    Lughofer E (2013) On-line assurance of interpretability criteria in evolving fuzzy systems. Achievements, new concepts and open issues. Inf Sci 251:22–46. doi: 10.1016/j.ins.2013.07.002 CrossRefGoogle Scholar
  29. 29.
    Majhi B, Panda G (2011) Robust identification of nonlinear complex systems using low complexity ANN and particle swarm optimization technique. Expert Syst Appl 38(1):321–333. doi: 10.1016/j.eswa.2010.06.070 CrossRefGoogle Scholar
  30. 30.
    Noorgard M, Ravn O, Poulsen NK, Hansen LK (2000) Neural networks for modelling and control of dynamic systems, 1st edn. Springer, BerlinCrossRefGoogle Scholar
  31. 31.
    Ordon̂ez FJ, Iglesias JA, de Toledo P, Ledezma A, Sanchis A (2013) Online activity recognition using evolving classifiers. Expert Syst Appl 40:1248–1255. doi: 10.1016/j.eswa.2012.08.066 CrossRefGoogle Scholar
  32. 32.
    Peralta-Donate J, Li X, Gutierrez-Sanchez G, Sanchis de Miguel A (2013) Time series forecasting by evolving artificial neural networks with genetic algorithms, differential evolution and estimation of distribution algorithm. Neural Comput Appl 22:11–20. doi: 10.1007/s00521-011-0741-0 CrossRefGoogle Scholar
  33. 33.
    Paduart J, Lauwers L, Pintelon R, Schoukens J (2009) Identification of a wiener-hammerstein system using the polynomial nonlinear state space approach. In: Proceedings of the 15th IFAC symposium on system identification, Saint-Malo, France, pp 1080–1085Google Scholar
  34. 34.
    Petre E, Selisteanu D, Sendrescu D, Ionete C (2010) Neural networks-based adaptive control for a class of nonlinear bioprocesses. Neural Comput Appl 19:169–178. doi: 10.1007/s00521-009-0284-9 CrossRefGoogle Scholar
  35. 35.
    Pratama M, Anavatti SG, Angelov PP, Lughofer E (2014) PANFIS: a novel incremental learning machine. IEEE Trans Neural Netw Learn Syst 25(1):55–68. doi: 10.1109/TNNLS.2013.2271933 CrossRefGoogle Scholar
  36. 36.
    Romero-Ugalde HM, Carmona JC, Alvarado VM, Reyes-Reyes J (2013) Neural network design and model reduction approach for black box nonlinear system identification with reduced number of parameters. Neurocomputing 101:170–180. doi: 10.1016/j.neucom.2012.08.013 CrossRefGoogle Scholar
  37. 37.
    Sahnoun MA, Ugalde HMR, Carmona JC, Gomand J (2013) Maximum power point tracking using p&o control optimized by a neural network approach: a good compromise between accuracy and complexity. Energy Procedia 42:650–659. doi: 10.1016/j.egypro.2013.11.067 CrossRefGoogle Scholar
  38. 38.
    Sayah S, Hamouda A (2013) A hybrid differential evolution algorithm based on particle swarm optimization for nonconvex economic dispatch problems. Appl Soft Comput 13:1608–1619. doi: 10.1016/j.asoc.2012.12.014 CrossRefGoogle Scholar
  39. 39.
    Schoukens J, Suykens J, Ljung L (2009) Wiener-Hammerstein benchmark. In: Proceedings of the 15th IFAC symposium on system identification, Saint-Malo, France, pp 1086–1091Google Scholar
  40. 40.
    Subudhi B, Jenab D (2011) A differential evolution based neural network approach to nonlinear system identification. Appl Soft Comput 11(1):861–871. doi: 10.1016/j.asoc.2010.01.006 CrossRefGoogle Scholar
  41. 41.
    Tzeng S (2010) Design of fuzzy wavelet neural networks using the GA approach for function approximation and system identification. Fuzzy Sets Syst 161(19):2585–2596. doi: 10.1016/j.fss.2010.06.002 CrossRefMathSciNetGoogle Scholar
  42. 42.
    Van Mulders A, Schoukens J, Volckaert M, Diehl M (2009) Two nonlinear optimization methods for black box identification compared. In: Proceedings of the 15th IFAC symposium on system identification, Saint-Malo, France, pp 1086–1091Google Scholar
  43. 43.
    Wang X, Syrmos V (2007) Nonlinear system identification and fault detection using hierarchical clustering analysis and local linear models. In: Mediterranean conference on control and automation, Greece, Athens, pp 1–6Google Scholar
  44. 44.
    Witters M, Swevers J (2010) Black-box model identification for a continuously variable, electro-hydraulic semi-active damper. Mech Syst Signal Process 24(1):4–18. doi: 10.1016/j.ymssp.2009.03.013 CrossRefGoogle Scholar
  45. 45.
    Xie W, Zhu Y, Zhao Z, Wong Y (2009) Nonlinear system identification using optimized dynamic neural network. Neurocomputing 72(13–15):3277–3287. doi: 10.1016/j.neucom.2009.02.004 CrossRefGoogle Scholar
  46. 46.
    Yan Z, Xiuxia L, Peng Y, Zengqiang C, Zhuzhi Y (2009) Modeling and control of nonlinear discrete-time systems based on compound neural networks. Chin J Chem Eng 17(3):454–459. doi: 10.1016/S1004-9541(08 CrossRefGoogle Scholar
  47. 47.
    Yu W (2006) Multiple recurrent neural networks for stable adaptive control. Neurocomputing 70(1–3):430–444. doi: 10.1016/j.neucom.2005.12.122 CrossRefGoogle Scholar
  48. 48.
    Yu W, Li X (2004) Fuzzy identification using fuzzy neural networks with stable learning algorithms. IEEE Trans Fuzzy Syst 12(3):411–420. doi: 10.1109/TFUZZ.2004.825067 CrossRefGoogle Scholar
  49. 49.
    Yu W, Morales A (2004) Gasoline blending system modeling via static and dynamic neural networks. Int J Model Simul 24(3):151–160Google Scholar
  50. 50.
    Yu W, Rodriguez FO, Moreno-Armendariz MA (2008) Hierarchical fuzzy CMAC for nonlinear systems modeling. IEEE Trans Fuzzy Syst 16(5):1302–1314. doi: 10.1109/TFUZZ.2008.926579 CrossRefGoogle Scholar
  51. 51.
    Zhang H, Wu W, Yao M (2012) Boundedness and convergence of batch back-propagation algorithm with penalty for feedforward neural networks. Neurocomputing 89:141–146. doi: 10.1016/j.neucom.2012.02.029 CrossRefGoogle Scholar
  52. 52.
    Zhang J, Zhu Q, Wu X, Li Y (2013) A generalized indirect adaptive neural networks backstepping control procedure for a class of non-affine nonlinear systems with pure-feedback prototype. Neurocomputing 21(9):131–139. doi: 10.1016/j.neucom.2013.04.015 Google Scholar
  53. 53.
    Zhang Z, Qiao J (2010) A node pruning algorithm for feedforward neural network based on neural complexity. In: International conference on intelligent control and information processing. IEEE, Dalian, China, pp 406–410Google Scholar
  54. 54.
    Zhao H, Zeng X, He Z (2011) Low-complexity nonlinear adaptive filter based on a pipelined bilinear recurrent neural network. IEEE Trans Neural Netw 22(9):1494–1507CrossRefGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2014

Authors and Affiliations

  • Hector M. Romero Ugalde
    • 1
    Email author
  • Jean-Claude Carmona
    • 2
  • Juan Reyes-Reyes
    • 3
  • Victor M. Alvarado
    • 3
  • Christophe Corbier
    • 4
  1. 1.Laboratoire Traitement du Signal et de l’Image, LTSI, Université de Rennes 1INSERM U1099RennesFrance
  2. 2.Laboratoire des Sciences de l’Information et des Systemes, UMR CNRS 7296ENSAMAix en ProvenceFrance
  3. 3.Centro Nacional de Investigacion y Desarrollo Tecnologico, CENIDETCuernavacaMexico
  4. 4.LASPI, F-42334 IUT de RoanneUniversité de Saint Etienne, Jean MonnetRoanneFrance

Personalised recommendations