Soft Computing

, Volume 21, Issue 22, pp 6811–6824 | Cite as

Bidirectional reservoir networks trained using SVM\(+\) privileged information for manufacturing process modeling

  • Ali Rodan
  • Alaa F. Sheta
  • Hossam Faris
Methodologies and Application


In the last decade, a wide range of machine learning approaches were proposed and experimented to model highly nonlinear manufacturing processes. However, improving the performance of such models is challenging due to the complexity and high dimensionality of the manufacturing processes in general. In this paper, we propose bidirectional echo state reservoir networks (Bi-ESNs) trained using support vector machine privileged information method (SVM\(+\)) to model a winding machine process. The proposed model will be applied, tested and compared to reported models in the literature such as classical ESN with linear regression, ESN with a linear SVM readout, genetic programming, feedfoward neural network with backpropagation, radial basis function network, adaptive neural fuzzy inference system and local linear wavelet neural network. The developed results show that Bi-ESNs trained with SVM\(+\) are promising. It was able to provide better generalization performance compared to other models.


Recurrent neural network Reservoir computing Support vector machines Privileged information Winding machines Engineering process 



The data set used in this research work was collected as part of a project supervised by Dr. M. Kamel at the Aluminum factory in Egypt and analyzed by Dr. Aboabbas Hussain.

Compliance with ethical standards

Conflict of interest

Ali Rodan declares that he has no conflict of interest. Alaa F. Sheta declares that he has no conflict of interest. Hossam Faris declares that he has no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. Abbasi Nozari H, Dehghan Banadaki H, Mokhtare M, Hekmati Vahed S (2012) Intelligent non-linear modelling of an industrial winding process using recurrent local linear neuro-fuzzy networks. J Zhejiang Univ Sci C 13:403–412CrossRefGoogle Scholar
  2. Al-Hiary H, Sheta A, Ayesh A (2008) Identification of a chemical process reactor using soft computing techniques. In: Proceedings of the 2008 international conference on fuzzy systems (FUZZ2008) within the 2008 IEEE world congress on computational intelligence (WCCI2008), Hong Kong, 1–6 June, pp 845–653Google Scholar
  3. Babinec S, Pospíchal J (2011) modular state space of echo state neural network in time series prediction. Comput Inform 30:321–334zbMATHGoogle Scholar
  4. Bastogne T, Noura H, Sibille P, Richard A (1998) Multivariable identification of a winding process by subspace methods for tension control. Control Eng Pract 6(9):1077–1088CrossRefGoogle Scholar
  5. Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5:157–166CrossRefGoogle Scholar
  6. Braatz R, Ogunnaike B, Featherstone A (1996) Identification, estimation and control of sheet and film processes. In: Proceedings of the 13th IFAC world congress. San Francisco, CA, pp 319–324Google Scholar
  7. Buesing L, Schrauwen B, Legenstein R (2010) Connectivity, dynamics and memory in reservoir computing with binary and analog neurons. Neural Comput 22:1272–1311Google Scholar
  8. Chan WC, Cheung KC, Harris CJ (2001) On the modelling of nonlinear dynamic system using support vector neural networks. Eng. Appl. Artif. Intell. 14:105–113CrossRefGoogle Scholar
  9. Chen Y, Yang B, Dong J (2006) Time-series prediction using a local linear wavelet neural network. Neurocomputing 69:449–456CrossRefGoogle Scholar
  10. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines: and other kernel-based learning methods. Cambridge University Press, New YorkCrossRefzbMATHGoogle Scholar
  11. Du H, Zhang N (2008) Time series prediction using evolving radial basis function networks with new encoding scheme. Neurocomputing 71:1388–1400CrossRefGoogle Scholar
  12. Ebler N, Arnason R, Michaelis G, D’Sa N (1993) Tension control: dancer rolls or load cells. IEEE Trans Ind Appl 29(4):727–739CrossRefGoogle Scholar
  13. Faris H, Sheta A, Öznergiz E (2013) Modelling hot rolling manufacturing process using soft computing techniques. Int J Comput Integr Manuf 26(8):762–771CrossRefGoogle Scholar
  14. Faris H, Sheta AF, Öznergiz E (2016) MGP-CC: a hybrid multigene GP-cuckoo search method for hot rolling manufacture process modeling. Syst Sci Control Eng 4:39–49CrossRefGoogle Scholar
  15. Faris H, Sheta AF (2016) A comparison between parametric and non-parametric soft computing approaches to model the temperature of a metal cutting tool. Int J Comput Integr Manuf 29(1):64–75Google Scholar
  16. Fukushima H, Kim T-H, Sugie T (2007) Brief paper: adaptive model predictive control for a class of constrained linear systems based on the comparison model. Automatica 43:301–308MathSciNetCrossRefzbMATHGoogle Scholar
  17. Good J, Roisum DR (2008) Machines, mechanics and measurements. DEStech Publication, LancasterGoogle Scholar
  18. Hailiang H, Zhong W, Xiaohong N, Jing S (2015) Robust decentralized control of web-winding systems without tension sensor. In: Proceedings of the 2015 34th Chinese control conference (CCC), pp 8850–8854, July 2015Google Scholar
  19. Haykin S (1998) Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall PTR, Upper Saddle RiverzbMATHGoogle Scholar
  20. Hoshino I, Maekawa Y, Fujimoto T, Kimura H (1988) Observer-based multivariable control of the aluminum cold tandem mill. Automatica 24(6):741–754MathSciNetCrossRefzbMATHGoogle Scholar
  21. Hussian A, Sheta A, Abdelwahab A (2001) Modeling of a winding machine using non-parametric neural network model. In: WSEAS international conference on scientific computation and soft computing. Athens, Greece, pp 528–553Google Scholar
  22. Hussian A, Sheta A, Kamel M, Telbany M, Abdelwahab A (2000) Modeling of a winding machine using genetic programming. In: Proceedings of the congress on evolutionary computation (CEC2000), vol 82, pp 398–402Google Scholar
  23. Inoue Y, Takanashi K, Miyazawa N, Hyuga Y (1984) Management and control systems in the steel industry. Comput Ind 5(2):143–152 (Special Issue: Computers in Japanese Industry)Google Scholar
  24. Isobe T (1970) Automatic control in the iron and steel industry. Automatica 6(1):111–121CrossRefGoogle Scholar
  25. Jaeger H (2001) The echo state approach to analysing and training recurrent neural networks. Technical report GMD report 148, German National Research Center for Information TechnologyGoogle Scholar
  26. Jang J-S (1993) ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23:665–685CrossRefGoogle Scholar
  27. Jian W, Xu Z, Minghong W, Dehong D (2007) Research on spindle drive control system of high speed winding machine. In: Proceedings of the 8th international conference on electronic measurement and instruments (ICEMI’07), pp 473–477, Aug 2007Google Scholar
  28. Kara Y, Acar Boyacioglu M, Baykan OK (2011) Predicting direction of stock price index movement using artificial neural networks and support vector machines. Expert Syst Appl 38:5311–5319CrossRefGoogle Scholar
  29. Lukoševičius M, Jaeger H (2009) Reservoir computing approaches to recurrent neural network training. Comput Sci Rev 3:127–149CrossRefzbMATHGoogle Scholar
  30. Lukoševičius M, Jaeger H, Schrauwen B (2012) Reservoir computing trends. KI - Künstliche Intelligenz, pp 1–7, 05/2012Google Scholar
  31. Lukoševičius M, Marozas V (2014) Noninvasive fetal qrs detection using an echo state network and dynamic programming. Physiol Meas 35:1685CrossRefGoogle Scholar
  32. Mo W, Wang M, Lin J-S, Zan H, Sun G (2014) Control system based on plc for winding machine. In: Proceedings of the 2014 international symposium on computer, consumer and control (IS3C), pp 74–77, June 2014Google Scholar
  33. Muller K, Smola A, Ratsch G, Scholkopf B, kohlmorgen J, Vapnik V (1997) Predicting rime series with support vector machines. In: Proceedings of the 7th international conference on artificial neural networks (ICANN’97), pp 999–1004Google Scholar
  34. Nozari HA, Banadaki HD, Mokhtare M, Vahed SH (2012) Intelligent non-linear modelling of an industrial winding process using recurrent local linear neuro-fuzzy networks. J Zhejiang Univ Sci C 13(6):403–412CrossRefGoogle Scholar
  35. Parant F, Coeffier C, Iung C (1992) Modeling of a web tension in a continuous annealing line. Iron and Steel Engineer, PittsburghGoogle Scholar
  36. Rodan A, Tiňo P (2010) Simple deterministically constructed recurrent neural networks. In: Fyfe C et al (eds) Intelligent data engineering and automated learning—IDEAL 2010. Springer, pp 267–274Google Scholar
  37. Rodan A, Tino P (2011) Negatively correlated echo state networks. In: ESANN, CiteseerGoogle Scholar
  38. Rumelhart DE, Hinton GE, Williams RJ (1988) Neurocomputing: foundations of research. Ch. Learning internal representations by error propagation. MIT Press, Cambridge, pp 673–695Google Scholar
  39. Scardapane S, Wang D, Panella M (2015) A decentralized training algorithm for echo state networks in distributed big data applications. Neural Netw 78:65–74CrossRefGoogle Scholar
  40. Sheta AF, Faris H, Öznergiz E (2014) Improving production quality of a hot-rolling industrial process via genetic programming model. Int J Comput Appl Technol 49:239–250Google Scholar
  41. Sheta A, Ahmed SE, Faris H (2015) A comparison between regression, artificial neural networks and support vector machines for predicting stock market index. Int J Adv Res Artif Intell IJARAI 4(7):55–63Google Scholar
  42. Sheta A, Al-Hiary H, Braik M (2009) Identification and model predictive controller design of the Tennessee Eastman chemical process using ANN. In: Proceedings of the 2009 international conference on artificial intelligence (ICAI’09), July 13–16, USA, vol 1, pp 25–31Google Scholar
  43. Sievers L, Balas M, von Flotow A (1988) Modeling of web conveyance systems for multivariable control. IEEE Trans Autom Control 33:524–531CrossRefzbMATHGoogle Scholar
  44. Sievers L, Balas M, Flotow AV (1988) Modeling of web conveyance systems for multivariable control. IEEE Trans Autom Control 33(6):524–531CrossRefzbMATHGoogle Scholar
  45. Smola AJ, Scholkopf B (1998) A tutorial on support vector regression. Technical report, Technical report NC2-TR-1998-030, University of London, Royal HollowayGoogle Scholar
  46. Tiňo P, Rodan A (2013) Short term memory in input-driven linear dynamical systems. Neurocomputing 112:58–63CrossRefGoogle Scholar
  47. Vapnik V (1995) The nature of statistical learning theory. Springer, New YorkCrossRefzbMATHGoogle Scholar
  48. Vapnik VN (1999) An overview of statistical learning theory. Trans Neural Netw 10:988–999CrossRefGoogle Scholar
  49. Vapnik V, Izmailov R (2015) Learning using privileged information: similarity control and knowledge transfer. J Mach Learn Res 16:2023–2049MathSciNetzbMATHGoogle Scholar
  50. Vapnik V, Vashist A (2009) A new learning paradigm: learning using privileged information. Neural Netw 22(5–6):544–557CrossRefzbMATHGoogle Scholar
  51. Walker N, Wyatt-Mair G (1995) Sensor signal validation using analytical redundancy for an aluminum cold rolling mill. Control Eng Pract 3(6):753–760CrossRefGoogle Scholar
  52. Wang WJ, Xu ZB, Lu WZ (2003) Determination of the spread parameter in the gaussian kernel for classification and regression. Neurocomputing 55:643–663CrossRefGoogle Scholar
  53. Wang X, Yan Heshan (2015) Optimizing the echo state network with a binary particle swarm optimization algorithm. Knowl Based Syst 86:09Google Scholar
  54. Wei C-H, Wu C-H (2004) A simulator of winding machine controller using LabView environment. In: Control, automation, robotics and vision conference, 2004. ICARCV 2004 8th vol 3, pp 2105–2110, Dec 2004Google Scholar
  55. Zhu GQ, Liu SR, Yu JS (2002) Support vector machine and its applications to function approximation. J East China Univ Sci Technol 5:555–559Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.King Abdullah II School for Information TechnologyThe University of JordanAmmanJordan
  2. 2.Computers and Systems DepartmentElectronics Research Institute (ERI)GizaEgypt

Personalised recommendations