# Bidirectional reservoir networks trained using SVM\(+\) privileged information for manufacturing process modeling

- 237 Downloads
- 2 Citations

## Abstract

In the last decade, a wide range of machine learning approaches were proposed and experimented to model highly nonlinear manufacturing processes. However, improving the performance of such models is challenging due to the complexity and high dimensionality of the manufacturing processes in general. In this paper, we propose bidirectional echo state reservoir networks (Bi-ESNs) trained using support vector machine *privileged information* method (SVM\(+\)) to model a winding machine process. The proposed model will be applied, tested and compared to reported models in the literature such as classical ESN with linear regression, ESN with a linear SVM readout, genetic programming, feedfoward neural network with backpropagation, radial basis function network, adaptive neural fuzzy inference system and local linear wavelet neural network. The developed results show that Bi-ESNs trained with SVM\(+\) are promising. It was able to provide better generalization performance compared to other models.

## Keywords

Recurrent neural network Reservoir computing Support vector machines Privileged information Winding machines Engineering process## Notes

### Acknowledgments

The data set used in this research work was collected as part of a project supervised by Dr. M. Kamel at the Aluminum factory in Egypt and analyzed by Dr. Aboabbas Hussain.

### Compliance with ethical standards

### Conflict of interest

Ali Rodan declares that he has no conflict of interest. Alaa F. Sheta declares that he has no conflict of interest. Hossam Faris declares that he has no conflict of interest.

### Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

## References

- Abbasi Nozari H, Dehghan Banadaki H, Mokhtare M, Hekmati Vahed S (2012) Intelligent non-linear modelling of an industrial winding process using recurrent local linear neuro-fuzzy networks. J Zhejiang Univ Sci C 13:403–412CrossRefGoogle Scholar
- Al-Hiary H, Sheta A, Ayesh A (2008) Identification of a chemical process reactor using soft computing techniques. In: Proceedings of the 2008 international conference on fuzzy systems (FUZZ2008) within the 2008 IEEE world congress on computational intelligence (WCCI2008), Hong Kong, 1–6 June, pp 845–653Google Scholar
- Babinec S, Pospíchal J (2011) modular state space of echo state neural network in time series prediction. Comput Inform 30:321–334MATHGoogle Scholar
- Bastogne T, Noura H, Sibille P, Richard A (1998) Multivariable identification of a winding process by subspace methods for tension control. Control Eng Pract 6(9):1077–1088CrossRefGoogle Scholar
- Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5:157–166CrossRefGoogle Scholar
- Braatz R, Ogunnaike B, Featherstone A (1996) Identification, estimation and control of sheet and film processes. In: Proceedings of the 13th IFAC world congress. San Francisco, CA, pp 319–324Google Scholar
- Buesing L, Schrauwen B, Legenstein R (2010) Connectivity, dynamics and memory in reservoir computing with binary and analog neurons. Neural Comput 22:1272–1311Google Scholar
- Chan WC, Cheung KC, Harris CJ (2001) On the modelling of nonlinear dynamic system using support vector neural networks. Eng. Appl. Artif. Intell. 14:105–113CrossRefGoogle Scholar
- Chen Y, Yang B, Dong J (2006) Time-series prediction using a local linear wavelet neural network. Neurocomputing 69:449–456CrossRefGoogle Scholar
- Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines: and other kernel-based learning methods. Cambridge University Press, New YorkCrossRefMATHGoogle Scholar
- Du H, Zhang N (2008) Time series prediction using evolving radial basis function networks with new encoding scheme. Neurocomputing 71:1388–1400CrossRefGoogle Scholar
- Ebler N, Arnason R, Michaelis G, D’Sa N (1993) Tension control: dancer rolls or load cells. IEEE Trans Ind Appl 29(4):727–739CrossRefGoogle Scholar
- Faris H, Sheta A, Öznergiz E (2013) Modelling hot rolling manufacturing process using soft computing techniques. Int J Comput Integr Manuf 26(8):762–771CrossRefGoogle Scholar
- Faris H, Sheta AF, Öznergiz E (2016) MGP-CC: a hybrid multigene GP-cuckoo search method for hot rolling manufacture process modeling. Syst Sci Control Eng 4:39–49CrossRefGoogle Scholar
- Faris H, Sheta AF (2016) A comparison between parametric and non-parametric soft computing approaches to model the temperature of a metal cutting tool. Int J Comput Integr Manuf 29(1):64–75Google Scholar
- Fukushima H, Kim T-H, Sugie T (2007) Brief paper: adaptive model predictive control for a class of constrained linear systems based on the comparison model. Automatica 43:301–308MathSciNetCrossRefMATHGoogle Scholar
- Good J, Roisum DR (2008) Machines, mechanics and measurements. DEStech Publication, LancasterGoogle Scholar
- Hailiang H, Zhong W, Xiaohong N, Jing S (2015) Robust decentralized control of web-winding systems without tension sensor. In: Proceedings of the 2015 34th Chinese control conference (CCC), pp 8850–8854, July 2015Google Scholar
- Haykin S (1998) Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall PTR, Upper Saddle RiverMATHGoogle Scholar
- Hoshino I, Maekawa Y, Fujimoto T, Kimura H (1988) Observer-based multivariable control of the aluminum cold tandem mill. Automatica 24(6):741–754MathSciNetCrossRefMATHGoogle Scholar
- Hussian A, Sheta A, Abdelwahab A (2001) Modeling of a winding machine using non-parametric neural network model. In: WSEAS international conference on scientific computation and soft computing. Athens, Greece, pp 528–553Google Scholar
- Hussian A, Sheta A, Kamel M, Telbany M, Abdelwahab A (2000) Modeling of a winding machine using genetic programming. In: Proceedings of the congress on evolutionary computation (CEC2000), vol 82, pp 398–402Google Scholar
- Inoue Y, Takanashi K, Miyazawa N, Hyuga Y (1984) Management and control systems in the steel industry. Comput Ind 5(2):143–152 (Special Issue: Computers in Japanese Industry)Google Scholar
- Isobe T (1970) Automatic control in the iron and steel industry. Automatica 6(1):111–121CrossRefGoogle Scholar
- Jaeger H (2001) The echo state approach to analysing and training recurrent neural networks. Technical report GMD report 148, German National Research Center for Information TechnologyGoogle Scholar
- Jang J-S (1993) ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23:665–685CrossRefGoogle Scholar
- Jian W, Xu Z, Minghong W, Dehong D (2007) Research on spindle drive control system of high speed winding machine. In: Proceedings of the 8th international conference on electronic measurement and instruments (ICEMI’07), pp 473–477, Aug 2007Google Scholar
- Kara Y, Acar Boyacioglu M, Baykan OK (2011) Predicting direction of stock price index movement using artificial neural networks and support vector machines. Expert Syst Appl 38:5311–5319CrossRefGoogle Scholar
- Lukoševičius M, Jaeger H (2009) Reservoir computing approaches to recurrent neural network training. Comput Sci Rev 3:127–149CrossRefMATHGoogle Scholar
- Lukoševičius M, Jaeger H, Schrauwen B (2012) Reservoir computing trends. KI - Künstliche Intelligenz, pp 1–7, 05/2012Google Scholar
- Lukoševičius M, Marozas V (2014) Noninvasive fetal qrs detection using an echo state network and dynamic programming. Physiol Meas 35:1685CrossRefGoogle Scholar
- Mo W, Wang M, Lin J-S, Zan H, Sun G (2014) Control system based on plc for winding machine. In: Proceedings of the 2014 international symposium on computer, consumer and control (IS3C), pp 74–77, June 2014Google Scholar
- Muller K, Smola A, Ratsch G, Scholkopf B, kohlmorgen J, Vapnik V (1997) Predicting rime series with support vector machines. In: Proceedings of the 7th international conference on artificial neural networks (ICANN’97), pp 999–1004Google Scholar
- Nozari HA, Banadaki HD, Mokhtare M, Vahed SH (2012) Intelligent non-linear modelling of an industrial winding process using recurrent local linear neuro-fuzzy networks. J Zhejiang Univ Sci C 13(6):403–412CrossRefGoogle Scholar
- Parant F, Coeffier C, Iung C (1992) Modeling of a web tension in a continuous annealing line. Iron and Steel Engineer, PittsburghGoogle Scholar
- Rodan A, Tiňo P (2010) Simple deterministically constructed recurrent neural networks. In: Fyfe C et al (eds) Intelligent data engineering and automated learning—IDEAL 2010. Springer, pp 267–274Google Scholar
- Rodan A, Tino P (2011) Negatively correlated echo state networks. In: ESANN, CiteseerGoogle Scholar
- Rumelhart DE, Hinton GE, Williams RJ (1988) Neurocomputing: foundations of research. Ch. Learning internal representations by error propagation. MIT Press, Cambridge, pp 673–695Google Scholar
- Scardapane S, Wang D, Panella M (2015) A decentralized training algorithm for echo state networks in distributed big data applications. Neural Netw 78:65–74CrossRefGoogle Scholar
- Sheta AF, Faris H, Öznergiz E (2014) Improving production quality of a hot-rolling industrial process via genetic programming model. Int J Comput Appl Technol 49:239–250Google Scholar
- Sheta A, Ahmed SE, Faris H (2015) A comparison between regression, artificial neural networks and support vector machines for predicting stock market index. Int J Adv Res Artif Intell IJARAI 4(7):55–63Google Scholar
- Sheta A, Al-Hiary H, Braik M (2009) Identification and model predictive controller design of the Tennessee Eastman chemical process using ANN. In: Proceedings of the 2009 international conference on artificial intelligence (ICAI’09), July 13–16, USA, vol 1, pp 25–31Google Scholar
- Sievers L, Balas M, von Flotow A (1988) Modeling of web conveyance systems for multivariable control. IEEE Trans Autom Control 33:524–531CrossRefMATHGoogle Scholar
- Sievers L, Balas M, Flotow AV (1988) Modeling of web conveyance systems for multivariable control. IEEE Trans Autom Control 33(6):524–531CrossRefMATHGoogle Scholar
- Smola AJ, Scholkopf B (1998) A tutorial on support vector regression. Technical report, Technical report NC2-TR-1998-030, University of London, Royal HollowayGoogle Scholar
- Tiňo P, Rodan A (2013) Short term memory in input-driven linear dynamical systems. Neurocomputing 112:58–63CrossRefGoogle Scholar
- Vapnik V (1995) The nature of statistical learning theory. Springer, New YorkCrossRefMATHGoogle Scholar
- Vapnik VN (1999) An overview of statistical learning theory. Trans Neural Netw 10:988–999CrossRefGoogle Scholar
- Vapnik V, Izmailov R (2015) Learning using privileged information: similarity control and knowledge transfer. J Mach Learn Res 16:2023–2049MathSciNetMATHGoogle Scholar
- Vapnik V, Vashist A (2009) A new learning paradigm: learning using privileged information. Neural Netw 22(5–6):544–557CrossRefMATHGoogle Scholar
- Walker N, Wyatt-Mair G (1995) Sensor signal validation using analytical redundancy for an aluminum cold rolling mill. Control Eng Pract 3(6):753–760CrossRefGoogle Scholar
- Wang WJ, Xu ZB, Lu WZ (2003) Determination of the spread parameter in the gaussian kernel for classification and regression. Neurocomputing 55:643–663CrossRefGoogle Scholar
- Wang X, Yan Heshan (2015) Optimizing the echo state network with a binary particle swarm optimization algorithm. Knowl Based Syst 86:09Google Scholar
- Wei C-H, Wu C-H (2004) A simulator of winding machine controller using LabView environment. In: Control, automation, robotics and vision conference, 2004. ICARCV 2004 8th vol 3, pp 2105–2110, Dec 2004Google Scholar
- Zhu GQ, Liu SR, Yu JS (2002) Support vector machine and its applications to function approximation. J East China Univ Sci Technol 5:555–559Google Scholar