Abstract
Process monitoring or fault detection and diagnosis have gained tremendous attention over the past decade in order to achieve better product quality, minimise downtime and maximise profit in process industries. Among various process monitoring techniques, data-based machine learning approaches have become immensely popular in the past decade. However, a promising machine learning technique Gaussian process regression has not yet received adequate attention for process monitoring. In this work, Gaussian process regression (GPR)-based process monitoring approach is applied to the benchmark Tennessee Eastman challenge problem. Effect of various GPR hyper-parameters on monitoring efficiency is also thoroughly investigated. The results of GPR model is found to be better than many other techniques which is reported in a comparative study in this work.
Similar content being viewed by others
Abbreviations
- abs:
-
Absolute value
- a, b, B, c, d :
-
Weights assigned
- cov:
-
Predicted covariance
- \({\mathcal{D}}\) :
-
Dataset
- D :
-
Number of variables
- \({\mathbb{E}}\left( . \right)\) :
-
Expectations
- \(f_{*} ,g_{*}\) :
-
Gaussian process posterior prediction
- \(\bar{f}_{*} ,\bar{g}\) :
-
Predicted mean
- \({\mathcal{G}\mathcal{P}}\) :
-
Gaussian process
- \(I\) :
-
Identity matrix
- \(k\) :
-
Covariance function for a single case
- \(k_{*}\) :
-
Covariance function for single test case
- K :
-
Covariance (gram) matrix
- l :
-
Length scale
- m :
-
Number of samples
- \(m\left( x \right)\) :
-
Mean function
- \({\mathcal{N}}\) :
-
Normal distribution function
- o :
-
Value associated with \(\left( {1 - \alpha } \right)\%\) confidence interval
- \(p\) :
-
Probability density
- \({\mathbb{V}}\) :
-
Variance of a sample
- x :
-
Sample vector
- X :
-
Matrix of training input
- \(X_{*}\) :
-
Matrix of testing input
- \(y\) :
-
Output training vector
- \(\beta\) :
-
Gaussian prior
- \(\bar{\beta }\) :
-
Constant
- \(\delta_{pq}\) :
-
Kronecker delta
- \(\epsilon\) :
-
Gaussian noise
- \(\sigma\) :
-
Variance associated with the predicted output
- \(\sigma_{f}^{2}\) :
-
Signal variance
- \(\sigma_{n}^{2}\) :
-
Noise variance
- \({\text{BIC}}_{\text{SPE}}\) :
-
Bayesian Inference-Based Criterion - Squared Prediction Error
- \({\text{BIP}}_{\text{S}}\) :
-
Bayesian inference-based probability in state space
- BIP:
-
Bayesian inference-based probability
- BSPCA:
-
Bayesian strategy PCA
- CCA-ERD:
-
Canonical correlation analysis-based explicit relation discovery
- D:
-
Dissimilarity index
- DiPCA:
-
Distributed PCA
- DR:
-
Distance from the hypersphere
- DTL:
-
Determining whether it is tight or loose
- EICA:
-
Ensemble ICA
- FA:
-
Factor analysis
- FA-ICA:
-
Factor analysis-ICA
- \(GT^{2}\) :
-
Statistic to monitor the factor space
- GLSA:
-
Global local structure analysis
- GMM:
-
Gaussian mixture models
- JIR-PCA-SVDD:
-
Just in time response-PCA-SVDD
- KGLPP:
-
Kernel global local preserving projections
- KICA:
-
Kernel independent component analysis
- KLPP:
-
Kernel local preserving projections
- LDS:
-
Linear dynamic system
- LGPCA:
-
Local global principal component analysis
- LGSS:
-
Linear Gaussian state Space
- LPP:
-
Local preserving projections
- Modified ICA:
-
Modified independent component analysis
- MCVA:
-
Mixture canonical variate analysis
- NGS:
-
Non Gaussian static
- OCSVM:
-
One class support vector machines
- OMMP:
-
Orthogonal multi manifold projections
- PCA-SVDD PCA:
-
Support vector data description
- PM-WKPCA:
-
Probability density estimation and moving weighted PCA
- PPCA:
-
Probabilistic PCA
- P-WKPCA:
-
Probability density estimation and weighted PCA
- RPCA:
-
Recursive PCA
- Semi NMU:
-
Semi nonnegative matrix approximations
- SFA:
-
Sensitive factor analysis
- SLDS:
-
Supervised linear dynamic system
- SPE:
-
Squared prediction error
- SVM:
-
Support vector machines
- SpPCA:
-
Sparce PCA
- \(T^{2}\) :
-
Hotelling \(T^{2}\)
- \(T_{f}^{2}\) :
-
Hotelling \(T^{2}\) in feature subspace
- \({\text{WGT}}^{2}\) :
-
Modified hotelling \(T^{2}\)
- WFA:
-
Weighted factor analysis
- wOCSVM:
-
Weighted one class support vector machines
- WPCA:
-
Weighted PCA
References
Russell, E.; Chiang, L.H.: Data-Driven Methods for Fault Detection and Diagnosis in Chemical Processes, 2001st ed., Springer, London (n.d.)
Ge, Z.; Song, Z.; Ding, S.X.; Huang, B.: Data mining and analytics in the process industry: the role of machine learning. IEEE Access. 5, 20590–20616 (2017). https://doi.org/10.1109/ACCESS.2017.2756872
Jackson, J.E.: Principal components and factor analysis: part I—principal components. J. Qual. Technol. 12, 201–213 (1980). https://doi.org/10.1080/00224065.1980.11980967
Russell, E.L.; Chiang, L.H.; Braatz, R.D.: Fault detection in industrial processes using canonical variate analysis and dynamic principal component analysis. Chemom. Intell. Lab. Syst. 51, 81–93 (2000). https://doi.org/10.1016/S0169-7439(00)00058-7
Shao, J.D.; Rong, G.; Lee, J.M.: Learning a data-dependent kernel function for KPCA-based nonlinear process monitoring. Chem. Eng. Res. Des. 87, 1471–1480 (2009). https://doi.org/10.1016/j.cherd.2009.04.011
Rato, T.J.; Reis, M.S.: Fault detection in the Tennessee Eastman benchmark process using dynamic principal components analysis based on decorrelated residuals (DPCA-DR). Chemom. Intell. Lab. Syst. 125, 101–108 (2013). https://doi.org/10.1016/J.CHEMOLAB.2013.04.002
Lee, J.M.; Yoo, C.K.; Lee, I.B.: Statistical monitoring of dynamic processes based on dynamic independent component analysis. Chem. Eng. Sci. 59, 2995–3006 (2004). https://doi.org/10.1016/j.ces.2004.04.031
Yin, L.; Wang, H.; Fan, W.: Active learning-based support vector data description method for robust novelty detection. Knowl. Based Syst. 153, 40–52 (2018). https://doi.org/10.1016/j.knosys.2018.04.020
Karami, M.; Wang, L.: Fault detection and diagnosis for nonlinear systems: a new adaptive Gaussian mixture modeling approach. Energy Build. 166, 477–488 (2018). https://doi.org/10.1016/j.enbuild.2018.02.032
Jia, Q.; Zhang, Y.: Quality-related fault detection approach based on dynamic kernel partial least squares. Chem. Eng. Res. Des. 106, 242–252 (2016). https://doi.org/10.1016/j.cherd.2015.12.015
Xiao, Y.; Wang, H.; Zhang, L.; Xu, W.: Two methods of selecting Gaussian kernel parameters for one-class SVM and their application to fault detection. Knowl. Based Syst. 59, 75–84 (2014). https://doi.org/10.1016/j.knosys.2014.01.020
Yang, Q.; Li, J.; Le Blond, S.; Wang, C.: Artificial neural network based fault detection and fault location in the DC microgrid. Energy Procedia 103, 129–134 (2016). https://doi.org/10.1016/j.egypro.2016.11.261
Chiang, L.H.; Kotanchek, M.E.; Kordon, A.K.: Fault diagnosis based on Fisher discriminant analysis and support vector machines. Comput. Chem. Eng. 28, 1389–1401 (2004). https://doi.org/10.1016/J.COMPCHEMENG.2003.10.002
Boškoski, P.; Gašperin, M.; Petelin, D.; Juričić, D.: Bearing fault prognostics using Rényi entropy-based features and Gaussian process models. Mech. Syst. Signal Process. 52–53, 327–337 (2015). https://doi.org/10.1016/j.ymssp.2014.07.011
Deng, H.; Liu, Y.; Li, P.; Zhang, S.: Active learning for modeling and prediction of dynamical fluid processes. Chemom. Intell. Lab. Syst. 183, 11–22 (2018). https://doi.org/10.1016/j.chemolab.2018.10.005
Liu, Y.; Chen, T.; Chen, J.: Auto-switch gaussian process regression-based probabilistic soft sensors for industrial multigrade processes with transitions. Ind. Eng. Chem. Res. 54, 5037–5047 (2015). https://doi.org/10.1021/ie504185j
Liu, Y.; Wu, Q.Y.; Chen, J.: Active selection of informative data for sequential quality enhancement of soft sensor models with latent variables. Ind. Eng. Chem. Res. 56, 4804–4817 (2017). https://doi.org/10.1021/acs.iecr.6b04620
Samuelsson, O.; Björk, A.; Zambrano, J.; Carlsson, B.: Gaussian process regression for monitoring and fault detection of wastewater treatment processes. Water Sci. Technol. 75, 2952–2963 (2017). https://doi.org/10.2166/wst.2017.162
Rasmussen, C.E.: Gaussian processes in machine learning. In: Bousquet, O., von Luxburg, U., Rätsch, G. (eds.) Advanced Lectures on Machine Learning. ML 2003. Lecture Notes in Computer Science, vol. 3176. Springer, Berlin, Heidelberg (2004). https://doi.org/10.1007/978-3-540-28650-9_4
Downs, J.J.; Vogel, E.F.: A plant-wide industrial process control problem. Comput. Chem. Eng. 17, 245–255 (1993). https://doi.org/10.1016/0098-1354(93)80018-I
Chen, J.; Liao, C.M.: Dynamic process fault monitoring based on neural network and PCA. J. Process Control 12, 277–289 (2002). https://doi.org/10.1016/S0959-1524(01)00027-0
Zhang, Y.: Enhanced statistical analysis of nonlinear processes using KPCA, KICA and SVM. Chem. Eng. Sci. 64, 801–811 (2009). https://doi.org/10.1016/j.ces.2008.10.012
Ge, Z.; Yang, C.; Song, Z.: Improved kernel PCA-based monitoring approach for nonlinear processes. Chem. Eng. Sci. 64, 2245–2255 (2009). https://doi.org/10.1016/j.ces.2009.01.050
Ben Khediri, I.; Limam, M.; Weihs, C.: Variable window adaptive Kernel Principal Component Analysis for nonlinear nonstationary process monitoring. Comput. Ind. Eng. 61, 437–446 (2011). https://doi.org/10.1016/j.cie.2011.02.014
Ma, H.; Hu, Y.; Shi, H.: A novel local neighborhood standardization strategy and its application in fault detection of multimode processes. Chemom. Intell. Lab. Syst. 118, 287–300 (2012). https://doi.org/10.1016/j.chemolab.2012.05.010
Jiang, Q.; Yan, X.: Chemical processes monitoring based on weighted principal component analysis and its application. Chemom. Intell. Lab. Syst. 119, 11–20 (2012). https://doi.org/10.1016/j.chemolab.2012.09.002
Fan, J.; Wang, Y.: Fault detection and diagnosis of nonlinear non-Gaussian dynamic processes using kernel dynamic independent component analysis. Inf. Sci. (Ny) 259, 369–379 (2014). https://doi.org/10.1016/J.INS.2013.06.021
Jing, C.; Hou, J.: SVM and PCA-based fault classification approaches for complicated industrial process. Neurocomputing 167, 636–642 (2015). https://doi.org/10.1016/j.neucom.2015.03.082
Hu, Y.; Ma, H.; Shi, H.: Robust online monitoring based on spherical-kernel partial least squares for nonlinear processes with contaminated modeling data. Ind. Eng. Chem. Res. 52, 9155–9164 (2013). https://doi.org/10.1021/ie4008776
Yin, S.; Zhu, X.; Kaynak, O.: Improved PLS focused on key-performance-indicator-related fault diagnosis. IEEE Trans. Ind. Electron. 62, 1651–1658 (2015). https://doi.org/10.1109/TIE.2014.2345331
Zhou, D.; Li, G.; Qin, S.J.: Total projection to latent structures for process monitoring. AIChE J. (2009). https://doi.org/10.1002/aic.11977
Odiowei, P.-E.P.; Cao, Yi: Nonlinear dynamic process monitoring using canonical variate analysis and kernel density estimations. IEEE Trans. Ind. Inform. 6, 36–45 (2010). https://doi.org/10.1109/TII.2009.2032654
Yin, S.; Wang, G.; Gao, H.: Data-driven process monitoring based on modified orthogonal projections to latent structures. IEEE Trans. Control Syst. Technol. 24, 1480–1487 (2016). https://doi.org/10.1109/TCST.2015.2481318
Xie, L.; Kruger, U.; Lieftucht, D.; Littler, T.; Chen, Q.; Wang, S.Q.: Statistical monitoring of dynamic multivariate processes: part 1. Modeling autocorrelation and cross-correlation. Ind. Eng. Chem. Res. 45, 1659–1676 (2006). https://doi.org/10.1021/ie050583r
Lee, J.M.; Qin, S.J.; Lee, I.B.: Fault detection and diagnosis based on modified independent component analysis. AIChE J. 52, 3501–3514 (2006). https://doi.org/10.1002/aic.10978
Zhang, Y.; Zhang, Y.: Fault detection of non-Gaussian processes based on modified independent component analysis. Chem. Eng. Sci. 65, 4630–4639 (2010). https://doi.org/10.1016/j.ces.2010.05.010
Wang, L.; Shi, H.: Multivariate statistical process monitoring using an improved independent component analysis. Chem. Eng. Res. Des. 88, 403–414 (2010). https://doi.org/10.1016/j.cherd.2009.09.002
Odiowei, P.P.; Cao, Y.: State-space independent component analysis for nonlinear dynamic process monitoring. Chemom. Intell. Lab. Syst. 103, 59–65 (2010). https://doi.org/10.1016/j.chemolab.2010.05.014
Jiang, Q.; Yan, X.: Non-Gaussian chemical process monitoring with adaptively weighted independent component analysis and its applications. J. Process Control 23, 1320–1331 (2013). https://doi.org/10.1016/j.jprocont.2013.09.008
Ding, S.X.; Yin, S.; Hao, H.; Zhang, P.; Haghani, A.: A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process. J. Process Control 22, 1567–1581 (2012). https://doi.org/10.1016/j.jprocont.2012.06.009
Yu, J.: Hidden Markov models combining local and global information for nonlinear and multimodal process monitoring. J. Process Control 20, 344–359 (2010). https://doi.org/10.1016/j.jprocont.2009.12.002
Tao, Y.; Shi, H.; Song, B.; Tan, S.: Parallel supervised additive and multiplicative faults detection for nonlinear process. J. Frankl. Inst. 356, 11716–11740 (2019). https://doi.org/10.1016/j.jfranklin.2019.06.020
Jiang, Q.; Yan, X.: Probabilistic monitoring of chemical processes using adaptively weighted factor analysis and its application. Chem. Eng. Res. Des. 92, 127–138 (2014). https://doi.org/10.1016/j.cherd.2013.06.031
Jiang, Q.; Yan, X.: Probabilistic Weighted NPE-SVDD for chemical process monitoring. Control Eng. Pract. 28, 74–89 (2014). https://doi.org/10.1016/j.conengprac.2014.03.008
Jiang, Q.; Yan, X.; Huang, B.: Performance-driven distributed PCA process monitoring based on fault-relevant variable selection and Bayesian inference. IEEE Trans. Ind. Electron. 63, 377–386 (2016). https://doi.org/10.1109/TIE.2015.2466557
Wang, B.; Li, Z.; Yan, X.: Multi-subspace factor analysis integrated with support vector data description for multimode process monitoring. J. Frankl. Inst. 355, 7664–7690 (2018). https://doi.org/10.1016/j.jfranklin.2018.07.044
Wen, Q.; Ge, Z.; Song, Z.: Multimode dynamic process monitoring based on mixture canonical variate analysis model. Ind. Eng. Chem. Res. 54, 1605–1614 (2015). https://doi.org/10.1021/ie503324g
Liu, Y.; Xie, M.: Rebooting data-driven soft-sensors in process industries: a review of kernel methods. J. Process Control 89, 58–73 (2020). https://doi.org/10.1016/j.jprocont.2020.03.012
Apsemidis, A.; Psarakis, S.; Moguerza, J.M.: A review of machine learning kernel methods in statistical process monitoring. Comput. Ind. Eng. 142, 106376 (2020). https://doi.org/10.1016/j.cie.2020.106376
Rasmussen, C.E.; Nickisch, H.: Gaussian processes for machine learning (GPML) toolbox. J. Mach. Learn. Res. 11, 3011–3015 (2010)
Ge, Z.; Song, Z.: Distributed PCA model for plant-wide process monitoring. Ind. Eng. Chem. Res. 52, 1947–1957 (2013). https://doi.org/10.1021/ie301945s
Ge, Z.; Chen, X.: Supervised linear dynamic system model for quality related fault detection in dynamic processes. J. Process Control. 44, 224–235 (2016). https://doi.org/10.1016/j.jprocont.2016.06.003
Rato, T.; Reis, M.; Schmitt, E.; Hubert, M.; De Ketelaere, B.: A systematic comparison of PCA-based Statistical Process Monitoring methods for high-dimensional, time-dependent Processes. AIChE J. 62, 1478–1493 (2016). https://doi.org/10.1002/aic.15062
Luo, L.; Bao, S.; Mao, J.; Tang, D.: Nonlinear process monitoring based on kernel global-local preserving projections. J. Process Control. 38, 11–21 (2016). https://doi.org/10.1016/j.jprocont.2015.12.005
Luo, L.; Bao, S.; Mao, J.; Tang, D.: Fault detection and diagnosis based on sparse PCA and two-level contribution plots. Ind. Eng. Chem. Res. 56, 225–240 (2017). https://doi.org/10.1021/acs.iecr.6b01500
Meng, S.; Tong, C.; Lan, T.; Yu, H.: Canonical correlation analysis-based explicit relation discovery for statistical process monitoring. J. Franklin Inst. (2020). https://doi.org/10.1016/j.jfranklin.2020.01.049s
Ge, Z.; Song, Z.: Performance-driven ensemble learning ICA model for improved non-Gaussian process monitoring. Chemom. Intell. Lab. Syst. 123, 1–8 (2013). https://doi.org/10.1016/j.chemolab.2013.02.001
Li, N.; Yang, Y.: Using semi-nonnegative matrix underapproximation for statistical process monitoring. Chemom. Intell. Lab. Syst. 153, 126–139 (2016). https://doi.org/10.1016/j.chemolab.2016.03.006
Ge, Z.; Zhang, M.; Song, Z.: Nonlinear process monitoring based on linear subspace and Bayesian inference. J. Process Control. 20, 676–688 (2010). https://doi.org/10.1016/j.jprocont.2010.03.003
Jiang, Q.; Yan, X.: Just-in-time reorganized PCA integrated with SVDD for chemical process monitoring. AIChE J. 60, 949–965 (2014). https://doi.org/10.1002/aic.14335
Xiao, Y.; Wang, H.; Xu, W.; Zhou, J.: Robust one-class SVM for fault detection. Chemom. Intell. Lab. Syst. 151, 15–25 (2016). https://doi.org/10.1016/j.chemolab.2015.11.010
Tong, C.; Shi, X.; Lan, T.: Statistical process monitoring based on orthogonal multi-manifold projections and a novel variable contribution analysis. ISA Trans. 65, 407–417 (2016). https://doi.org/10.1016/j.isatra.2016.06.017
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Maran Beena, A., Pani, A.K. Fault Detection of Complex Processes Using nonlinear Mean Function Based Gaussian Process Regression: Application to the Tennessee Eastman Process. Arab J Sci Eng 46, 6369–6390 (2021). https://doi.org/10.1007/s13369-020-05052-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13369-020-05052-x