Phase II monitoring of auto-correlated linear profiles using linear mixed model

Abstract

In many circumstances, the quality of a process or product is best characterized by a given mathematical function between a response variable and one or more explanatory variables that is typically referred to as profile. There are some investigations to monitor auto-correlated linear and nonlinear profiles in recent years. In the present paper, we use the linear mixed models to account autocorrelation within observations which is gathered on phase II of the monitoring process. We undertake that the structure of correlated linear profiles simultaneously has both random and fixed effects. The work enhanced a Hotelling’s T² statistic, a multivariate exponential weighted moving average (MEWMA), and a multivariate cumulative sum (MCUSUM) control charts to monitor process. We also compared their performances, in terms of average run length criterion, and designated that the proposed control charts schemes could effectively act in detecting shifts in process parameters. Finally, the results are applied on a real case study in an agricultural field.

Introduction

Control charts are used to detect anomalies in the processes. They are most often used to monitor production-related processes. In many business-related processes, the quality of a process or product can be characterized by a relationship between a response variable and one or more explanatory variables which is referred to as profile. The purpose of the analyzing of profile in phase I is to determine the stability of the process and estimate parameters, however, in phase II, analyzers are interested in rapidly detecting the significant shifts in the process parameters. Phase I analysis of simple linear profiles has been investigated by a number of authors such as Stover and Brill (1998), Kang and Albin (2000), Kim et al. (2003) and Mahmoud et al. (Mahmoud and Woodall 20042007). Many authors including Kang and Albin (2000), Kim et al. (2003), Noorossana et al. (2004), Gupta et al. (2006), Zou et al. (2006), Saghaei et al. (2009), and Mahmoud et al. (2009) have investigated phase II monitoring of simple linear profiles. Noorossana et al. (2010a, b) investigated monitoring of multivariate simple linear profiles on phase II. Zou et al. (2007) and Kazemzadeh et al. (2009a, b) considered cases when the profiles can be characterized by multiple and polynomial regression models respectively. Mahmoud (2008) considered phase I monitoring of multiple linear profiles, and Kazemzadeh et al. (2008) proposed three methods for monitoring the k th-order polynomial profile in phase I. Ding et al. (2006), Moguerza et al. (2007), Williams et al. (2007), and Vaghefi et al. (2009) investigated nonlinear profiles. In these studies, it is implicitly assumed that the error terms within or between profiles is independently and identically normally distributed; however in some cases, these assumptions can be violated. Noorossana et al. (2010a, b) analyzed the effects of non-normality on the monitoring of simple linear profiles. Noorossana et al. (2008) and Kazemzadeh et al. (2009a, b) investigated autocorrelation between successive simple linear and polynomial profiles respectively. Soleimani et al. (2009) proposed a transformation to eliminate the autocorrelation between observations within a simple linear profile in phase II. Jensen et al. (2008) proposed two T² control charts based on linear mixed model (LMM) to account for the autocorrelation within linear profiles in phase I. They concluded that the linear mixed model is superior to the least square approach for unbalanced or missing data, especially when the number of observation within a profile is small and the correlation is weak. Jensen and Birch (2009) used nonlinear mixed model to account correlation within nonlinear profiles. Qie et al. (2010) investigated nonparametric profile monitoring with arbitrary design using mixed models. They proposed a control chart that combines the exponentially weighted moving average control chart based on local linear kernel smoothing and a nonparametric regression test under the assumption that observations within and between individual profiles are independent of each other.

The present study acts as an extension of the work of Jensen et al. (2008) in applying a linear mixed model on the presence of autocorrelation within linear profiles on phase I control chart applications; conversely, our focus is on phase II of profile monitoring in which one could use the proposed control charts to detect any departures from the given profile parameters.

The remainder of the paper is organized as follows. In ‘Linear mixed model’ Section, the LMM is mathematically presented. In the ‘Proposed methods’ Section, our methods including three modified multivariate control charts namely Hotelling T², multivariate exponential weighted moving average (MEWMA) and a multivariate cumulative sum control charts (MCUSUM) are illustrated. In ‘Simulation studies’ Section, the results of simulation study to evaluate the performance of the methods are presented. In addition, a case study from an agriculture field is investigated on the Section ‘Case study’. The final section closes with concluding remarks.

Linear mixed model

Linear mixed models (Laird and Ware 1982) are popular for analysis of longitudinal data. A linear mixed model contains fixed and random effects and is linear in these effects. This model allows us to account autocorrelation within profiles. In matrix notation, a mixed model can be represented as

$$\mathbf{y}=\mathbf{X\beta }+\mathbf{Zb}+\mathcal{ϵ},$$

(1)

where y is a vector of observations, with mean E(y) = Xβ, β is a vector of fixed effects, b is a vector of independent and identically distributed (IID) random effects with mean E(b) = 0 and variance-covariance matrix Var(b) = D_,ϵ is a vector of IID random error terms with mean $E\left(ϵ\right)=\mathbf{0}$ and variance $\mathrm{Var}\left(\mathbf{ϵ}\right)=\mathbf{R}$, and X and Z are matrices of regressors relating the observations y to β and b.

In the 1950s, Charles Roy Henderson provided the best linear unbiased estimate ( BLUE) of fixed effects and best linear unbiased predictions (BLUP) of random effects. Subsequently, mixed modeling has become a major area of statistical research, including work on the computation of maximum likelihood estimates, nonlinear mixed effect models, missing data in mixed effects models, and Bayesian estimation of mixed effects models (West et al. 2007).

Henderson’s ‘mixed model equations’ (MME) are (Robinson 1991) as follows:

$$\left(\begin{array}{ll}\hfill {\mathbf{X}}^{\text{'}}{\mathbf{R}}^{-1}\mathbf{X}\hfill & \hfill {\mathbf{X}}^{\text{'}}{\mathbf{R}}^{-1}\mathbf{Z}\hfill \\ \hfill {\mathbf{Z}}^{\text{'}}{\mathbf{R}}^{-1}\mathbf{X}\hfill & \hfill {\mathbf{Z}}^{\text{'}}{\mathbf{R}}^{-1}\mathbf{Z}+{\mathbf{D}}^{-1}\hfill \end{array}\right)\left(\begin{array}{l}\hfill \hat{\mathbf{\beta }}\hfill \\ \hfill \hat{\mathbf{b}}\hfill \end{array}\right)=\left(\begin{array}{l}\hfill {\mathbf{X}}^{\text{'}}{\mathbf{R}}^{-1}\mathbf{y}\hfill \\ \hfill {\mathbf{Z}}^{\text{'}}{\mathbf{R}}^{-1}\mathbf{y}\hfill \end{array}\right)$$

(2)

The solutions to the MME, $\hat{\mathbf{\beta }}$, and $\hat{\mathbf{b}}$ are BLUEs and BLUPs for β and b, respectively.

Mixed models require somewhat sophisticated computing algorithms to fit. Solutions to the MME are obtained by methods similar to those used for linear least squares. For complicated models and large datasets, iterative methods may be needed.

In profile monitoring, one could suppose that the j th response follows a LMM; therefore,

$${\mathbf{y}}_j={\mathbf{X}}_j\mathbf{\beta }+{\mathbf{Z}}_j{\mathbf{b}}_j+{\mathbf{ϵ}}_j;j=1,2,\dots ,m,$$

(3)

where X _i is a (n _j  × p) matrix of regressors, and Z _j is a (n _j  × q) matrix associated with random effects. β is a (p × 1) vector of fixed effects, and y _j is the (n _j  × 1) response vector for the j th profile. The coefficient vector of the random effect terms is b _j  ~ MN (0,D), and D is assumed to be a diagonal matrix; thus, the random effects are assumed not to be correlated with each other. In addition, it is assumed that Cov(ϵ _j , b _j ) = 0 and ϵ _j is (n _j  × 1) vector of errors where ϵ_j ~ MN (0,R _j ). If the errors are assumed to be independent, R _j  = σ²I, but correlated, the functional structure for the error terms may be used.

As noted before, it is considered that $\hat{\mathbf{\beta }}$ is an estimator of β, and ${\hat{\mathbf{b}}}_j$ is a predictor of b _j , then ${\hat{\mathbf{y}}}_j={\mathbf{X}}_j\hat{\mathbf{\beta }}$ is the population average, and ${\hat{\mathbf{y}}}_j={\mathbf{X}}_j\hat{\mathbf{\beta }}+{\mathbf{Z}}_j{\hat{\mathbf{b}}}_j$ is the profile specific prediction; so if D and R _j are known, then it can be shown as follows:

$${\mathbf{\beta }}^{̂}={\left({\displaystyle {\sum }_{j=1}^m{\mathbf{X}}_j^{\text{'}}{\mathbf{V}}_j^{-1}{\mathbf{X}}_j}\right)}^{-1}\left({\displaystyle {\sum }_{j=1}^m{\mathbf{X}}_j^{\text{'}}{\mathbf{V}}_j^{-1}{\mathbf{y}}_j}\right),$$

(4)

and the BLUP of b is

$${\hat{\mathbf{b}}}_j=\mathbf{D}{\mathbf{Z}}_j^{\text{'}}{\mathbf{V}}_j^{-1}\left({\mathbf{y}}_j-{\mathbf{X}}_j{\mathbf{\beta }}^{̂}\right),$$

(5)

where ${\mathbf{V}}_j={\mathbf{Z}}_j\mathbf{D}{\mathbf{Z}}_j^{\text{'}}+{\mathbf{R}}_j$ is the overall estimated variance covariance matrix (Schabenberger and Pierce 2002).

Proposed methods

In this paper, we propose a linear mixed model approach for accounting the correlation within linear profiles in phase II. It is assumed that profiles are correlated based on first-order autoregressive (AR(1)) structure.

If the errors follow an auto-correlated structure such as an AR(1) structure, then R _j by Schabenberger and Pierce (2002) is given as Equation 6. More details on the types of correlated errors structures could be acquired in this reference as

$$\mathbf{R}j={\sigma }^2\left[\begin{array}{lllll}\hfill 1\hfill & \hfill \rho \hfill & \hfill {\rho }^2\hfill & \hfill \cdots \hfill & \hfill {\rho }^{\mathit{nj}-1}\hfill \\ \hfill \rho \hfill & \hfill 1\hfill & \hfill \rho \hfill & \hfill \cdots \hfill & \hfill {\rho }^{\mathit{nj}-2}\hfill \\ \hfill {\rho }^2\hfill & \hfill \rho \hfill & \hfill 1\hfill & \hfill \cdots \hfill & \hfill {\rho }^{\mathit{nj}-3}\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill \ddots \hfill & \hfill ⋮\hfill \\ \hfill {\rho }^{\mathit{nj}-1}\hfill & \hfill {\rho }^{\mathit{nj}-2}\hfill & \hfill {\rho }^{\mathit{nj}-3}\hfill & \hfill \cdots \hfill & \hfill 1\hfill \end{array}\right].$$

(6)

It is assumed that for the j th sample collected over time, our observations are (X _i ,y _ij ), i = 1,2,…,n and j = 1,2,…,m. We considered the case that all the fixed effects have a corresponding random effect,  (X _j  = Z _j ). If the process is in control, the problem can be formulated as follows:

$$\begin{array}{ll}{y}_{\mathit{ij}}=& \left({\beta }_0+b_{0j}\right)+\left({\beta }_1+b_{1j}\right)x_{i1}+\dots \\ +\left({\beta }_{p-1}+b_{p-1j}\right)x_{p-1i}+{ϵ}_{\mathit{ij}}\end{array}$$

(7)

and

$${ϵ}_{\mathit{ij}}=\rho {ϵ}_{\mathit{ij}-1}+a_{\mathit{ij}},$$

(8)

where ϵ _ij are the correlated error terms and a _ij are white noises as a _ij  ~ N(0,σ²). The β₀, β₁,…,β_{p − 1} are fixed effects that are the same for all profiles. The b_0j,b_1j,…,b_{p − 1j} are random effects for the j th profile and they are normal random variables with zero mean and variance of ${\sigma }_0^2,{\sigma }_1^2,\dots ,{\sigma }_{p-1}^2$, respectively, which are not to be correlated with each other and also not to be correlated with the errors. The x values are fixed and constant from profile to profile. In this article we especially focused on phase II of the monitoring process, so all profile’s parameters, process variance, and correlation coefficient are known in phase I. Accordingly, we utilized the modified Jensen et al. (2008) approach to monitor autocorrelation on phase II.

The Hotelling’s T²statistic control chart

As a first proposed control chart, we use T² statistic to monitor the fixed effects for each sample. This statistic is given by

$$T_j^2={\left({\hat{\mathbf{\beta }}}_j-{\mathbf{\beta }}_0\right)}^{\text{'}}{{\displaystyle \sum }}^{-1}\left({\hat{\mathbf{\beta }}}_j-{\mathbf{\beta }}_0\right),$$

(9)

where ${\hat{\mathbf{\beta }}}_j={\left({\mathbf{X}}_j^{-1}{\mathbf{V}}_j^{-1}{\mathbf{X}}_j\right)}^{-1}\left({\mathbf{X}}_j^{-1}{\mathbf{V}}_j^{-1}{\mathbf{y}}_j\right)$ and β₀ denote the in-control value of β.

In Equation 9 the variance covariance matrix of fixed effects is ${\displaystyle {\sum }_{\mathbf{\beta }}={\left({\mathbf{X}}_j^{\text{'}}{\mathbf{V}}_j^{-1}{\mathbf{X}}_j\right)}^{-1}}$.

The upper control limit, UCL, is chosen to achieve a specified in control average run length (ARL).

The MEWMA control chart

Our second proposed control chart is based on MEWMA proposed by Lowry et al. (1992) for monitoring the vector of ${\hat{\mathbf{\beta }}}_j$. Here the MEWMA statistics is as follows:

$${\mathbf{z}}_j=\theta \left({\hat{\mathbf{\beta }}}_j-{\mathbf{\beta }}_0\right)+\left(1-\theta \right){\mathbf{z}}_{j-1},$$

(10)

where z₀ = 0 and θ(0 < θ < 1) is the smoothing parameter. Therefore, the chart statistic denotes by MEWMA _j is given by

$${\mathrm{MEWMA}}_j={\mathbf{z}}_j^{\text{'}}{{\displaystyle \sum }}_{\mathbf{Z}}{}^{-1}{\mathbf{z}}_j,$$

(11)

where ${\displaystyle {\sum }_{\mathbf{z}}=\frac{\theta }{2-\theta }{\displaystyle {\sum }_{\mathbf{\beta }}=\frac{\theta }{2-\theta }{\left({\mathbf{X}}_j^{\text{'}}{\mathbf{V}}_j^{-1}{\mathbf{X}}_j\right)}^{-1}}}$.

This control chart gives a signal when EWMA _j  > UCL, where (UCL > 0) is chosen to achieve a specified in control ARL.

The MCUSUM control chart

The third suggested method is based on the MCUSUM control chart which is proposed by Crosier (1988). In this method, the statistic is given by

$${\mathbf{s}}_j=\left\{\begin{array}{ll}0& \mathrm{for}{c}_j\le k\\ \left({\mathbf{s}}_{j-1}+{\hat{\mathbf{\beta }}}_j-{\mathbf{\beta }}_0\right)\left(1-k/c_j\right);& \mathrm{for}{c}_j>k\end{array}\right.,$$

(12)

where

$${\mathbf{c}}_j={\left[{\left({\mathbf{s}}_{j-1}+{\hat{\mathbf{\beta }}}_j-{\mathbf{\beta }}_0\right)}^{\text{'}}{{\displaystyle \sum }}^{-1}\left({\mathbf{s}}_{j-1}+{\hat{\mathbf{\beta }}}_j-{\mathbf{\beta }}_0\right)\right]}^{1/2}$$

and s₀ = 0 and k is a selected constant.

The estimator of variance covariance matrix is

$${\displaystyle {\sum }_{\mathbf{\beta }}={\left({\mathbf{X}}_j^{\text{'}}{\mathbf{V}}_j^{-1}{\mathbf{X}}_j\right)}^{-1}}.$$

(13)

The chart gives a signal if ${\left({\mathbf{s}}_j^{\text{'}}{{\displaystyle \sum }}^{-1}{\mathbf{s}}_j\right)}^{1/2}>\mathrm{UCL}$ where (UCL > 0) is chosen to achieve the desired in-control ARL.

Simulation studies

To show the performance of the proposed methods, we considered the underlying linear profile as Equation 14:

$$y_{\mathit{ij}}=3+b_{0j}+\left(2+b_{1j}\right)x_i+{ϵ}_{\mathit{ij}},$$

(14)

where

$${ϵ}_{\mathit{ij}}=\rho {ϵ}_{i-1j}+a_{\mathit{ij}},$$

(15)

and a _ij  ~ N(0,1),b_0j ~ N(0,.1),b_1j ~ N(0,.1). In our simulation investigation, we considered three significant different autocorrelation coefficients: a ρ = 0. 1 to designate a weak type auto-correlated process, intermediate autocorrelation by ρ = 0. 5, and strong autocorrelation by ρ = 0.9. The in-control ARL is roughly set equal to 200 and the ARL values were evaluated through 10,000 simulation replications under different shifts in intercept, slope, and errors (standard deviation). For MEWMA control chart, the smoothing parameter θ is chosen to be 0.2. As a general rule, to design MCUSUM control chart with the k approach, one chooses k to be half of the delta shift which is the amount of shift in the process that we wish to detect, expressed as a multiple of the standard deviation of the data points. Accordingly, we set k equal to 0.5 (for more detail see Montgomery 2005). UCLs of control charts are designed to achieve a specified in control ARL of 200. The simulated UCLs for each proposed control chart are shown in Table 1.

Table 1 Simulated UCL for control charts

The three proposed control charts are compared on different scenarios of the example in terms of ARL, and the calculated amounts for the different changes in the intercept is shown in Table 2.

Table 2 ARL comparisons under different λσ shift in intercept

According to the Table 2, under $\mathit{\lambda }$σ shift in the intercept, when autocorrelation is weak (ρ = 0.1), the MEWMA method performs relatively similar to MCUSUM control chart, and they also have better performance for detecting the small, moderate, and large shifts than the T² control chart. In the intermediate and strong autocorrelation circumstance (ρ = 0.5) and (ρ = 0.9), MCUSUM performs uniformly better than the other two methods. Moreover, MEWMA uniformly performs better than T² control chart. Figure 1 presented the derivative ARL under different shifts in intercept when autocorrelation is different in three levels. Table 3 shows the simulation results under different shifts in slope.

Figure 1

ARL comparisons under λσ shift in intercept with ρ = 0.1, 0.5, and 0.9.

Table 3 ARL comparisons under different β σ shift in slope

From Table 3, under βσ shift in slope, while the autocorrelation is weak (ρ = 0.1), the proposed MCUSUM method uniformly performs better than MEWMA method. Also, MEWMA performs consistently better than T² method. In addition, similar results are obtained when the autocorrelation is intermediate (ρ = 0.5). Once the amount of autocorrelation coefficient is high, MCUSUM and MEWMA methods perform uniformly better than the T² method and also, MCUSUM method performs relatively similar to MEWMA method. Figure 2 illustrates ARL under different shifts in slope once autocorrelation be changed in the aforementioned levels.

Figure 2

ARL comparisons under βσ shift in slope with ρ = 0.1, 0.5, and 0.9.

Next comparisons of the proposed three control charts in terms of ARL under δσ shift in the standard deviation followed. Table 4 shows that the proposed T² chart performs significantly better than MEWMA and MCUSUM charts in different amount of correlation coefficients. In addition for strong and intermediate autocorrelation condition, MEWMA and MCUSUM have similar manners and when the autocorrelation is weak, MEWMA relatively achieves better performance. Derivative ARL under different shifts of standard deviation is presented in Figure 3 when autocorrelation is different.

Figure 3

ARL comparisons under δσ shift of standard deviation with ρ = 0.1, 0.5, and 0.9.

Table 4 ARL comparisons under δσ shift in standard deviation

Based on the simulation results, it is evident that the proposed MEWMA and MCUSUM methods act relatively better than the T² chart in detecting shift in the parameters of profile; conversely, the proposed T² chart performs better than the MEWMA and MCUSUM in detecting shift in the variation.

Case study

Consider the case study carried out by Schabenberger and Pierce (2002). It was a real data set from ten apple trees which 25 apples are randomly chosen on each tree. Their focus was on the analysis of the apples in the largest size, with initial diameters exceeded 2.75 in. Totally there were 80 apples in aspiration size. Diameters of the apples were recorded in every 2 weeks during 12 weeks. Figure 4 shows 16 diameters out of 80 apples in the time domain. In their investigation, functional profile between time and diameter considered as quality characteristic that needs to be monitored over time. Schabenberger and Pierce (2002) and also later Soleimani et al. (2009) modeled such correlation between observations by a first-order autoregressive model of AR(1). Based on the preceding analysis, the following statements hold a linear mixed model equation for the declared case study:

$$y_{\mathit{ij}}=2.8321+b_{0j}+\left(0.02875+b_{1j}\right)x_i+{ϵ}_{\mathit{ij}},$$

(16)

$${ϵ}_{\mathit{ij}}=.3825{ϵ}_{i-1j}+a_{\mathit{ij}},$$

where b_0j ~ N(0,0.008653), b_1j ~ N(0,0.00005), and a_0j ~ N(0,0.000365).

Figure 4

Measured diameters of apples during a period of 12 weeks. (Schabenberger and Pierce 2002).

The estimator of variance-covariance matrix of fixed effects gives by

$${\displaystyle \sum \left[\begin{array}{ll}\hfill 1.4244\hfill & \hfill -.3034\hfill \\ \hfill -.3034\hfill & \hfill 0.0870\hfill \end{array}\right]}.$$

(17)

Consequences of simulation run in the previous section leads us to use MEWMA and MCUSUM which have relatively similar performance on detecting shift in the profile parameters rather than the T² method. Hence, the proposed MEWMA control chart was applied in monitoring the linear profile. The smoothing constant (θ) is set equal to 0.2. In order to achieve an in control ARL of 200, the upper control limit is set equal to 7 based on 10,000 simulation runs. In order to examine performance of the control chart, six random samples from the in control simple linear profile are initially generated. Formerly, three random samples are generated to show an out-of-control condition under the intercept shift coefficient of 0.6. Figure 5 illustrates sensitivity of the MEWMA control chart based on our proposed method which temperately depicts quick signal.

Figure 5

MEWMA control chart under intercept shift coefficient of 0.6.

Concluding remarks

We have studied the sensitivity of three multivariate control charts to detect one-step permanent shift in any parameters of a mixed model linear profile. Our specially designed MCUSUM, MEWMA, and T² control charts were also studied as competitors of each other to depict shifts in intercept and slope parameters and also process variation while first-order autoregressive model describes correlations within observations.

The performances of the methods were compared in terms of average run length criteria. Table 5 shows the summarized results.

Table 5 Comparisons on the performance of control charts under different shifts and autocorrelation status

The following summary recommendations are made:

• 1 The proposed approach has good performance across the range of possible shifts and it can be used in phase II of linear profile monitoring on the presence of autocorrelation within observations.

• 2 The anticipated MEWMA and MCUSUM methods almost uniformly perform better efficiency than the T ² Hotelling control chart under different step shifts in the intercept and slope parameters of linear profile.

• 3 The Hotelling T ² control chart has better performance in comparison with the MEWMA and MCUSUM methods under shifts in the process standard deviation.

• 4 The process circumstance in terms of correlation coefficient has no significant effects on selecting the best choice for monitoring method.

Authors’ information

SR (IRAN, Male, 1962). Ph.D., Associate Professor, worked at School of Industrial Engineering in Islamic Azad University, Tehran South Branch (IAU-STB). He obtained his Ph.D., Master and Bachelor degrees in Industrial Engineering. He was engaged in the industrial system engineering technology development and the technical consultant from 1988 up to year. He worked in different management positions both in private sector and university. The last one was deputy of research and planning in Islamic Azad University-Tehran South Branch. During his management period more than 10 scientific journals initiated and research activities facilitated. Currently, Dr. Raissi act as Editor-in-Chief of Journal of Industrial Engineering, International (JIEI) a Springer open journal and also has major responsibility on Ph.D. department of Industrial Engineering in IAU-STB. His main research fields are statistical quality control, quality engineering, system simulation and statistical methods in engineering. PS is an assistant professor of Industrial Engineering at IAU-STB. She especially focused on multivariate profile monitoring techniques under correlated process in her MS and PhD thesis. Her research interests are in the areas of Statistical Quality Control, Time Series, Multivariate Analysis and Metaheuristic algorithms. AN received his Master's of Science degrees in Industrial Engineering from IAU-STB. He especially focused on developing profile monitoring techniques as his academicals task under supervision of PS and SR. His research interests are in the areas of Time Series Analysis and Statistical Quality Control.