Estimating multivariate linear profiles change point with a monotonic change in the mean of response variables

Abstract

In this paper, a maximum likelihood estimator (MLE) is developed to estimate change point when monotonic change occurs in the mean of response variables in multivariate linear profiles in Phase II. Performance of the proposed estimator is compared to the performance of step change and linear drift estimators under different shift types. To conduct comparisons, accuracy and precision of the estimators are considered as performance measures. Simulation results show that the average change point estimate of the proposed estimator is less biased than the one for the step and drift estimators in small shifts, because ${\overline{\widehat{\tau}}}_{\mathrm{monotonic}}$ is closer to the actual change point of 25 in small shifts. Also, the precision of the proposed estimator is approximately better than that of the step and drift estimators, because its precision values are higher. Hence, the proposed estimator has better performance in terms of both accuracy and precision in small shifts under any kinds of increasing changes. In single step and linear drift changes when the magnitude of shifts increases, the accuracy and precision of their corresponding estimators become better than the accuracy and precision of the proposed estimator. However, the proposed estimator has an advantage that it does not require assumptions about the change type, and its only assumption is that the mean of the response variables changes in an increasing manner. Additional evaluations on the effect of smoothing constant show that with smaller values of the smoothing constant, the proposed change point estimator has less biased estimates and smaller values of mean square error in small shifts rather than the step and drift estimators, leading to a better performance. Also, the larger values of smoothing constant lead to the better performance of the monotonic estimator in large shifts. Finally, the application of the proposed estimator is shown through a real case in the calibration process in the automotive industry.

Appendices

Appendix 1: Derivation of the linear drift estimator

The logarithm of likelihood function for drift estimator is as follows (see Kazemzadeh et al. [38] for more details):

$$ \begin{array}{l} Ln\left( L\left(\tau, \mathbf{K}\Big|\mathbf{Y},\mathbf{X}\right)\right)= U-\frac{1}{2}{\displaystyle \sum_{j=1}^{\tau} tr\left[\left({\mathbf{Y}}_{\mathrm{j}}-\mathbf{XB}\right){\displaystyle {\sum}^{-1}{\left({\mathbf{Y}}_{\mathrm{j}}-\mathbf{XB}\right)}^{\mathrm{T}}}\right]}\hfill \\ {}-\frac{1}{2}{\displaystyle \sum_{j=\tau +1}^T tr\left[\left[{\mathbf{Y}}_{\mathrm{j}}-\mathbf{XB}-\left( j-\tau \right)\mathbf{XK}\right]{\displaystyle {\sum}^{-1}{\left[{\mathbf{Y}}_{\mathrm{j}}-\mathbf{XB}-\left( j-\tau \right)\mathbf{XK}\right]}^{\mathrm{T}}}\right]}\hfill \end{array} $$

(23)

where U is a constant value and tr[W] is the trace of the matrix of W. Taking a derivative of the aforementioned function with respect to the matrix of XK to estimate the slope of the changes and solving for XK leads to

$$ \frac{\partial Ln\left( L\left(\tau, \mathbf{K}\Big|\mathbf{Y},\mathbf{X}\right)\right)}{\partial \mathbf{XK}}={\displaystyle \sum_{j=\tau +1}^T\left[\left( j-\tau \right)\left({\mathbf{Y}}_j-\mathbf{XB}-\left( j-\tau \right)\mathbf{XK}\right){{\displaystyle \sum}}^{-1}\right]} $$

(24)

$$ \frac{\partial Ln\left( L\left(\tau, \mathbf{K}\Big|\mathbf{Y},\mathbf{X}\right)\right)}{\partial \mathbf{XK}}=0,\kern0.5em so\kern0.5em \widehat{\mathbf{XK}}=\frac{\left[{\displaystyle {\sum}_{j=\tau +1}^T}\left( j-\tau \right)\left({\mathbf{Y}}_j-\mathbf{XB}\right)\right]}{{\displaystyle {\sum}_{j=\tau +1}^T}{\left( j-\tau \right)}^2} $$

(25)

Finally, the change point estimator is as follows:

$$ {\widehat{\tau}}_{\boldsymbol{LD}}=\underset{0\le t\le T-1}{ \arg \kern0.5em \max}\left\{ - \frac{1}{2}{\displaystyle \sum_{j=1}^t} tr\left[\left({\mathbf{Y}}_j-\mathbf{XB}\right){{\displaystyle \sum}}^{-1}{\left({\mathbf{Y}}_j-\mathbf{XB}\right)}^{\mathrm{T}}\right]-\frac{1}{2}{\displaystyle \sum_{j= t+1}^T} tr\left[\left[{\mathbf{Y}}_j-\mathbf{XB}-\left( j-\tau \right)\hat{\mathbf{XK}}\right]{{\displaystyle \sum}}^{-1}{\left[{\mathbf{Y}}_j-\mathbf{XB}-\left( j-\tau \right)\widehat{\mathbf{XK}}\right]}^{\mathrm{T}}\right]\right\} $$

(26)

For the special cases of multivariate linear profiles, i.e., multiple and simple linear profiles, the only difference is that in the above equations, matrixes of Y _j , B, and K reduce to vectors of y _j , β, and k, respectively.

Appendix 2: Derivation of the step change estimator

The logarithm of the likelihood function with the assumption of step change yields

$$ Ln\left( L\left(\tau, {\mathbf{B}}_1\Big|\mathbf{Y},\mathbf{X}\right)\right)= U-\frac{1}{2}{\displaystyle \sum_{j=1}^{\tau}} tr\left[\left({\mathbf{Y}}_j-\mathbf{XB}\right){\boldsymbol{\Sigma}}^{-1}{\left({\mathbf{Y}}_j-\mathbf{XB}\right)}^{\mathrm{T}}\right]-\frac{1}{2}{\displaystyle \sum_{j=\tau +1}^T} tr\left[\left[{\mathbf{Y}}_j-{\left(\mathbf{XB}\right)}_1\right]{\boldsymbol{\Sigma}}^{-1}{\left[{\mathbf{Y}}_j-{\left(\mathbf{XB}\right)}_1\right]}^{\mathrm{T}}\right] $$

(27)

U is constant, so the derivative of the logarithm of the likelihood function with respect to (XB)₁ is

$$ \frac{\partial Ln\left( L\left(\tau, {\mathbf{B}}_1\Big|\mathbf{Y},\mathbf{X}\right)\right)}{\partial {\left(\mathbf{XB}\right)}_1}={\displaystyle \sum_{j=\tau +1}^T}\left[\left({\mathbf{Y}}_j-{\left(\mathbf{XB}\right)}_1\right){\boldsymbol{\Sigma}}^{-1}\right] $$

(28)

Also, the maximum likelihood estimator of (XB)₁ is

$$ {\left(\widehat{\mathbf{XB}}\right)}_1=\frac{\left[{\displaystyle {\sum}_{j=\tau +1}^T}{\mathbf{Y}}_j\right]}{\left( T-\tau \right)} $$

(29)

$$ {\widehat{\tau}}_{\boldsymbol{SC}}= \arg \underset{0\le t\le T-1}{ \max}\left\{ - \frac{1}{2}{\displaystyle \sum_{j=1}^t} tr\left[\left({\mathbf{Y}}_j-\mathbf{XB}\right){{\displaystyle \sum}}^{-1}{\left({\mathbf{Y}}_j-\mathbf{XB}\right)}^{\mathrm{T}}\right]-\frac{1}{2}{\displaystyle \sum_{j= t+1}^T} tr\left[\left[{\mathbf{Y}}_j-{\left(\widehat{\mathbf{XB}}\right)}_1\right]{{\displaystyle \sum}}^{-1}{\left[{\mathbf{Y}}_j-{\left(\widehat{\mathbf{XB}}\right)}_1\right]}^{\mathrm{T}}\right]\right\} $$

(30)

For multiple and simple linear profiles, the matrixes of Y _j and B reduce to vectors of y _j and β, respectively.

Appendix 3: Computations of the proposed change point estimators

Table 8 Computations of change point estimators for torqometer calibration case study at Irankhodro Corporation. The first out-of-control sample occurs on the 26th sample (τ ₁ = 25)

Appendix 4

Table 9 Data of torqometer calibration case study at Irankhodro Corporation