Keywords

1 Introduction

In geodetic quality control, statistical testing procedures often consist of two steps: detection and identification (Baarda 1968; Teunissen 1985; Caspary and Borutta 1987; Kösters and Van der Marel 1990; Amiri Simkooei 2001; Perfetti 2006; Lehmann and Lösler 2017; Klein et al. 2019; Nowel 2020). In the detection step, the validity of the null hypothesis \(\mathcal {H}_0\) is checked. If \(\mathcal {H}_0\) is rejected in the detection step, an identification is carried out as to which of the alternative hypotheses to select. In case there is only one alternative hypothesis, say \(\mathcal {H}_{1}\), the rejection of \(\mathcal {H}_{0}\) is equivalent to the selection of \(\mathcal {H}_{1}\). Thus, ‘correct detection’ of mismodelling error would be equivalent to ‘correct identification’ of it when working with a single alternative hypothesis. This is however not the case if one has to deal with multiple alternative hypotheses. In this contribution, for multiple-alternative testing, we study the performance of the detection and identification steps using the concepts of the minimal detectable bias (MDB) and the minimal identifiable bias (MIB), respectively, and highlight the factors driving the difference between them.

This contribution is structured as follows. In Sect. 2, we describe the null and alternative hypotheses, and highlight the role of the misclosure space partitioning in testing these hypotheses. The testing decisions and their probabilities are discussed, whereby the following events are defined: correct acceptance (CA), false alarm (FA), correct detection (CD), missed detection (MD), correct identification (CI) and wrong identification (WI). The concepts of MDB and MIB are discussed in Sect. 3 for a testing procedure comprising detection and identification steps. It is hereby highlighted that the MDB provides information about correct detection and not about correct identification. To provide insight into the difference between the MDB and the MIB, we compare them in Sect. 4, for a simple multiple-hypothesis testing example. It is demonstrated, in graphical form, that the MIB could be significantly larger than the MDB. The MDB-MIB comparison is then continued for actual deformation measurement system examples in Sect. 5. Finally a summary with conclusions is presented in Sect. 6.

We use the following notation: The n-dimensional space of real numbers is denoted as \(\mathbb {R}^{n}\), and the set of points on the circumference of the n-dimensional zero-centered unit sphere as \(\mathbb {S}^{n}\). Random vectors are indicated by use of the underlined symbol ‘\( \underline {\cdot }\)’. Thus \( \underline {t} \in \mathbb {R}^{n}\) is a random vector, while t is not. The squared weighted norm of a vector, with respect to a positive-definite matrix Q, is defined as \(\|\cdot \|{ }^{2}_{Q}=(\cdot )^{T}Q^{-1}(\cdot )\). \(\mathcal {H}\) is reserved for statistical hypotheses, \(\mathcal {P}\) for regions partitioning the misclosure space, and \(\mathcal {N}(x,Q)\) for the normal distribution with mean x and variance matrix Q. \(\mathsf {P}(\cdot )\) denotes the probability of the occurrence of the event within parentheses. The symbol \(\overset {\mathcal {H}}{\sim }\) should be read as ‘distributed as \(\ldots \) under \(\mathcal {H}\)’. The superscripts \(^{T}\) and \(^{-1}\) are used to denote the transpose and the inverse of a matrix.

2 Statistical Hypothesis Testing

In any quality control procedure, a set of hypotheses, including a null and several alternative hypotheses, are postulated to explain the phenomenon in question. For example, in geodetic deformation monitoring, the null hypothesis describes the ‘all-stable, no movement’ model, while the alternative hypotheses capture different dynamic behaviors of the structure under consideration. Let the observational model under the null hypothesis \(\mathcal {H}_{0}\), a.k.a. working hypothesis, be given as

$$\displaystyle \begin{aligned} {} \mathcal{H}_{0}:\quad \mathsf{E}(\underline{y})\;=\;Ax;\quad \mathsf{D}(\underline{y})\;=\;Q_{yy} \end{aligned} $$
(1)

with \(\mathsf {E}(\cdot )\) the expectation operator, \(\mathsf {D}(\cdot )\) the dispersion operator, \( \underline {y}\in \mathbb {R}^{m}\) the normally distributed random vector of observables linked to the estimable unknown parameters \(x\in \mathbb {R}^{n}\) through the design matrix \(A\in \mathbb {R}^{m\times n}\) of rank\((A)=n\), and \(Q_{yy}\in \mathbb {R}^{m\times m}\) the positive-definite variance matrix of \( \underline {y}\). The redundancy of \(\mathcal {H}_{0}\) is \(r = m -{\mathrm {rank(A)}} = m - n\).

The validity of the null hypothesis can be violated if the functional model and/or the stochastic model are misspecified. Here we assume that a misspecification is restricted to an underparametrization of the mean of \( \underline {y}\), which is the most common error that occurs when formulating the model (Teunissen 2017). Thus, the alternative hypothesis \(\mathcal {H}_{i}\) is formulated as

$$\displaystyle \begin{aligned} {} \mathcal{H}_{i}:\quad \mathsf{E}(\underline{y})\;=\;Ax\,+\,C_{i}b_{i};\quad \mathsf{D}(\underline{y})\;=\;Q_{yy} \end{aligned} $$
(2)

for some vector \(C_{i}b_{i}\in \mathbb {R}^{m}\setminus \{0\}\) such that \([A~~C_{i}]\) is a known matrix of full rank and \(b_i\) is an unknown vector.

2.1 Misclosure Space Partitioning

Let us assume that there are k types of mismodelling errors in the form of \(C_{i}b_{i}\) (cf. 2) when parametrizing the mean of observations. The information required to validate the hypotheses at hand is contained in the misclosure vector \( \underline {t}\in \mathbb {R}^{r}\) given as (Teunissen 2006)

$$\displaystyle \begin{aligned} {} \underline{t}\;=\;B^{T}\underline{y} \end{aligned} $$
(3)

where \(B\in \mathbb {R}^{m\times r}\) is a full-rank matrix, with rank\((B)=r\), such that \([A~~B]\in \mathbb {R}^{m\times m}\) is invertible and \(A^{T}B=0\). With \(C_{0}b_{0}=0\) and given that \( \underline {y}\overset {\mathcal {H}_{i}}{\sim }\mathcal {N}(Ax+C_{i}b_{i},Q_{yy})\) for \(i=0,1,\ldots ,k\), the misclosure vector is then distributed as

$$\displaystyle \begin{aligned} {} \underline{t}\overset{\mathcal{H}_{i}}{\sim}\mathcal{N}(C_{t_{i}}b_{i},Q_{tt}=B^{T}Q_{yy}B),\quad {\mathrm{for}}~~~i=0,1,\ldots k \end{aligned} $$
(4)

with \(C_{t_{i}}=B^{T}C_{i}\). As \( \underline {t}\) has a known Probability Density Function (PDF) under \(\mathcal {H}_{0}\), which is the PDF of \(\mathcal {N}(0, Q_{tt})\), any statistical testing procedure is driven by the misclosure vector \( \underline {t}\) and its known PDF under \(\mathcal {H}_{0}\).

An unambiguous testing procedure can be established through assigning the outcomes of \( \underline {t}\) to the statistical hypotheses \(\mathcal {H}_{i}\) for \(i=0,1,\ldots ,k\), which can be realized through a partitioning of the misclosure space \(\mathbb {R}^{r}\) (Teunissen 2018). Let \(\mathcal {P}_{i}\subset \mathbb {R}^{r}\) (\(i=0,1,\ldots ,k\)) be a partitioning of the misclosure space, i.e. \(\cup _{i=0}^{k}\,\mathcal {P}_{i}=\mathbb {R}^{r}\) and \(\mathcal {P}_{i}\cap \mathcal {P}_{j}=\emptyset \) for \(i\neq j\). The unambiguous testing procedure is then defined as

$$\displaystyle \begin{aligned} {} \begin{array}{lll} {{\mathrm{select}}~\mathcal{H}_{i}}&~{\mathrm{if}~\mathrm{and}~\mathrm{only}~\mathrm{if}}&~\underline{t}\in\mathcal{P}_{i}\quad {\mathrm{for}}~ i=0,1\ldots,k \end{array} \end{aligned} $$
(5)

We note, although in (5) the statistical testing is formulated in the misclosure vector \( \underline {t}\), that one can equally well work with the least-squares residual vector \(\hat { \underline {e}}_{0}= \underline {y}-A\hat { \underline {x}}_{0}\) where \(\hat { \underline {x}}_{0}=(A^{T}Q_{yy}^{-1}A)^{-1}A^{T}Q_{yy}^{-1} \underline {y}\). By using the relation \( \underline {t}=B^{T}\hat { \underline {e}}_{0}\), there is no explicit need of having to compute \( \underline {t}\) as testing can be expressed directly in \(\hat { \underline {e}}_{0}\) (Teunissen 2006).

2.2 Testing Decisions

As (5) shows, the testing decisions are driven by the outcome of the misclosure vector \( \underline {t}\). Under each hypothesis \(\mathcal {H}_{i}\) (\(i=0,1,\ldots ,k\)), the outcome of \( \underline {t}\) can lead to \(k+1\) different decisions out of which only one is correct, i.e. when \( \underline {t}\in \mathcal {P}_{i}\). With \(k+1\) hypotheses \(\mathcal {H}_{i}\)’s (\(i=0,1,\ldots ,k\)), one can define different statistical events including Correct Acceptance (CA), False Alarm (FA), Missed Detection (MD), Correct Detection (CD), Correct Identification (CI) and Wrong Identification (WI). The definitions of these events together with their links are illustrated in Fig. 1. In this figure, the events under alternative hypotheses are given an identifying index, as they differ from alternative to alternative. In addition, the contributions of different alternative hypotheses to the events of false alarm and wrong identification are distinguished by means of an index.

Fig. 1
figure 1

An overview of testing decisions, driven by the misclosure vector \( \underline {t}\), under null and alternative hypotheses

Full size image

Given the translational property of the PDF of \( \underline {t}\) under the null and alternative hypotheses (cf. 4), the probabilities of the events in Fig. 1 can be computed based on the misclosure PDF under \(\mathcal {H}_{0}\), denoted by \(f_{ \underline {t}}(\tau \vert \mathcal {H}_{0})\), as

(6)

The probability of false alarm \(\mathsf {P}_{\mathrm {FA}}\) is usually set a priori by the user. We note that the last four probabilities all depend on the unknown \(b_{i}\) which one needs to set to evaluate the mentioned four probabilities.

Here, it is important to note the difference between the probabilities of correct detection and correct identification, i.e. \(\mathsf {P}_{{\mathrm {CD}}_{i}}\ge \mathsf {P}_{{\mathrm {CI}}_{i}}\). These two probabilities would be identical if there is only one alternative hypothesis, say \(\mathcal {H}_{i}\), since then \(\mathcal {P}_{i}=\mathbb {R}^{r}\setminus \mathcal {P}_{0}\). Similar to the CD- and CI-probability, we have the concepts of the minimal detectable bias (MDB) (Baarda 1968) and the minimal identifiable bias (MIB) (Teunissen 2018). In the following sections, we highlight the difference between the MDB (\(\mathsf {P}_{{\mathrm {CD}}_{i}}\)) and the MIB (\(\mathsf {P}_{{\mathrm {CI}}_{i}}\)).

3 Testing Performance

Statistical testing procedures employed in quality control often comprises two steps (Baarda 1968; Teunissen 1985; Caspary and Borutta 1987; Kösters and Van der Marel 1990; Amiri Simkooei 2001; Perfetti 2006; Lehmann and Lösler 2017; Nowel 2020), as follows

  • Detection: The null hypothesis \(\mathcal {H}_{0}\) undergoes a validity check, without considering a particular set of alternatives.

  • Identification: If \(\mathcal {H}_{0}\) is rejected in the detection step, i.e. \( \underline {t}\notin \mathcal {P}_{0}\), a search is carried out among the specified alternatives \(\mathcal {H}_{i}\) (\(i=1,\ldots ,k\)) to pinpoint the potential source of model error.

The testing performance is thus not only led by its ability to detect biases but to correctly identify them as well. While the former is measured by means of the MDB (or alternatively CD-probability), the latter should be measured using the MIB (or alternatively CI-probability) (Teunissen 2018; Zaminpardaz and Teunissen 2019; Imparato et al. 2019). Note, in single-redundancy case \(r = 1\), that \(\mathcal {P}_{1}=\ldots = \mathcal {P}_{k}=\mathbb {R}^{r}\setminus \mathcal {P}_{0}\), implying that the alternative hypotheses are not distinguishable from one another, and thus identification would not be possible.

3.1 Minimal Detectable Bias (MDB)

The concept of the MDB was introduced in Baarda (1967, 1968) as a diagnostic tool for measuring the ability of the testing procedure to detect misspecifications of the model. The MDB, for each alternative hypothesis \(\mathcal {H}_{i}\), is defined as the smallest size of \(b_{i}\) that can be detected given a certain CD- and FA-probability. As the third equality in (6) shows, \(\mathsf {P}_{{\mathrm {CD}}_{i}}\) depends, in addition to the PDF of \( \underline {t}\) under \(\mathcal {H}_{0}\) and \(b_{i}\), also on \(\mathcal {P}_{0}\) which is commonly defined as (Baarda 1968; Teunissen 2006)

$$\displaystyle \begin{aligned} {} \begin{array}{lll} \mathcal{P}_{0}&=&\left\{t\in \mathbb{R}^{r}\vert\|t\|{}^{2}_{Q_{tt}}\le \chi^{2}_{1-\mathsf{P}_{\mathrm{FA}}}(r,0)\right\} \end{array} \end{aligned} $$
(7)

where \(\chi _{1-\mathsf {P}_{\mathrm {FA}}}^{2}(r,0)\) is the \((1-\mathsf {P}_{\mathrm {FA}})\) quantile of the central Chi-square distribution with r degrees of freedom. Using (7), one in fact compares the test statistic \(\| \underline {t}\|{ }^{2}_{Q_{tt}}\) against the critical value \(\chi ^{2}_{1-\mathsf {P}_{\mathrm {FA}}}(r,0)\), with user-defined \(\mathsf {P}_{\mathrm {FA}}\), to decide whether \(\mathcal {H}_{0}\) is valid or not. This testing process is called the overall model test, which would be a Uniformly Most Powerful Invariant (UMPI) detector test in case of dealing with a single alternative hypothesis (Arnold 1981; Teunissen 2006; Lehmann and Voß-Böhme 2017).

With (7), the CD-probability of \(\mathcal {H}_{i}\) is given by

$$\displaystyle \begin{aligned} \mathsf{P}_{{\mathrm{CD}}_{i}}=\mathsf{P}\left(\|\underline{t}\|{}^{2}_{Q_{tt}}> \chi^{2}_{1-\mathsf{P}_{\mathrm{FA}}}(r,0)\vert\mathcal{H}_{i}\right) \end{aligned} $$
(8)

where, according to (4), \(\| \underline {t}\|{ }^{2}_{Q_{tt}}\) under \(\mathcal {H}_{i}\) has a non-central Chi-square distribution with r degrees of freedom and the non-centrality parameter \(\lambda _{i}^{2}=\|C_{t_{i}}b_{i}\|{ }^{2}_{Q_{tt}}\). One can compute \(\lambda _{i}^{2}=\lambda ^{2}(\mathsf {P}_{\mathrm {FA}}, \mathsf {P}_{{\mathrm {CD}}_{i}},r)\) from the Chi-square distribution for a given model redundancy r, CD-probability \(\mathsf {P}_{{\mathrm {CD}}_{i}}\) and FA-probability \(\mathsf {P}_{\mathrm {FA}}\). If \(b_{i}\in \mathbb {R}\) is a scalar, then \(C_{t_{i}}\) takes the form of a vector \(c_{t_{i}}\), and the MDB is given by (Baarda 1968; Teunissen 2006)

$$\displaystyle \begin{aligned} {} b_{i}\in\mathbb{R}:~\vert b_{i,\mathrm{MDB}}\vert=\dfrac{\lambda(\mathsf{P}_{\mathrm{FA}}, \mathsf{P}_{{\mathrm{CD}}_{i}},r)}{\|c_{t_{i}}\|{}_{Q_{tt}}} \end{aligned} $$
(9)

which shows that for a given set of \(\{\mathsf {P}_{\mathrm {FA}}, \mathsf {P}_{{\mathrm {CD}}_{i}},r\}\), the MDB depends on \({\|c_{t_{i}}\|{ }_{Q_{tt}}}\). For the higher-dimensional case when \(b_{i}\in \mathbb {R}^{q>1}\) is a vector instead of a scalar, a similar expression can be obtained. Let the bias vector be parametrized, in terms of its magnitude \(\|b_{i}\|\) and its unit direction vector d, as \(b_{i}=\|b_{i}\|\,d\). Then the MDB along the direction \(d\in \mathbb {S}^{q-1}\) is given by (Teunissen 2006)

$$\displaystyle \begin{aligned}{} b_{i}\in\mathbb{R}^{q>1}:~\|b_{i,\mathrm{MDB}}(d)\|=\dfrac{\lambda(\mathsf{P}_{\mathrm{FA}}, \mathsf{P}_{{\mathrm{CD}}_{i}},r)}{\|C_{t_{i}}d\|{}_{Q_{tt}}};\;d\in\mathbb{S}^{q-1} \end{aligned} $$
(10)

If the unit vector d sweeps the surface of the unit sphere \(\mathbb {S}^{q-1}\), an ellipsoidal region is obtained of which the boundary defines the MDBs in different directions. The shape and the orientation of this ellipsoidal region is governed by the variance matrix \(Q_{\hat {b}_{i}\hat {b}_{i}}=(C^{T}_{t_{i}}Q_{tt}^{-1}C_{t_{i}})^{-1}\), and its size is determined by \(\lambda (\mathsf {P}_{\mathrm {FA}}, \mathsf {P}_{{\mathrm {CD}}_{i}},r)\) (Zaminpardaz et al. 2015; Zaminpardaz 2016).

The MDB concept expresses the sensitivity of the detection step of the testing procedure. One can compare the MDBs of different alternative hypotheses for a given set of \(\{\mathsf {P}_{\mathrm {FA}}, \mathsf {P}_{\mathrm {CD}},r\}\), which provides information on how sensitive is the rejection of \(\mathcal {H}_0\) for the \(\mathcal {H}_{i}\)-biases the size of their MDBs. The smaller the MDB is, the more sensitive is the rejection of \(\mathcal {H}_0\).

3.2 Minimal Identifiable Bias (MIB)

As the last equality in (6) shows, a high CD-probability \(\mathsf {P}_{{\mathrm {CD}}_{i}}\) does not necessarily imply a high CI-probability \(\mathsf {P}_{{\mathrm {CI}}_{i}}\) unless we have the special case of only a single alternative hypothesis. Therefore, in case of multiple hypotheses, the MDB does not provide information about correct identification. To assess the sensitivity of the identification step, one can analyse the MIBs of the alternative hypotheses. The MIB of the alternative hypothesis \(\mathcal {H}_{i}\) is defined as the smallest size of \(b_{i}\) that can be identified given a certain CI probability (Teunissen 2018).

The MIB corresponding with \(\mathcal {H}_{i}\) can be found from inverting the fifth equality in (6). This inversion is, however, not trivial as \(\mathsf {P}_{{\mathrm {CI}}_{i}}\) is an r-fold integral over the complex region \(\mathcal {P}_i\). One can take resort to numerical evaluation techniques. For example, the MIBs in Sect. 4 are numerically computed as follows. The probability \(\mathsf {P}_{{\mathrm {CI}}_{i}}\) is computed, by means of Monte Carlo simulation, see e.g. Teunissen (2018), at discrete biases \(b_{i}\) and then the bias at which \(\mathsf {P}_{{\mathrm {CI}}_{i}}\) gets close enough to the pre-set CI-probability is the MIB sought.

According to the fifth equality in (6), the MIB for a given \(\mathsf {P}_{{\mathrm {CI}}_{i}}\) depends on the probability mass of the PDF of \( \underline {t}\) under \(\mathcal {H}_{i}\) over \(\mathcal {P}_{i}\). This probability mass is driven by the shape and size of \(\mathcal {P}_{i}\), magnitude of \(\mathsf {E}\left ( \underline {t}\vert \mathcal {H}_{i}\right )\) and its direction with respect to the borders of \(\mathcal {P}_{i}\). Note, if \(b_{i}\in \mathbb {R}^{q>1}\) is a vector, then, a given CI-probability yields different MIBs along different directions in \(\mathbb {R}^{q}\). In this case, a pre-set CI-probability defines a region in \(\mathbb {R}^{q}\) the boundary of which defines the MIBs in different directions. The MIB of \(\mathcal {H}_{i}\) for a given CI-probability is denoted by \(\vert b_{i,{\mathrm {MIB}}}\vert \) if \(b_{i}\in \mathbb {R}\), and \(\|b_{i,{\mathrm {MIB}}}(d)\|\) along the unit direction \(d\in \mathbb {S}^{q-1}\) if \(b_{i}\in \mathbb {R}^{q>1}\).

4 MDB Versus MIB

As for a given bias \(b_{i}\), the CD-probability exceeds the CI-probability, i.e. \(\mathsf {P}_{{\mathrm {CD}}_{i}}\ge \mathsf {P}_{{\mathrm {CI}}_{i}}\), then for a given \(\mathsf {P}_{{\mathrm {CD}}_{i}}=\mathsf {P}_{{\mathrm {CI}}_{i}}\), we have

$$\displaystyle \begin{aligned} {} \begin{array}{lll} b_{i}\in\mathbb{R}&:&\vert b_{i,{\mathrm{MIB}}}\vert\ge \vert b_{i,{\mathrm{MDB}}}\vert\\ b_{i}\in\mathbb{R}^{q>1}&:&\|b_{i,{\mathrm{MIB}}}(d)\|\ge\|b_{i,{\mathrm{MDB}}}(d)\|~{\mathrm{for}~\mathrm{any}~}d\in\mathbb{S}^{q-1} \end{array} \end{aligned} $$
(11)

The following example elaborates more on the above link between the MDB and the MIB.

Example

Let \( \underline {y}\in \mathbb {R}^{4}\) contain two pairs of observations of an unknown distance \(x\in \mathbb {R}\) made using two different instruments, e.g., two different tape measures. The observations are assumed uncorrelated and equally precise with the same standard deviation \(\sigma \). Under the null hypothesis \(\mathcal {H}_{0}\), the observations are assumed to be bias-free, whereas under the alternative hypotheses \(\mathcal {H}_{i}\) (\(i=1,2\)), it is assumed that the observation pair made by one of the instruments are biased by \(C_{i}b_{i}\) (\(i=1,2\)) with \(C_{i}\in \mathbb {R}^{4\times 2}\) and \(b_{i}\in \mathbb {R}^{2}\). These hypotheses are formulated as

$$\displaystyle \begin{aligned} {} \begin{array}{lllll} \mathcal{H}_{0}&:&\mathsf{E}(\underline{y})=e_{4}\,x,&\mathsf{D}(\underline{y})=\sigma^{2}I_{4}\\ \mathcal{H}_{i}&:&\mathsf{E}(\underline{y})=e_{4}\,x+\left(u_{i}^{2}\otimes I_{2}\right)b_{i},&\mathsf{D}(\underline{y})=\sigma^{2}I_{4} \end{array} \end{aligned} $$
(12)

where \(\otimes \) shows the Kronecker product (Henderson and Pukelsheim 1983), \(e_{*}\in \mathbb {R}^{*}\) the vector of ones, \(I_{*}\in \mathbb {R}^{*\times *}\) the identity matrix, and \(u^{2}_{i}\in \mathbb {R}^{2}\) the canonical unit vector having one as its ith element and zeros otherwise.

The redundancy of \(\mathcal {H}_{0}\)-model is \(r=4-1=3>1\), which means, upon the rejection of \(\mathcal {H}_{0}\), that the identification of potential source of error would be possible. Under \(\mathcal {H}_{1}\), it is assumed that the mean-difference of the observables of the second instrument is zero, while under \(\mathcal {H}_{2}\), this is assumed for the first instrument. To test the three hypotheses in consideration, the following detection and identification steps are exercised:

  • Detection: The null hypothesis \(\mathcal {H}_{0}\) is accepted if \( \underline {t}\in \mathcal {P}_{0}\) with \(\mathcal {P}_{0}\) given by (7).

  • Identification: If \(\mathcal {H}_{0}\) is rejected in the detection step, then \(\mathcal {H}_{i}\) (\(i=1,2\)) is selected if \( \underline {t}\in \mathcal {P}_{i}\) with

    $$\displaystyle \begin{aligned} {} \mathcal{P}_{i}=\left\{t\in\mathbb{R}^{r}\setminus\mathcal{P}_{0}\middle\vert~T_{i}=\underset{j\in\{1,\ldots,k\}}{\max} T_{j}\right\} \end{aligned} $$
    (13)

    where

    $$\displaystyle \begin{aligned} {} T_{i}=t^{T}Q_{tt}^{-1}C_{t_{i}}\left(C_{t_{i}}^{T}Q_{tt}^{-1}C_{t_{i}}\right)^{-1}C_{t_{i}}^{T}Q_{tt}^{-1}t \end{aligned} $$
    (14)

    would be a realization of the Generalized Likelihood Ratio (GLR) test statistic in case there is only one single alternative hypothesis (Teunissen 2006).

We note that the vector of misclosures \( \underline {t}\) is not uniquely defined. This, however, does not affect the outcome of the above testing procedure as both the detector \(\| \underline {t}\|{ }^2_{Q_{tt}}\) and the test statistic \( \underline {T}_{i}\) remain invariant for any linear one-to-one transformation of the misclosure vector. Therefore, instead of \( \underline {t}\), one can for instance also work with

$$\displaystyle \begin{aligned} {} \bar{\underline{t}}\;=\;\mathcal{G}^{-T}\underline{t}\left\{\begin{array}{lll} \overset{\mathcal{H}_{0}}{\sim}\mathcal{N}(0,~I_{r})\\ \overset{\mathcal{H}_{i}}{\sim}\mathcal{N}(\bar{C}_{t_{i}}b_{i},~I_{r}) \end{array} \right. \end{aligned} $$
(15)

with \(\bar {C}_{t_{i}}=\mathcal {G}^{-T}C_{t_{i}}\) and the Cholesky-factor \(\mathcal {G}^{T}\) of the Cholesky-factorisation \(Q_{tt}=\mathcal {G}^{T}\mathcal {G}\). The advantage of using \(\bar { \underline {t}}\) over \( \underline {t}\) lies in the ease of visualizing certain effects due to the identity-variance matrix of \(\bar { \underline {t}}\) (Zaminpardaz and Teunissen 2019). The partitioning corresponding with \(\bar { \underline {t}}\) is denoted by \(\overline {\mathcal {P}}_{i}\) for \(i=0,1,2\).

The misclosure space (\(\mathbb {R}^{3}\)) partitioning corresponding with (7) and (13) is shown in Fig. 2. For the sake of visualization, instead of \( \underline {t}\), we work with \( \underline {\bar {t}}\) defined in (15). The blue sphere shows the boundary of \(\overline {\mathcal {P}}_{0}\) choosing \(\mathsf {P}_{\mathrm {FA}}=0.1\), while the green and red planes separate \(\overline {\mathcal {P}}_{1}\) from \(\overline {\mathcal {P}}_{2}\). The two planes are orthogonal to each other implying that \(\overline {\mathcal {P}}_{1}\) and \(\overline {\mathcal {P}}_{2}\) are the same in shape and size.

Fig. 2
figure 2

Partitioning of the misclosure space \( \mathbb {R}^{3}\) corresponding with \( \underline {\bar {t}}\) (15) using (7) and (13). The blue sphere shows the boundary of \(\overline {\mathcal {P}}_{0}\) with \(\mathsf {P}_{\mathrm {FA}}=0.1\), while the orthogonal green and red planes separate \(\overline {\mathcal {P}}_{1}\) from \(\overline {\mathcal {P}}_{2}\)

Full size image

As \(b_{i}\) in (12) is a 2-vector, i.e. \(b_{i}=[b_{i,1},~b_{i,2}]^{T}\), the MDBs and the MIBs of the alternative hypotheses are dependent not only on the pre-set CD- and CI-probability, but also on the bias direction in \(\mathbb {R}^{2}\). Figure 3 shows the MDB and MIB curves for \(\mathcal {H}_{i}\) (\(i=1,2\)) given \(\sigma =0.1\), \(\mathsf {P}_{\mathrm {FA}}=0.1\) and for different values of \(\mathsf {P}_{{\mathrm {CD}}_{i}}=\mathsf {P}_{{\mathrm {CI}}_{i}}\). In each panel, in agreement with (11), it can be seen that the MIB curve encompasses the MDB curve.

Fig. 3
figure 3

Illustration of the MDB versus the MIB curves for testing the hypotheses in (12) using (7) and (13), given \(\sigma =0.1\) and \(\mathsf {P}_{\mathrm {FA}}=0.1\). The panels from left to right correspond to \(\mathsf {P}_{{\mathrm {CD}}_{i}}=\mathsf {P}_{{\mathrm {CI}}_{i}}\) of 0.4, 0.6, 0.8 and 0.99, respectively

Full size image

Note, if \(\mathsf {E}(\bar { \underline {t}}\vert \mathcal {H}_{i})=\bar {C}_{t_{i}}b_{i}\) lies on the border of \(\overline {\mathcal {P}}_{1}\) and \(\overline {\mathcal {P}}_{2}\), that the CI-probability of \(\mathcal {H}_{i}\) cannot reach above \(0.5\). As shown in Fig. 2, the regions \(\overline {\mathcal {P}}_{1}\) and \(\overline {\mathcal {P}}_{2}\) are separated from each other by the following two planes

$$\displaystyle \begin{aligned} \bar{\tau}^{T}\left(\dfrac{\bar{C}_{t_{1}}^{\perp}}{\|\bar{C}_{t_{1}}^{\perp}\|}\pm \dfrac{\bar{C}_{t_{2}}^{\perp}}{\|\bar{C}_{t_{2}}^{\perp}\|}\right)=0;~\bar{\tau}\in\mathbb{R}^{3} \end{aligned} $$
(16)

with \(\bar {C}_{t_{i}}^{\perp }\in \mathbb {R}^{3}\) being a vector of which the range space is the orthogonal complement of the range space of \(\bar {C}_{t_{i}}\). It can be easily verified, if \(b_{i}\) is parallel to \([1,~1]^{T}\), that \(\mathsf {E}(\bar { \underline {t}}\vert \mathcal {H}_{i})\) will lie on the intersection of the above planes. This explains the bands around the direction of \([1,~1]^{T}\) in Fig. 3 when \(\mathsf {P}_{{\mathrm {CI}}_{i}}\) is set to be larger than \(0.5\). On the other hand, when \(b_{i}\) is parallel to \([1,~-1]^{T}\), the MDB and the MIB are very close to each other. A bias along the direction of \([1,~-1]^{T}\) makes \(\mathsf {E}(\bar { \underline {t}}\vert \mathcal {H}_{i})\) lie at its farthest position from the planar borders of \(\overline {\mathcal {P}}_{1}\) and \(\overline {\mathcal {P}}_{2}\). Thus, under \(\mathcal {H}_{i}\) (\(i=1,2\)), most of the probability mass of the PDF of \(\bar { \underline {t}}\) that lies outside \(\overline {\mathcal {P}}_{0}\) falls into the region \(\overline {\mathcal {P}}_{i}\). As a result \(\mathsf {P}_{{\mathrm {CD}}_{i}}\) and \(\mathsf {P}_{{\mathrm {CI}}_{i}}\) are very close to each other for a given bias along \([1,~-1]^{T}\), or alternatively the MDB and the MIB are very close to each other along \([1,~-1]^{T}\) for a pre-set \(\mathsf {P}_{{\mathrm {CD}}_{i}}=\mathsf {P}_{{\mathrm {CI}}_{i}}\). â–¡

The above example clearly shows that the detection and identification performance of a testing procedure could be completely different from each other.

5 Deformation Monitoring

In this section, we continue our MDB-MIB comparison for a dam deformation monitoring case, inspired by an example in Heunecke et al. (2013, p. 227), see also (Zaminpardaz et al. 2020). Figure 4 [top] shows a top view of a dam over a lake, together with two different 2-D terrestrial survey networks designed to monitor the dam’s horizontal displacement. For simplicity, it is assumed that the dam is vertically stable. The survey networks consist of two object points on the dam subject to displacement (points 5, 6), and four reference points in a stable area close to the dam (points 1, 2, 3, 4). To determine horizontal deformations of the dam, two sets of measurements are collected at two times (or epochs), \(l=1,2\).

Fig. 4
figure 4

Deformation monitoring of a dam (Zaminpardaz et al. 2020). [Top] The horizontal monitoring network consists of four reference points around the dam and two object points on the dam (points 5 and 6). The blue lines indicate the distance+direction measurements between their ending points, and the arrows point from total station to target. [Bottom] The graphs of MDB (solid lines) and MIB (dashed lines) of different alternative hypotheses in (18) as function of the pre-set probability. The results correspond with the testing procedure in (7) and (13), given \(\mathsf {P}_{\mathrm {FA}}=0.01\)

Full size image

In the survey network shown in Fig. 4 [top-left], each measurement set contains 60 measurements; five distance measurements and five direction measurements taken from each of the six points to the rest of the points by a total station. The distance and direction measurements are assumed to be normally distributed with standard deviations of 1 cm and 10 s of arc, respectively. The measurements are assumed to be all uncorrelated. To make the scale, orientation and location of the 2-D survey network estimable, the coordinates of the reference points 1 and 2 (black triangles in Fig. 4 [top]) are assumed given. The 60 distance and direction observations at epoch l are then used to estimate the Easting and Northing of points \(i=3,\ldots ,6\), together with the unknown instrument scale factor (one for the whole network) and six unknown orientations (one per instrument set-up).

To analyse the dam’s horizontal displacement, we make use of the epoch-wise estimated coordinatesof points \(i=3,\ldots ,6\) and their corresponding variance matrices. Let \(x_{i,l}\in \mathbb {R}^{2}\) (for \(i=3,\ldots ,6\) and \(l=1,2\)) be the coordinate vector of point i at epoch l, and let \(x_{l}=[x^{T}_{3,l},~x^{T}_{4,l},~x^{T}_{5,l},~x^{T}_{6,l}]^{T}\in \mathbb {R}^{8}\) for \(l=1,2\). Under the null hypothesis \(\mathcal {H}_{0}\), where deformation is absent, we assume

$$\displaystyle \begin{aligned} {} \mathcal{H}_{0}: x_{2}=x_{1}\;\;({\mathrm{all}\;\mathrm{stable}}) \end{aligned} $$
(17)

The redundancy under \(\mathcal {H}_{0}\) is \(r=8\). The dam is supposed to be subject to load of the water in the lake, and hence it is assumed that either only one or both of the dam points may be pushed back in the direction perpendicular to the dam. Thus we have three alternative hypotheses as

$$\displaystyle \begin{aligned}{} \begin{array}{lll} \mathcal{H}_{i}: x_{2}=x_{1}+(u^{4}_{i+2}\otimes d)\,b_{i}\;\; ({\mathrm{point}}\;i+4\;{\mathrm{is}\;\mathrm{unstable}},\;i=1,2)\\ \mathcal{H}_{3}: x_{2}=x_{1}+(u\otimes d)\,b_{3}\;\; ({\mathrm{points}}\;5\;{\mathrm{and}}\;6\;{\mathrm{are\;unstable}}) \end{array} \end{aligned} $$
(18)

with \(u^4_{i+2}\in \mathbb {R}^{4}\) the canonical unit vector having one as its \(({i+2})\)th element and zeros otherwise, \(u=u^4_{3}+u^4_{4}\), \(d\in \mathbb {S}\) the known unit vector in the direction perpendicular to the dam, and \(b_{i}\in \mathbb {R}\) the unknown scalar deformation size parameter. Note, under \(\mathcal {H}_{3}\), that we assume that the object points 5 and 6 deform with the same amount.

We note that since \(r=8>1\), our testing procedure involves both the detection and identification step (7) and (13). Assuming \(\mathsf {P}_{\mathrm {FA}}=0.01\), Fig. 4 [bottom-left] shows the MDB as a function of the CD-probability in solid curves, and the MIB as a function of the CI-probability in dashed curves for the three hypotheses in (18). For each hypothesis, its MIB graph lies above its MDB graph corroborating the first inequality in (11). For example, for a given pre-set probability of \(\mathsf {P}_{{\mathrm {CD}}_{i}}=\mathsf {P}_{{\mathrm {CI}}_{i}}=0.98\), there is an offset of almost 6mm between the MIB and the MDB in case of \(\mathcal {H}_{1}\) and \(\mathcal {H}_{3}\), while the \(\mathcal {H}_{2}\)’s MDB and MIB difference is at sub-mm level.

The MIB-MDB difference will change if the survey network measurement set-up changes. Figure 4 [top-right] shows a survey network obtained by removing 17 pairs of distance/direction measurements from the top-left network. As a result of loosing 34 measurements compared to the previous survey network, both the MDBs and the MIBs increase as shown in Fig. 4 [bottom-right]. It is observed that the MIB and the MDB can differ significantly from each other. For example, for a given pre-set probability of \(\mathsf {P}_{{\mathrm {CD}}_{i}}=\mathsf {P}_{{\mathrm {CI}}_{i}}=0.98\), there is an offset of almost 16mm between the MIB and the MDB in case of \(\mathcal {H}_{1}\) and \(\mathcal {H}_{3}\).

As shown in Fig. 4 [bottom], the MDB and the MIB, for a pre-set probability, differ from hypothesis to hypothesis. For example, for the range of probabilities shown in Fig. 4 [bottom-left], it is observed that

$$\displaystyle \begin{aligned} {} \begin{array}{llll} \vert b_{2,{\mathrm{MDB}}}\vert>\vert b_{3,{\mathrm{MDB}}}\vert>\vert b_{1,{\mathrm{MDB}}}\vert\\ \vert b_{2,{\mathrm{MIB}}}\vert>\vert b_{3,{\mathrm{MIB}}}\vert>\vert b_{1,{\mathrm{MIB}}}\vert \end{array} \end{aligned} $$
(19)

As the MDB, for a given set of \(\left \{\mathsf {P}_{\mathrm {FA}},\mathsf {P}_{{\mathrm {CD}}_{i}},r\right \}\), is driven by \(\|c_{t_{i}}\|{ }_{Q_{tt}}\), the first expression in the above equation can be explained by comparing \(\|c_{t_{i}}\|{ }_{Q_{tt}}\) for \(i=1,2,3\). The larger the value of \(\|c_{t_{i}}\|{ }_{Q_{tt}}\), the smaller the MDB is expected to be. For example, for the survey network shown in Fig. 4 [top-left], we have

$$\displaystyle \begin{aligned} {} \begin{array}{llll} \|c_{t_{1}}\|{}_{Q_{tt}}\approx 180;~\|c_{t_{2}}\|{}_{Q_{tt}}\approx 105 ;~\|c_{t_{3}}\|{}_{Q_{tt}}\approx 158 \end{array} \end{aligned} $$
(20)

which are driven by the network geometry, measurement precision and the direction of displacement. The above equation implies that \(\mathcal {H}_{1}\) and \(\mathcal {H}_{2}\) should, respectively, have the smallest and the largest MDBs among the three alternatives for a pre-set CD-probability. The MIB inequalities in (19) are due to a combination of (20), the shape and size of \(\mathcal {P}_{i}\), magnitude of \(\mathsf {E}( \underline {t}\vert \mathcal {H}_{i})\) and its direction with respect to the borders of \(\mathcal {P}_{i}\).

6 Summary and Concluding Remarks

In this contribution, a comparative analysis was provided of the detection and identification steps of statistical testing procedures. The detection step aims to validate the null hypothesis \(\mathcal {H}_{0}\), while the identification step, upon the rejection of \(\mathcal {H}_{0}\), aims to select the most likely alternative hypothesis among those in consideration.In case there is only one alternative hypothesis, say \(\mathcal {H}_{1}\), the rejection of \(\mathcal {H}_{0}\) is equivalent to the identification of \(\mathcal {H}_{1}\). This is however not the case when working with multiple alternatives. Having different functionalities, the detection and identification performance of the testing procedure should then be assessed using two different diagnostic tools. The detection capability of a testing regime is usually assessed by its Minimal Detectable Bias (MDB), whereas the testing identification performance should be evaluated by its Minimal Identifiable Bias (MIB).

Using the concept of misclosure space partitioning, we discussed testing decisions and their probabilities. Through this partitioning, it was shown that the distribution of the misclosure vector can be used to determine the correct detection (CD) and correct identification (CI) probabilities of each of the alternative hypotheses. One can then ‘invert’ these probabilities to determine their corresponding minimal biases, i.e. the MDB and the MIB. It was highlighted that a small MDB (or high probability of correct detection) does not necessarily imply a small MIB (or a high probability of correct identification), unless one is dealing with the special case of having only one single alternative hypothesis. The factors driving the difference between detection and identification performance were illustrated using a simple multiple-alternative testing example. Our evaluations were extended to basic deformation measurement system examples with multiple alternative hypotheses, where monitoring measurements were provided by a 2D terrestrial survey network.