Let us do the twist again

Krämer (Sankhy$$\bar{\mathrm{a }}$$ 42:130–131, 1980) posed the following problem: “Which are the $$\mathbf{y}$$, given $$\mathbf{X}$$ and $$\mathbf{V}$$, such that OLS and Gauss–Markov are equal?”. In other words, the problem aimed at identifying those vectors $$\mathbf{y}$$ for which the ordinary least squares (OLS) and Gauss–Markov estimates of the parameter vector $$\varvec{\beta }$$ coincide under the general Gauss–Markov model $$\mathbf{y} = \mathbf{X} \varvec{\beta } + \mathbf{u}$$. The problem was later called a “twist” to Kruskal’s Theorem, which provides conditions necessary and sufficient for the OLS and Gauss–Markov estimates of $$\varvec{\beta }$$ to be equal. The present paper focuses on a similar problem to the one posed by Krämer in the aforementioned paper. However, instead of the estimation of $$\varvec{\beta }$$, we consider the estimation of the systematic part $$\mathbf{X} \varvec{\beta }$$, which is a natural consequence of relaxing the assumption that $$\mathbf{X}$$ and $$\mathbf{V}$$ are of full (column) rank made by Krämer. Further results, dealing with the Euclidean distance between the best linear unbiased estimator (BLUE) and the ordinary least squares estimator (OLSE) of $$\mathbf{X} \varvec{\beta }$$, as well as with an equality between BLUE and OLSE are also provided. The calculations are mostly based on a joint partitioned representation of a pair of orthogonal projectors.


Introduction
Let us consider the general Gauss-Markov model where y is an n × 1 observable random vector, X is a known n × p model matrix, β is a p × 1 vector of unknown parameters, and u is an n × 1 random error vector. The expectation vector and the covariance matrix of u are E(u) = 0 and Cov(u) = σ 2 V, respectively, where σ 2 > 0 is an unknown constant and V is a known n×n nonnegative definite matrix. Both X and V may be rank deficient. It is assumed beforehand that the model (1) is consistent, i.e., y ∈ R(X : V), where R(.) stands for the column space of a matrix argument and (X : V) denotes the n × ( p + n) columnwise partitioned matrix obtained by juxtaposing matrices X and V; cf. Rao (1973, p. 297) or Puntanen et al. (2011, pp. 43, 125). In his paper, Krämer (1980, p. 130) posed the following problem: "Which are the y, given X and V, such that ordinary least squares (OLS) and Gauss-Markov are equal?" In other words, the problem aimed at identifying those vectors y for which the OLS and Gauss-Markov estimates of the parameter vector β coincide. Referring to this problem, in a follow-up paper Krämer et al. (1996) called this a "twist" to Kruskal's Theorem (Kruskal 1968), which provides conditions necessary and sufficient for the OLS and Gauss-Markov estimates of β to be equal. In Krämer et al. (1996) "another twist" to Kruskal's Theorem is dealt with, and rather than asking when is the OLS equal to the Gauss-Markov for the full regression vector β, a condition for the equality of the OLS and Gauss-Markov for a subparameter of β is provided. A more general "final twist" is considered in Jaeger and Krämer (1998), where the single vectors y are characterized that yield identical OLS and Gauss-Markov estimators for such a subparameter.
Inspired by Jaeger and Krämer (1998), Krämer (1980), and Krämer et al. (1996), in what follows we "do the twist again". However, unlike in the three papers, we do not assume that X and V are of full (column) rank, which means that the vector β is not necessarily unbiasedly estimable. For this reason, instead of the estimation of β, we consider the estimation of the systematic part E(y) = Xβ. Note that this parameter function always has a linear unbiased estimator, namely y itself.
An important role in the subsequent considerations will be played by the notion of a projector. It is known that any n × n idempotent matrix, say F ∈ R n×n , is an oblique projector onto its column space R(F) along its null space N (F), where R(F) ⊕ N (F) = R n,1 . Among many conditions characterizing idempotent matrices one finds for instance: When idempotent F projects onto R(F) along the orthogonal complement of R(F), then it is called an orthogonal projector. It can be verified that F is an orthogonal projector if and only if it is both idempotent and symmetric, i.e., F 2 = F = F . Projectors are widely used in Statistics and Econometrics as a basic tool for estimation and test procedures.
Let G be an n × n matrix. An estimator Gy for Xβ, which is unbiased and of minimal covariance matrix in the Löwner sense fulfills the conditions where H = XX † and M = I n − H are the orthogonal projectors onto, respectively, R(X), the column space of X, and the orthogonal complement of R(X) which coincides with N (X ), the null space of X . The symbol X † denotes the Moore-Penrose inverse of X. The conditions (2) can be rewritten as where P VM = VM(VM) † is the orthogonal projector onto R(VM). It was pointed out in Baksalary and Trenkler (2012, Remark 3.1) that Eq. (3) always has a solution G and that each G satisfying (3) yields a representation of the best linear unbiased estimator BLUE(Xβ) of Xβ. All these representations coincide; see Groß (2004, Corollary 3).
In the Appendix given below it is demonstrated that there may exist, however, quite useless versions of BLUE(Xβ). To avoid this discrepancy subsequently we strengthen the consistency condition y ∈ R(X : V) to Then, according to Groß (2004, Corollary 4), BLUE(Xβ) is unique.
In the next section some representations of the best linear unbiased estimator (BLUE) and the ordinary least squares estimator (OLSE) are provided, whereas Sect. 3 deals with "another twist" to Kruskal's Theorem, which was briefly mentioned above. Section 4 is concerned with bounds for the Euclidean distance between BLUE and OLSE of Xβ, and the last section of the paper revisits the problem of when BLUE equals OLSE.

Representations of BLUE and OLSE
Let P be an orthogonal projector in R n,1 , i.e., an n × n real symmetric idempotent matrix. Assume that the rank of P is r. It is known that there exists an orthogonal matrix U such that see Trenkler (1994, Theorem 13). Any other orthogonal projector of the same size, say Q ∈ R n×n , can be represented as with symmetric matrices A and D of orders r and n − r , respectively.
In what follows we assume that H = XX † is represented by P of the form (5) and P VM is represented by Q defined in (6), i.e., P = H = XX † and Q = P VM = VM(VM) † . It can be verified that T = (P VM H) † , where P VM = I n − P VM , is an idempotent matrix; see Greville (1974, p. 830). From (5) and (6) we obtain indicates that the two subspaces involved in the direct sum are orthogonal; see Baksalary and Trenkler (2010, Theorem 2). From (see Baksalary and Trenkler 2009, Theorem 1), we arrive at N (H) + N (P VM ) = R n,1 , which leads to the conclusion that T is the oblique projector onto R(H) along Furthermore, it follows that T takes the form see Baksalary and Trenkler (2010, Sect. 2). It is well known that (7) ensures that Eq. (3) is solvable. One of the solutions, namely T = (P VM H) † , gives a representation of the BLUE for Xβ, i.e., BLUE(Xβ) = Ty. There is a number of further expressions for the BLUE (see Baksalary and Trenkler 2009, Sect. 4), but they all coincide by the assumption (4). Observe that the OLSE of Xβ is OLSE(Xβ) = Hy.

Another twist
As in Krämer (1980), we consider the problem of identifying those observation vectors y which yield the same value of OLSE(Xβ) and BLUE(Xβ). This amounts to an analysis of the subspace L of R n,1 which is the null space of H − T, i.e., L = N (H − T). For this purpose the following result is useful.
Lemma 1 Let R and S be idempotent matrices of the same size. Then: Proof For a proof see Baksalary and Trenkler (2013, Theorems 1 and 9).
Lemma 1 leads to the following result. Another relevant fact is that with H of the form (5) and T given in (8) This discrepancy can be explained on account of the identity following from Lemma 1. The subspace of Theorem 2 coincides with (9)  Corollary 1 corresponds to Krämer's (1980, Theorem), where the identity BLUE(β) = OLSE(β) is explored under the assumption that X and V are of full (column) rank.
Recall that the projector P introduced in (5) was determined by the model matrix X, for P = H = XX † . In consequence, rank of P coincides with the ranks of H and X, i.e., r = rank(H) = rank(X). Consider now an oblique projector of rank r having the form with K ∈ R r ×n−r . It was shown by Baksalary and Trenkler (2011, Sect. 3) that when K = −W 12 (DW 22 D) † , where D ∈ R n−r ×n−r is a symmetric idempotent matrix and W 12 ∈ R r ×n−r and W 22 ∈ R n−r ×n−r originate from the representation of V given by then Ly is an unbiased estimator of Xβ whose efficiency lies between that of BLUE(Xβ) and OLSE(Xβ). In what follows we identify those observation vectors y which yield the same estimators, compared to BLUE(Xβ) and OLSE(Xβ). The resulting formulas give an impression how close the three estimators can be.
Theorem 3 Under the model (1) On the other hand, from Theorem 1 we have N (T − L) = N (TL), which completes the proof.
It is seen from Theorem 4 that μ * =μ for all y if and only if BD † = 0, or, equivalently, B = 0, which means that H and P VM commute.

Equality of BLUE and OLSE
The commutativity HP VM = P VM H, just mentioned in the preceding section, is not contained in the standard catalogue of conditions necessary and sufficient for the equality BLUE(Xβ) = OLSE(Xβ). Among the most important conditions equivalent to the equality are: Note that the condition (v) can be rewritten as R(VX) = R(X) when V is nonsingular; see Krämer (1980) for a discussion related to Kruskal's theorem and Puntanen et al. (2011, Proposition 10.1). Motivated by Theorem 4 we obtain the following result. Proof For the proof of the equivalence (i) ⇔ (ii) see Puntanen et al. (2011, Proposition 10.1).
The fact that (iii) ⇒ (v) is visibly seen by taking the transpose of HP VM = 0. For the proof of the reverse implication, recall that BLUE(Xβ) exists if and only if Eq. (7) are satisfied. By condition (v) we have HP VM x = P VM Hx for any vector x ∈ R n,1 . Thus, x ∈ R(H) ∩ R(P VM ), whence HP VM x = 0 for any x, i.e., HP VM = 0.
The part (v) ⇔ (vi) is also known in the literature; see e.g., Baksalary et al. (2002, Theorem). Krämer (1980) showed how his theorem characterizing the vectors y ensuring the coincidence of BLUE(β) and OLSE(β) can be used to prove Kruskal's Theorem. This is done in a similar fashion in the present set-up. The part (i) ⇒ (ii) is established in a similar fashion by reversing the preceding chain.
From the discussion preceding Corollary 1 it follows that N (HP VM ), specified in (10), coincides with R(P VM ) when (4) holds. In such a case, the condition (ii) of Theorem 6 reduces to N (H) = R(P VM ) ∩ N (H), or, equivalently, to R(P VM ) ⊆ N (H), i.e., R(VM) ⊆ R(M). When V is nonsingular, we get R(VM) = R(M), which is the final condition of Kruskal's Theorem in Krämer (1980). Another observation is that the conditions of Theorems 5 and 6, unlike the customary conditions given on the top of the present section, predominantly deal with orthogonal projectors. Thus, the equality of BLUE(Xβ) and OLSE(Xβ) is characterized in a more symmetric way.
When P VM has representation (6), then we get the following equivalences among the statements of Theorem 5: see Baksalary and Trenkler (2008).
Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.