A New Look at the Models for Ordinal Categorical Data Analysis

Abstract

The multinomial/categorical responses, whether they are nominal or ordinal, are recorded in counts under all categories/cells involved. The analysis of this type of multinomial data is traditionally done by exploiting the marginal cell probabilities-based likelihood function. As opposed to the nominal setup, the computation of the marginal probabilities is not easy in the ordinal setup. However, as the ordinal responses in practice are interpreted by collapsing multiple categories either to binary data using a single cut-point or to tri-nomial data using two cut-points, most of the studies over the last four decades, first modeled the associated cumulative probabilities using a suitable such as logit, probit, log-log, or complementary log-log link function. Next the marginal cell probabilities were computed by subtraction, in order to construct the desired estimating function such as moment or likelihood function. In this paper we take a new look at this ordinal categorical data analysis problem. As opposed to the existing studies, we first model the ordinal categories using a multinomial logistic marginal approach by pretending that the adjacent categories are nominal, and then construct the cumulative probabilities to develop the final model for ordinal responses. For inferences, we develop the cut points based likelihood or generalized quasi-likelihood (GQL) estimating functions for the purpose of the estimation of the underlying regression parameters. The new GQL estimation approach is developed in details by utilizing both tri-nomial and binary (or binomial) collapsed structures. The likelihood analysis is also discussed. A data example is given to illustrate the proposed models and the estimation methodologies. Furthermore, we also examine the asymptotic properties of the likelihood and GQL estimators for the regression parameters for both tri-nomial and binary types of cumulative response based models.

Appendices

Appendix A: GQL Estimation Details for Both Tri-nomial and Binary Types Ordinal Models

1.1 A1. GQL Estimation aids for tri-nomial type ordinal model

In (3.3), the observations S_[ℓ]j for j = 1 and j = J are scalar, and their means and variances are given by

$$\begin{array}{@{}rcl@{}} E[S_{[\ell]1}]&=&E[K_{[\ell]1}]=K_{[\ell]}\pi_{[\ell]1}, \text{var}[S_{[\ell]1}]=K_{[\ell]}\pi_{[\ell]1}[1-\pi_{[\ell]1}], \end{array} $$

(a.1)

and

$$\begin{array}{@{}rcl@{}} E[S_{[\ell]J}]&=&E[K_{[\ell]J}]=K_{[\ell]}\pi_{[\ell]J}, \text{var}[S_{[\ell]J}]=K_{[\ell]}\pi_{[\ell]J}[1-\pi_{[\ell]J}], \end{array} $$

(a.2)

respectively. However, for j = 2,…,J − 1, the two components of the trinomial observation, namely C_[ℓ]j and K_[ℓ]j are correlated. More specifically, cov[C_[ℓ]j,K_[ℓ]j] = −K_[ℓ]F_{[ℓ](j− 1)}π_[ℓ]j. Hence, for j = 2,…,J − 1, we obtain

$$\begin{array}{@{}rcl@{}} E[S_{[\ell],j}]&=&[\mu_{[\ell]j,1},\mu_{[\ell]j,2}]^{\prime} =K_{[\ell]}[F_{[\ell](j-1)},\pi_{[\ell]j}]^{\prime} =K_{[\ell]}\tilde{\mu}_{[\ell]j}(\beta)=\mu_{[\ell]j}({\beta}) \\ \text{cov}[S_{[\ell],j}]&=&{\Sigma}_{[\ell]j}({\beta})= \left( \begin{array}{ll}\sigma_{[\ell]j,11} & \sigma_{[\ell]j,12} \\ \sigma_{[\ell]j,21} & \sigma_{[\ell]j,22} \end{array}\right)=K_{[\ell]}\tilde{\Sigma}_{[\ell]j}({\beta}) \\ &=&K_{[\ell]}\left( \begin{array}{cl}F_{[\ell](j-1)}(1- F_{[\ell](j-1)})& -F_{[\ell](j-1)}\pi_{[\ell]j} \\ -F_{[\ell](j-1)}\pi_{[\ell]j} &\pi_{[\ell]j}(1-\pi_{[\ell]j}) \end{array}\right). \end{array} $$

(a.3)

For the computational purpose, the GQL estimating equation in (3.3) may then be written as

$$\begin{array}{@{}rcl@{}} G_{1}({\beta})\!\!&=&\!\!\sum\limits^{p + 1}_{\ell= 1}K_{[\ell]}\frac{\partial \pi_{[\ell]1}}{\partial {\beta}} [\pi_{[\ell]1}(1-\pi_{[\ell]1})]^{-1} \left\{\frac{K_{[\ell]1}}{K_{[\ell]}}-\pi_{[\ell]1}\right\} \\ &&\!\!+\sum\limits^{p + 1}_{\ell= 1}\sum\limits^{J-1}_{j = 2}K_{[\ell]}\frac{\partial \tilde{\mu}^{\prime}_{[\ell]j}}{\partial {\beta}}{\tilde{\Sigma}}^{-1}_{[\ell]j}({\beta}) (\frac{S_{[\ell],j}}{K_{[\ell]}}-\tilde{\mu}_{[\ell]j}({\beta})) \\ &&\!\!+\sum\limits^{p + 1}_{\ell= 1}K_{[\ell]}\frac{\partial \pi_{[\ell]J}}{\partial {\beta}} [\pi_{[\ell]J}(1 - \pi_{[\ell]J})]^{-1} \left\{\frac{K_{[\ell]J}}{K_{[\ell]}}-\pi_{[\ell]J}\right\} = 0, \end{array} $$

(a.4)

which may be solved by using the iterative formula

$$\begin{array}{@{}rcl@{}} \hat{{\beta}}(r + 1)\!\!\!&=&\!\!\!\hat{{\beta}}(r)+\left[\left\{ \sum\limits^{p + 1}_{\ell= 1}K_{[\ell]}\frac{\partial \pi_{[\ell]1}}{\partial {\beta}} [\pi_{[\ell]1}(1-\pi_{[\ell]1})]^{-1} \frac{\partial \pi_{[\ell]1}}{\partial {\beta}^{\prime}} \right. \right.+ \sum\limits^{p + 1}_{\ell= 1}\sum\limits^{J-1}_{j = 2} \\ &&\qquad\qquad \times K_{[\ell]}\frac{\partial \tilde{\mu}^{\prime}_{[\ell]j}}{\partial \beta}{\tilde{\Sigma}}^{-1}_{[\ell]j}({\beta})\frac{\partial \tilde{\mu}_{[\ell]j}}{\partial {\beta}^{\prime}} +\sum\limits^{p + 1}_{\ell= 1}K_{[\ell]}\frac{\partial \pi_{[\ell]J}}{\partial {\beta}}\\ &&\qquad\qquad\left.\left.\times[\pi_{[\ell]J}(1-\pi_{[\ell]J})]^{-1} \frac{\partial \pi_{[\ell]J}}{\partial {\beta}^{\prime}} \right\}^{-1} G_{1}({\beta})\right]_{|{\beta}=\hat{{\beta}}(r)}, \end{array} $$

(a.5)

until convergence.

1.2 A2. GQL Estimation aids for binary type ordinal model

Write the mean and covariance matrix of $S^{*}_{i}$ (defined in (3.7)) as

$$\begin{array}{@{}rcl@{}} E[S^{*}_{i}]\!\!&=&\!\![\pi^{*}_{i1},\ldots,\pi^{*}_{ij},\ldots,\pi^{*}_{i,J-1}]' =\pi^{*}_{i}(\beta) \\ \text{cov}[S^{*}_{i}] \!\!&=&\!\!\left( \begin{array}{ccccc}V_{i11} & {\cdots} & V_{i1j} & {\cdots} & V_{i1,J-1} \cr {\vdots} & {\vdots} & {\vdots} & {\vdots} & {\vdots} \cr V_{ij1} & {\cdots} & V_{ijj} & {\cdots} & V_{ij,J-1} \cr {\vdots} & {\vdots} & {\vdots} & {\vdots} & {\vdots} \cr V_{i,J-1,1} & {\cdots} & V_{i,J-1,j} &{\cdots} & V_{i,J-1,J-1} \end{array}\right) = V^{*}_{i}(\beta). \end{array} $$

(a.6)

Notice that the formulas for $\pi ^{*}_{ij}$ and V_ijj are already given in (3.6). Now, for two different cut points j < k, the covariance elements V_ijk in (6) have the formulas

$$\begin{array}{@{}rcl@{}} \text{cov}[S_{ij},S_{ik}]&=&\text{cov}\left[\sum\limits^{J}_{c=j + 1}y_{ic}, \sum\limits^{J}_{d=k + 1}y_{id}\right]\\ &=&\text{cov}\left[\sum\limits^{k}_{c=j + 1}y_{ic}+\sum\limits^{J}_{c=k + 1}y_{ic},\sum\limits^{J}_{d=k + 1} y_{id}\right] \\ &=&-\sum\limits^{k}_{c=j + 1}\sum\limits^{J}_{d=k + 1}\pi_{ic}\pi_{id}+\text{var}[S_{ik}]\\ &=&-\sum\limits^{k}_{c=j + 1}\sum\limits^{J}_{d=k + 1}\pi_{ic}\pi_{id}+V_{ikk}. \end{array} $$

(a.7)

1.3 A3. How to write $S^{*}_{[\ell ]j}$ in practice ?

Once again the elements in the $S^{*}_{[\ell ]g}$ vector are either 0 or 1. By following the definition of the cumulative response given in (3.4), we write the observation vectors for g = 1,…,J, as

$$\begin{array}{@{}rcl@{}} S^{*}_{[\ell]1}&=&[S_{[\ell]1,1}= 0,S_{[\ell]1,2}= 0,\ldots, S_{[\ell]1,j}= 0,\ldots,S_{[\ell]1,J-2}= 0,S_{[\ell]1,J-1}= 0]' \\ S^{*}_{[\ell]2}&=&[S_{[\ell]1,1}= 1,S_{[\ell]1,2}= 0,\ldots, S_{[\ell]1,j}= 0,\ldots,S_{[\ell]1,J-2}= 0,S_{[\ell]1,J-1}= 0]' \\ S^{*}_{[\ell](J-1)}&=&[S_{[\ell](J-1),1}= 1,S_{[\ell](J-1),2}= 1,\ldots, S_{[\ell](J-1),j}= 1,\ldots, \\ &&\ldots,S_{[\ell](J-1),J-2}= 1,S_{[\ell](J-1),J-1}= 0]' \\ S^{*}_{[\ell]J}&=&[S_{[\ell]J,1}= 1,S_{[\ell]J,2}= 1,\ldots, S_{[\ell]J,j}= 1,\ldots,S_{[\ell]J,J-2}= 1,S_{[\ell]J,J-1}= 1]'.\\ \end{array} $$

(a.8)

Appendix B: GQL Asymptotics for Both CTN and CBN Ordinal Models

2.1 B1. GQL asymptotics for the CTN ordinal model

For convenience, we re-express the GQL estimating equation in (5.4) as

$$\begin{array}{@{}rcl@{}} G_{1}({\beta})&=&\sum\limits^{p + 1}_{\ell= 1}\sum\limits^{K_{[\ell]}}_{i\in \ell}\left[ \frac{\partial {\mu^{*}}^{\prime}_{[\ell]}({\beta})}{\partial {\beta}}D^{-1}_{[\ell]}({\beta}) \left\{h_{i\in \ell}-{\mu^{*}}_{[\ell]}({\beta})\right\}\right] \\ &=&\sum\limits^{p + 1}_{\ell= 1}\sum\limits^{K_{[\ell]}}_{i\in \ell}g_{i[\ell],1}({\beta})= 0, \end{array} $$

(b.1)

where

$$\begin{array}{@{}rcl@{}} h^{\prime}_{i\in \ell}\!\!\!\!\!\!&=&\!\!\!\!\!\![y_{i \in \ell,1},S^{\prime}_{i \in \ell,2},\ldots,S^{\prime}_{i \in \ell,j},\ldots,S^{\prime}_{i \in \ell,J - 1},y_{i\in \ell,J}] \\ {\mu^{*}}^{\prime}_{[\ell]}({\beta})\!\!\!\!\!\!&=&\!\!\!\!\!\![\pi_{[\ell]1},\tilde{\mu}^{\prime}_{[\ell]2}({\beta}), \ldots, \tilde{\mu}^{\prime}_{[\ell]j}({\beta}),\ldots, \tilde{\mu}^{\prime}_{[\ell](J - 1)}({\beta}),\pi_{[\ell]J}({\beta})] \\ \frac{\partial {\mu^{*}}^{\prime}_{[\ell]}({\beta})}{\partial {\beta}} \!\!\!\!\!\!&=&\!\!\!\!\!\!\left[\frac{\partial \pi_{[\ell]1}}{\partial {\beta}},\frac{\partial \tilde{\mu}^{\prime}_{[\ell]2}({\beta})}{\partial {\beta}},\ldots, \frac{\partial \tilde{\mu}^{\prime}_{[\ell]j}({\beta})}{\partial {\beta}},\ldots, \frac{\partial \tilde{\mu}^{\prime}_{[\ell](J - 1)}({\beta})}{\partial {\beta}}, \frac{\partial \pi_{[\ell]J}}{\partial {\beta}} \right], \\ &&\!\!\!\!\!\! and \\ D_{[\ell]}({\beta}) \!\!\!\!\!\!&=&\!\!\!\!\!\!\left( \begin{array}{ccccccc} \pi_{[\ell]1}(1 - \pi_{[\ell]1}) & 0 & {\cdots} & 0 & {\cdots} &0 & 0\cr 0 & \tilde{\Sigma}_{[\ell]2}({\beta}) & {\cdots} & 0 & {\cdots} & 0 &0\cr {\vdots} & {\vdots} & {\vdots} & {\vdots} & {\vdots} & {\vdots} & {\vdots} \cr 0 & 0 & {\cdots} & \tilde{\Sigma}_{[\ell]j}({\beta}) & {\cdots} &0 & 0 \cr {\vdots} & {\vdots} & {\vdots} & {\vdots} & {\vdots} & {\vdots} &{\vdots} \cr 0 & 0 & {\cdots} & 0 &{\cdots} & \tilde{\Sigma}_{[\ell](J - 1)}({\beta})& 0 \cr 0 & 0 & {\cdots} & 0 & {\cdots} & 0 & \pi_{[\ell]J}(1 - \pi_{[\ell]J}) \end{array}\right) \\ &&\!\!\!\!\!\!: \{(J - 1)2\} \times \{(J - 1)2\} , \\ .\end{array} $$

(b.2)

We now exploit (b.1) and for true β define

$$\begin{array}{@{}rcl@{}} \bar{G}_{K,1}({\beta})&=&\sum\limits^{p + 1}_{\ell= 1}\frac{1}{K_{[\ell]}} \sum\limits^{K_{[\ell]}}_{i\in \ell}g_{i[\ell],1}({\beta}) \\ &=&\sum\limits^{p + 1}_{\ell= 1}\frac{1}{K_{[\ell]}} \sum\limits^{K_{[\ell]}}_{i\in \ell}\left[ \frac{\partial {\mu^{*}}^{\prime}_{[\ell]}({\beta})}{\partial {\beta}}D^{-1}_{[\ell]}({\beta}) \left\{h_{i\in \ell}-{\mu^{*}}_{[\ell]}({\beta})\right\}\right], \end{array} $$

(b.3)

where for a given value of $\ell , h_{1},\ldots ,h_{i},\ldots ,h_{K_{\ell }}$ are independent to each other as they are collected from K_[ℓ] independent individuals, and they are also identically distributed because

$$ h_{i\in \ell} \sim [{\mu^{*}}_{[\ell]}(\beta),D_{[\ell]}({\beta})] $$

(b.4)

where for a given ℓ, the mean vectors and covariance matrices remain the same for individuals. Furthermore, by (b.4), it follows from (b.3) that

$$\begin{array}{@{}rcl@{}} E[\bar{G}_{K,1}({\beta})]&=&0 \\ \text{cov}[\bar{G}_{K,1}({\beta})]&=&\sum\limits^{p + 1}_{\ell= 1}\frac{1}{K^{2}_{[\ell]}} \sum\limits^{K_{[\ell]}}_{i\in \ell}\text{cov}[g_{i[\ell],1}({\beta})] \\ &=&\sum\limits^{p + 1}_{\ell= 1}\frac{1}{K^{2}_{[\ell]}} \sum\limits^{K_{[\ell]}}_{i\in \ell}\left[ \frac{\partial {\mu^{*}}^{\prime}_{[\ell]}({\beta})}{\partial {\beta}}D^{-1}_{[\ell]}({\beta}) \frac{\partial {\mu^{*}}_{[\ell]}({\beta})}{\partial {\beta}^{\prime}} \right] \\ &=& \sum\limits^{p + 1}_{\ell= 1}\frac{1}{K^{2}_{[\ell]}}V^{*}_{K_{[\ell]}}({\beta}) \\ &=&V^{*}_{K}({\beta}). \end{array} $$

(b.5)

By (b.4) and (b.5), it then follows from the well known multivariate central limit theorem (see Mardia, Kent and Bibby (1979, p. 51), for example) that as min{K_[ℓ];ℓ = 1,…,p + 1}→∞, the limiting distribution of $Z_{K}=[V^{*}_{K}]^{-\frac {1}{2}}\bar {G}_{K,1}({\beta })$, say

$$\lim_{\min\{K_{[\ell]};\ell= 1,\ldots,p + 1\} \rightarrow \infty}f(Z_{K})$$

is multivariate normal. More specifically,

$$ \lim_{\min\{K_{[\ell]};\ell= 1,\ldots,p + 1\} \rightarrow \infty}f(Z_{K}) \rightarrow N_{(p + 1)(J-1)}(0,I_{(p + 1)(J-1)}). $$

(b.6)

Now because $\hat {{\beta }}_{GQL}$ obtained from (5.4) or (b.1), is a solution of $G_{1}(\hat {{\beta }}_{GQL})$ = 0, one may use (b.3) and solve

$$ \sum\limits^{p + 1}_{\ell= 1}\frac{1}{K_{[\ell]}}\sum\limits^{K_{[\ell]}}_{i\in \ell}g_{i[\ell],1}(\hat{{\beta}}_{GQL})= 0, $$

(b.7)

which by first order Taylor’s series expansion produces

$$ \sum\limits^{p + 1}_{\ell= 1}\frac{1}{K_{[\ell]}}\sum\limits^{K_{[\ell]}}_{i\in \ell}g_{i[\ell],1}({\beta})+(\hat{{\beta}}_{GQL}-{\beta}) \sum\limits^{p + 1}_{\ell= 1}\frac{1}{K_{[\ell]}}{\sum}^{K_{[\ell]}}_{i\in \ell}\frac{\partial g_{i[\ell],1}({\beta})}{\partial {\beta}^{\prime}}= 0. $$

(b.8)

That is,

$$\begin{array}{@{}rcl@{}} \hat{{\beta}}_{GQL}-{\beta} &=&-\left[\sum\limits^{p + 1}_{\ell= 1}\frac{1}{K_{[\ell]}}\sum\limits^{K_{[\ell]}}_{i\in \ell}\frac{\partial g_{i[\ell],1}({\beta})}{\partial {\beta}^{\prime}}\right]^{-1} \sum\limits^{p + 1}_{\ell= 1}\frac{1}{K_{[\ell]}}\sum\limits^{K_{[\ell]}}_{i\in \ell}g_{i[\ell],1}({\beta}) \\ &=& -\left[-\sum\limits^{p + 1}_{\ell= 1}\frac{1}{K^{2}_{[\ell]}} \sum\limits^{K_{[\ell]}}_{i\in \ell} \frac{\partial {\mu^{*}}^{\prime}_{[\ell]}({\beta})}{\partial {\beta}}D^{-1}_{[\ell]}({\beta}) \frac{\partial {\mu^{*}}_{[\ell]}({\beta})}{\partial {\beta}^{\prime}} \right]^{-1}\\ &&\times \sum\limits^{p + 1}_{\ell= 1}\frac{1}{K_{[\ell]}}\sum\limits^{K_{[\ell]}}_{i\in \ell}g_{i[\ell],1}({\beta}) \\ &=& \left[V^{*}_{K}({\beta})\right]^{-1}\bar{G}_{K,1}({\beta}) \text{by (b.5)} \\ &=& \left[V^{*}_{K}({\beta})\right]^{-\frac{1}{2}}Z_{K}. \end{array} $$

(b.9)

It then follows by (b.6) that

$$ \lim_{\min\{K_{[\ell]};\ell= 1,\ldots,p + 1\} \rightarrow \infty}f(\hat{{\beta}}_{GQL}-{\beta} ) \rightarrow N_{(p + 1)(J-1)}(0,{V^{*}}^{-1}_{K}({\beta})), $$

(b.10)

yielding the normal distribution given in (5.5).

2.2 B2. GQL Asymptotics for the CBN ordinal model

In order to demonstrate the limiting distribution in (5.8) (under Section 5.2), we begin with $E[S^{*}_{i}]=\pi ^{*}_{i}({\beta })$ and $\text {cov}[S^{*}_{i}]=V^{*}_{i}({\beta })$ by (3.8), and also because $S^{*}_{i}$s are independent, one obtains

$$\begin{array}{@{}rcl@{}} E[\bar{G}_{2}({\beta})]&=&0, \text{and} \\ \text{cov}[\bar{G}_{2}({\beta})]&=& \frac{1}{K^{2}}\sum\limits^{K}_{i = 1}\frac{\partial \pi^{*'}_{i}}{\partial {\beta}}{V^{*}}^{-1}_{i}(\hat{{\beta}})\frac{\partial \pi^{*}_{i}}{\partial {\beta}^{\prime}} \\ &=& \frac{1}{K^{2}}\sum\limits^{K}_{i = 1}Q^{*}_{i}({\beta})=Q^{*}_{K}({\beta}). \end{array} $$

(b.11)

Next by similar calculations as in (b.9), one writes

$$\begin{array}{@{}rcl@{}} \hat{{\beta}}-{\beta}&=&[Q^{*}_{K}({\beta})]^{-1}\bar{G}_{2}({\beta}) \\ &=&[Q^{*}_{K}({\beta})]^{-\frac{1}{2}}Z_{K}, \end{array} $$

(b.12)

where

$$ \lim_{K \rightarrow \infty}f(Z_{K}) \rightarrow N_{(p + 1)(J-1)}(0,I_{(p + 1)(J-1)}) $$

(b.13)

by using the multivariate central limit theorem as in (b.6). The limiting distributional result in (5.8) follows by by applying (b.13)-(b.12).

Appendix C: Likelihood Asymptotics for the CBN Ordinal Model: An Illustration

As indicated in Section 2.3 we demonstrate here that the likelihood regression estimators, for example, under the CBN model are consistent for the regression parameters and they asymptotically follow the Gaussian distribution with suitable mean vector and covariance matrix.

For the purpose, we re-express the likelihood estimating equation (2.17) for β as

$$\begin{array}{@{}rcl@{}} \frac{\partial \text{Log} L_{2}({\beta})}{\partial {\beta}} &=&{\sum}^{p + 1}_{\ell}{\sum}^{K_{[\ell]}}_{i = 1}{\sum}^{J-1}_{j = 1} \left[\frac{\partial F_{i[\ell]j}}{\partial {\beta}} \left[F_{i[\ell]j}\{1- F_{i[\ell]j}\}\right]^{-1} \right. \\ &&\times \left. \left( \{1-b^{(j)}_{i \in \ell}(c_{i})\}-F_{i[\ell]j}\right)\right]= 0. \end{array} $$

(c.1)

We now perform the last summation ${\sum }^{J-1}_{j = 1}\left [\cdot \right ]$ in (c.1) as follows. Use the elements shown in (⋅) for j = 1,…,J − 1, and construct the (J − 1) × 1 vector

$$\begin{array}{@{}rcl@{}} d_{i\in \ell}(c_{i})&=&[\{1-b^{(1)}_{i\in \ell}(c_{i})\}-F_{i\ell 1},\ldots,\{1-b^{(j)}_{i\in \ell}(c_{i})\}\\ &&-F_{i[\ell]j}, \ldots, \{1-b^{(J-1)}_{i\in \ell}(c_{i})\}-F_{i[\ell](J-1)}]' \\ &=&[\tilde{b}^{(1)}_{i\in \ell}(c_{i})-F_{i[\ell]1},\ldots,\tilde{b}^{(j)}_{i\in \ell}(c_{i})-F_{i[\ell]j},\ldots, \tilde{b}^{(J-1)}_{i\in \ell}(c_{i})-F_{i[\ell](J-1)} \\ &=&[\tilde{b}^{*}_{i\in \ell}-F^{*}_{i[\ell]}]. \end{array} $$

(c.2)

Next write

$$\begin{array}{@{}rcl@{}} F^{\prime}_{i[\ell]}&=&[F_{i[\ell]1},\ldots,F_{i[\ell]j},\ldots,F_{i[\ell](J-1)}]: 1 \times (J-1), \text{and} \end{array} $$

(c.3)

$$\begin{array}{@{}rcl@{}} P_{i\in \ell}(c_{i})&=&\text{diag}\left[\{(1- F_{i[\ell]1})F_{i[\ell]1}\},\ldots,\{(1- F_{i[\ell]j})F_{i[\ell]j}\},\ldots, \right. \\ && \left. \{(1- F_{i[\ell](J-1)})F_{i[\ell](J-1)}\}\right]: (J-1) \times (J-1). \end{array} $$

(c.4)

Combining (c.2), (c.3) and (c.5), the summation ${\sum }^{J-1}_{j = 1}\left [\cdot \right ]$ in (c.1) may be expressed as

$$\begin{array}{@{}rcl@{}} &&\sum\limits^{J-1}_{j = 1} \left[\frac{\partial F_{i[\ell]j}}{\partial {\beta}} \left[F_{i[\ell]j}\{1- F_{i[\ell]j}\}\right]^{-1} \left( \{1-b^{(j)}_{i \in \ell}(c_{i})\}-F_{i[\ell]j}\right)\right] \\ &=& \frac{\partial F^{\prime}_{i[\ell]}}{\partial {\beta}} P^{-1}_{i\in \ell}(c_{i})d_{i\in \ell}(c_{i}): (p + 1)(J-1) \times 1. \end{array} $$

(c.5)

Consequently, the complete likelihood estimating equation in (c.1) reduces to

$$\begin{array}{@{}rcl@{}} \frac{\partial \text{Log} L_{2}({\beta})}{\partial {\beta}} &=&\sum\limits^{p + 1}_{\ell}{\sum}^{K_{[\ell]}}_{i = 1} \left[\frac{\partial F^{\prime}_{i[\ell]}}{\partial {\beta}} P^{-1}_{i\in \ell}(c_{i})d_{i\in \ell}(c_{i})\right]= 0. \end{array} $$

(c.6)

However, instead of solving (c.7), it is convenient to solve

$$\begin{array}{@{}rcl@{}} \frac{\partial \text{Log} \bar{L}_{2}({\beta})}{\partial {\beta}} &=&\sum\limits^{p + 1}_{\ell}\frac{1}{K_{[\ell]}}\sum\limits^{K_{[\ell]}}_{i = 1} \left[\frac{\partial F^{\prime}_{i[\ell]}}{\partial {\beta}} P^{-1}_{i\in \ell}(c_{i})d_{i\in \ell}(c_{i})\right]= 0. \end{array} $$

(c.7)

Notice that in (c.8), E[d_i∈ℓ(c_i)] = 0 and cov[d_i∈ℓ(c_i)] = P_i∈ℓ. Also all K_[ℓ] individuals are independent and covariate groups are mutually exclusive. It then follows that

$$\begin{array}{@{}rcl@{}} E\left[\frac{\partial \text{Log} \bar{L}_{2}({\beta})}{\partial {\beta}} \right] &=&0 \\ \text{cov}\left[\frac{\partial \text{Log} \bar{L}_{2}({\beta})}{\partial {\beta}}\right]&=&\sum\limits^{p + 1}_{\ell}\frac{1}{K^{2}_{[\ell]}}\sum\limits^{K_{[\ell]}}_{i = 1} \left[\frac{\partial F^{\prime}_{i[\ell]}}{\partial {\beta}} P^{-1}_{i\in \ell}(c_{i})\frac{\partial F_{i[\ell]}}{\partial {\beta}^{\prime}}\right] \\ &=&\sum\limits^{p + 1}_{\ell}\frac{1}{K^{2}_{[\ell]}}M^{*}_{K_{[\ell]}}({\beta}) \\ &=&M^{*}_{K}({\beta}) \text{(say)}. \end{array} $$

(c.8)

Suppose that we denote the solution of the likelihood equation (c.8) by $\hat {{\beta }}_{MLE}$. By applying the multivariate central limit theorem and following the calculations as in the Appendix B.1 (see Eqns. (b.6)-(b.10)), one obtains the limiting distribution of $\hat {{\beta }}_{MLE}$ as

$$ \lim_{\min\{K_{[\ell]};\ell= 1,\ldots,p + 1\} \rightarrow \infty}f(\hat{{\beta}}_{MLE}) \rightarrow N_{(p + 1)(J-1)}({\beta},{M^{*}}^{-1}_{K}({\beta})), $$

(c.9)

showing that $\hat {{\beta }}_{MLE}$ is consistent for β.