Regression Models for Ordinal Categorical Time Series Data

Abstract

Regression analysis for multinomial/categorical time series is not adequately discussed in the literature. Furthermore, when categories of a multinomial response at a given time are ordinal, the regression analysis for such ordinal categorical time series becomes more complex. In this paper, we first develop a lag 1 transitional logit probabilities based correlation model for the multinomial responses recorded over time. This model is referred to as a multinomial dynamic logits (MDL) model. To accommodate the ordinal nature of the responses we then compute the binary distributions for the cumulative transitional responses with cumulative logits as the binary probabilities. These binary distributions are next used to construct a pseudo likelihood function for inferences for the repeated ordinal multinomial data. More specifically, for the purpose of model fitting, the likelihood estimation is developed for the regression and dynamic dependence parameters involved in the MDL model.

Appendix

Derivation for $\frac{\partial F_{(1)j}}{\partial \beta }:$

Recall from Sect. 2.1 that $F_{(1)j}=\sum ^j_{c=1}\pi _{(1)c},$ where $\pi _{(1)c}$ is given by (2). It then follows that

$$\begin{aligned} \frac{\partial F_{(1)j}}{\partial \beta } =\frac{\partial }{\partial \beta }\sum ^j_{c=1}\pi _{(1)c} = \frac{\partial }{\partial \beta }\sum ^j_{c=1}\frac{\exp \left( x'_1\beta _{c}\right) }{1+\sum ^{J-1}_{g=1}\exp \left( x'_1\beta _{g}\right) }. \end{aligned}$$

(45)

Now because

$$\begin{aligned} \frac{\partial \pi _{(1)c}}{\partial \beta _c} = \pi _{(1)c}[1- \pi _{(1)c}]x_1,\;\text{ and }\; \frac{\partial \pi _{(1)c}}{\partial \beta _k} =- [\pi _{(1)c}\pi _{(1)k}]x_{1}, \end{aligned}$$

(46)

it follows that

$$\begin{aligned} \frac{\partial \pi _{(1)c}}{\partial \beta }= & {} \begin{pmatrix}-\pi _{(1)1}\pi _{(1)c} \\ \vdots \\ \pi _{(1)c}[1-\pi _{(1)c}] \\ \vdots \\ -\pi _{(1)(J-1)}\pi _{(1)c}\\ \end{pmatrix} \otimes x_{1} : (J-1)(p+1) \times 1 \nonumber \\= & {} \left[ \pi _{(1)c}(\delta _{(1)c}-\pi _{(1)})\right] \otimes x_{1}. \end{aligned}$$

(47)

The formula for $\frac{\partial F_{(1)j}}{\partial \beta }$ in (25) follows by using (47) and (45).

Derivation for $\frac{\partial \tilde{\lambda }^{(2)}_{gj}(g^*)}{\partial \beta }:$

By using the formula for $\tilde{\lambda }^{(2)}_{gj}(g^*)$ from (13) we write

$$\begin{aligned} \frac{\partial \tilde{\lambda }^{(2)}_{gj}(g^*)}{\partial \beta } =\frac{\partial }{\partial \beta }\left\{ \begin{array}{ll} \frac{1}{g}\sum ^g_{c_1=1}\sum ^J_{c_2=j+1}\lambda ^{(c_2)}_{t|t-1}(c_1) &{}\quad \text{ for }\; g^*=1 \\ \frac{1}{J-g}\sum ^J_{c_1=g+1}\sum ^J_{c_2=j+1}\lambda ^{(c_2)}_{t|t-1}(c_1) &{}\quad \text{ for }\; g^*=2, \end{array} \right. \end{aligned}$$

(48)

where $\lambda ^{(c_2)}_{t|t-1}(c_1)$ is given in (5), that is,

$$\begin{aligned} \eta ^{(c_2)}_{t|t-1}(c_1)=\left\{ \begin{array}{ll}\frac{{\exp \,\left[ x^{'}_{t}\beta _{c_2} +\gamma '_{c_2}\delta _{(t-1)c_1}\right] }}{{1\,+\,\sum ^{J-1}_{v=1}\exp \,\left[ x^{'}_{t}\beta _v+\gamma '_v \delta _{(t-1)c_1}\right] }}, &{}\quad \text{ for }\;c_2=1,\ldots ,J-1 \\ \frac{1}{{1\,+\,\sum ^{J-1}_{v=1}\exp \,\left[ x^{'}_{t}\beta _v +\gamma '_v\delta _{(t-1)c_1}\right] }}, &{}\quad \text{ for }\;c_2=J.\\ \end{array}\right. \end{aligned}$$

(49)

Now, for $t=2,\ldots ,T,$ it follows from (49) that

$$\begin{aligned} \frac{\partial \eta ^{(c_2)}_{t|t-1}(c_1)}{\partial \beta _{c_2}}= & {} \eta ^{(c_2)}_{t|t-1}(c_1)\left[ 1- \eta ^{(c_2)}_{t|t-1}(c_1)\right] x_{t} \nonumber \\ \frac{\partial \eta ^{(c_2)}_{t|t-1}(c_1)}{\partial \beta _k}= & {} - \left[ \eta ^{(c_2)}_{t|t-1}(c_1)\eta ^{(k)}_{t|t-1}(c_1)\right] x_{t}, \end{aligned}$$

(50)

yielding

$$\begin{aligned} \frac{\partial \eta ^{(c_2)}_{t|t-1}(c_1)}{\partial \beta }= & {} \begin{pmatrix}-\eta ^{(1)}_{t|t-1}(c_1)\eta ^{(c_2)}_{t|t-1}(c_1) \\ \vdots \\ \eta ^{(c_2)}_{t|t-1}(c_1)[1-\eta ^{(c_2)}_{t|t-1}(c_1)] \\ \vdots \\ -\eta ^{(J-1)}_{t|t-1}(c_1)\eta ^{(c_2)}_{t|t-1}(c_1)\\ \end{pmatrix} \otimes x_{t} : (J-1)(p+1) \times 1 \nonumber \\= & {} \left[ \eta ^{(c_2)}_{t|t-1}(c_1)(\delta _{(t-1)c_2}-\eta _{t|t-1}(c_1))\right] \otimes x_{t}. \end{aligned}$$

(51)

The formula for the derivative in (26) follows now by applying (50) into (48).

Derivation for $\frac{\partial \tilde{\lambda }^{(2)}_{gj}(g^*)}{\partial \gamma }:$

By using the formula for $\tilde{\lambda }^{(2)}_{gj}(g^*)$ from (13) we write

$$\begin{aligned} \frac{\partial \tilde{\lambda }^{(2)}_{gj}(g^*)}{\partial \gamma } =\frac{\partial }{\partial \gamma }\left\{ \begin{array}{ll} \frac{1}{g}\sum ^g_{c_1=1}\sum ^J_{c_2=j+1}\lambda ^{(c_2)}_{t|t-1}(c_1) &{} \quad \text{ for }\; g^*=1 \\ \frac{1}{J-g}\sum ^J_{c_1=g+1}\sum ^J_{c_2=j+1}\lambda ^{(c_2)}_{t|t-1}(c_1) &{}\quad \text{ for }\; g^*=2, \end{array} \right. \end{aligned}$$

(52)

where $\lambda ^{(c_2)}_{t|t-1}(c_1)$ is given in (5) [see also (49)].

Next, for $t=2,\ldots ,T,$ it follows from (49) that

$$\begin{aligned} \frac{\partial \eta ^{(c_2)}_{t|t-1}(c_1)}{\partial \gamma _{c_2}}= & {} \eta ^{(c_2)}_{t|t-1}(c_1)\left[ 1- \eta ^{(c_2)}_{t|t-1}(c_1)\right] \delta _{(t-1)c_1} \nonumber \\ \frac{\partial \eta ^{(c_2)}_{t|t-1}(c_1)}{\partial \gamma _k}= & {} - \left[ \eta ^{(c_2)}_{t|t-1}(c_1)\eta ^{(k)}_{t|t-1}(c_1)\right] \delta _{(t-1)c_1}, \end{aligned}$$

(53)

where

$$\delta _{(t-1)c_1} = \left\{ \begin{array}{ll} [01'_{c_1-1},1,01'_{J-1-c_1}]' &{}\quad \text{ for }\;c_1=1,\ldots ,J-1\\ 01_{J-1} &{}\quad \text{ for }\; c_1=J. \end{array} \right. $$

These derivatives in (53) may further be re-expressed as

$$\begin{aligned} \frac{\partial \eta ^{(c_2)}_{t|t-1}(c_1)}{\partial \gamma }= & {} \begin{pmatrix}-\eta ^{(1)}_{t|t-1}(c_1)\eta ^{(c_2)}_{t|t-1}(c_1) \\ \vdots \\ \eta ^{(c_2)}_{t|t-1}(c_1)[1-\eta ^{(c_2)}_{t|t-1}(c_1)] \\ \vdots \\ -\eta ^{(J-1)}_{t|t-1}(c_1)\eta ^{(c_2)}_{t|t-1}(c_1)\\ \end{pmatrix} \otimes \delta _{(t-1)c_1} : (J-1)^2 \times 1 \nonumber \\= & {} \left[ \eta ^{(c_2)}_{t|t-1}(c_1)(\delta _{(t-1)c_2}-\eta _{t|t-1}(c_1))\right] \otimes \delta _{(t-1)c_1}. \end{aligned}$$

(54)

The formula for the derivative in (36) now follows by applying (53) into (52).