A family of the adjusted estimators maximizing the asymptotic predictive expected log-likelihood

Abstract

A family of the estimators adjusting the maximum likelihood estimator by a higher-order term maximizing the asymptotic predictive expected log-likelihood is introduced under possible model misspecification. The negative predictive expected log-likelihood is seen as the Kullback–Leibler distance plus a constant between the adjusted estimator and the population counterpart. The vector of coefficients in the correction term for the adjusted estimator is given explicitly by maximizing a quadratic form. Examples using typical distributions in statistics are shown.

Appendix

1.1 The asymptotic predictive expected log-likelihood for the maximum likelihood estimator

In this appendix, ${\text{E}}_{\text{T}} (\hat{\bar{l}}_{\text{ML}}^{*} )_{{ \to O(n^{ - 2} )}}$ in (14) is obtained, where

${\text{E}}_{\text{T}} (\hat{\bar{l}}_{\text{ML}}^{*} ) = {\text{E}}_{\text{T}} \{ \bar{l}^{*} ({\hat{\varvec{\uptheta }}}_{\text{ML}} )\} = \int_{{R({\mathbf{X}})}} {\bar{l}^{*} \{ {\varvec{\uptheta}}_{\text{ML}} ({\mathbf{X}})\} f_{\text{T}} ({\mathbf{X}}|{\varvec{\upzeta}}_{0} ){\text{d}}{\mathbf{X}}}$ (see (10)). For this expectation, we use the expansion of ${\hat{\varvec{\uptheta }}}_{\text{ML}}$ by Ogasawara (2010, p. 2151) as follows:

$$ {\hat{\varvec{\uptheta }}}_{\text{ML}} - {\varvec{\uptheta}}_{0} = \sum\limits_{j = 1}^{3} {{\varvec{\Lambda}}^{(j)} {\mathbf{l}}_{0}^{(j)} } + O_{p} (n^{ - 2} )\,\,\,({\varvec{\Lambda}}^{(j)} = O(1),\,\,{\mathbf{l}}_{0}^{(j)} = O_{p} (n^{ - j/2} ),\,\,\,j = 1,2,3\,), $$

$$ \begin{aligned} {\varvec{\Lambda}}^{(1)} {\mathbf{l}}_{0}^{(1)} = - {\varvec{\Lambda}}^{ - 1} \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0} }}, \hfill \\ {\varvec{\Lambda}}^{(2)} {\mathbf{l}}_{0}^{(2)} = {\varvec{\Lambda}}^{ - 1} {\mathbf{M\Lambda }}^{ - 1} \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0} }} - \frac{1}{2}{\varvec{\Lambda}}^{ - 1} {\text{E}}_{\text{T}} ({\mathbf{J}}_{0}^{(3)} )\left( {{\varvec{\Lambda}}^{ - 1} \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0} }}} \right)^{ \langle 2 \rangle } \hfill \\ {\varvec{\Lambda}}^{(3)} {\mathbf{l}}_{0}^{(3)} = - {\varvec{\Lambda}}^{ - 1} {\mathbf{M\Lambda }}^{ - 1} {\mathbf{M\Lambda }}^{ - 1} \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0} }} + \frac{1}{2}{\varvec{\Lambda}}^{ - 1} {\mathbf{M\Lambda }}^{ - 1} {\text{E}}_{\text{T}} ({\mathbf{J}}_{0}^{(3)} )\left( {{\varvec{\Lambda}}^{ - 1} \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0} }}} \right)^{ \langle 2 \rangle } \hfill \\ \end{aligned} $$

$$ + {\varvec{\Lambda}}^{ - 1} {\text{E}}_{\text{T}} ({\mathbf{J}}_{0}^{(3)} )\left\{ {\left( {{\varvec{\Lambda}}^{ - 1} {\mathbf{M\Lambda }}^{ - 1} \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0} }}} \right) \otimes \left( {{\varvec{\Lambda}}^{ - 1} \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0} }}} \right)} \right\} - \frac{1}{2}{\varvec{\Lambda}}^{ - 1} \{ {\mathbf{J}}_{0}^{(3)} - {\text{E}}_{\text{T}} ({\mathbf{J}}_{0}^{(3)} )\} \left( {{\varvec{\Lambda}}^{ - 1} \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0} }}} \right)^{ \langle 2 \rangle } $$

$$ \begin{aligned} - \frac{1}{2}{\varvec{\Lambda}}^{ - 1} {\text{E}}_{\text{T}} ({\mathbf{J}}_{0}^{(3)} )\left[ {\left( {{\varvec{\Lambda}}^{ - 1} \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0} }}} \right) \otimes \left\{ {{\varvec{\Lambda}}^{ - 1} {\text{E}}_{\text{T}} ({\mathbf{J}}_{0}^{(3)} )\left( {{\varvec{\Lambda}}^{ - 1} \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0} }}} \right)^{ \langle 2 \rangle } } \right\}} \right] \hfill \\ + \frac{1}{6}{\varvec{\Lambda}}^{ - 1} {\text{E}}_{\text{T}} ({\mathbf{J}}_{0}^{(4)} )\left( {{\varvec{\Lambda}}^{ - 1} \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0} }}} \right)^{ \langle 3 \rangle } , \hfill \\ {\mathbf{J}}_{0}^{(i)} \equiv \frac{{\partial^{j} \bar{l}}}{{\partial {\varvec{\uptheta}}_{0} (\partial {\varvec{\uptheta}}_{0} ^{\prime })^{ \langle j - 1 \rangle } }} = O_{p} (1)\,\,\,(j = 3,4), \hfill \\ \end{aligned} $$

(129)

$$ \begin{aligned} {\mathbf{l}}_{0}^{(1)} = \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0} }},\,\,\,{\mathbf{l}}_{0}^{(2)} = \left\{ {{\text{v}}^{{\prime}} ({\mathbf{M}}) \otimes \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0}^{\prime} }},\,\,\,\left( {\frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0}^{\prime} }}} \right)^{ \langle 2 \rangle } } \right\}^{{\prime}} \equiv ({\mathbf{l}}_{0}^{(2 - 1)'} ,{\mathbf{l}}_{0}^{(2 - 2)'} )^{{\prime}} , \hfill \\ {\mathbf{l}}_{0}^{(3)} = \left[ {{\text{v}}^{{\prime}} ({\mathbf{M}})^{ \langle 2 \rangle } \otimes \frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0}^{\prime} }},\,\,\,{\text{v}}^{{\prime}} ({\mathbf{M}}) \otimes \left( {\frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0}^{\prime} }}} \right)^{ \langle 2 \rangle } } \right., \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,{\text{vec}}^{\prime} \{ {\mathbf{J}}_{0}^{(3)} - {\text{E}}_{\text{T}} ({\mathbf{J}}_{0}^{(3)} )\} \otimes \left( {\frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0}^{\prime} }}} \right)^{ \langle 2 \rangle } ,\,\,\,\,\left. {\left( {\frac{{\partial \,\bar{l}}}{{\partial \,{\varvec{\uptheta}}_{0}^{\prime} }}} \right)^{ \langle 3 \rangle } } \right]^{\prime} \hfill \\ \equiv ({\mathbf{l}}_{0}^{(3 - 1)'} ,{\mathbf{l}}_{0}^{(3 - 2)'} ,{\mathbf{l}}_{0}^{(3 - 3)'} ,{\mathbf{l}}_{0}^{(3 - 4)'} )^{{\prime}} , \hfill \\ \end{aligned} $$

where ${\text{v}}( \cdot )$ is the vectorizing operator taking the non-duplicated elements of a symmetric matrix and ${\text{v}}^{{\prime}} ( \cdot ) = \{ {\text{v}}( \cdot )\}^{{\prime}}$.

Using (129), the matrices ${\varvec{\Lambda}}^{(2 - j)} \,\,(j = 1,2)$ and ${\varvec{\Lambda}}^{(3 - j)} \,\,(j = 1, \ldots ,4)$ are implicitly defined by

$$ {\varvec{\Lambda}}^{(2)} {\mathbf{l}}_{0}^{(2)} = \sum\limits_{j = 1}^{2} {{\varvec{\Lambda}}^{(2 - j)} {\mathbf{l}}_{0}^{(2 - j)} } ,\,\,\,{\varvec{\Lambda}}^{(3)} {\mathbf{l}}_{0}^{(3)} = \sum\limits_{j = 1}^{4} {{\varvec{\Lambda}}^{(3 - j)} {\mathbf{l}}_{0}^{(3 - j)} } . $$

(130)

The expectation to be derived is

$$ \begin{aligned} {\text{E}}_{\text{T}} (\hat{\bar{l}}_{\text{ML}}^{*} ) = \{ \bar{l}^{*} ({\varvec{\uptheta}}_{0} )\}_{O(1)} \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, + \, {\text{E}}_{\text{T}} \left[ {\sum\limits_{j = 2}^{4} {\left\{ {\frac{{\partial^{j} \bar{l}^{*} }}{{j!(\partial {\varvec{\uptheta}}_{0}^{\prime} )^{ \langle j \rangle } }}({\hat{\varvec{\uptheta }}}_{\text{ML}} - {\varvec{\uptheta}}_{0} )^{ \langle j \rangle } } \right\}_{{O_{p} (n^{ - j/2} )}} } } \right]_{{ \to O(n^{ - 2} )}} +\, O(n^{ - 3} ). \hfill \\ \end{aligned} $$

(131)

In (131), the asymptotic expectation is derived term by term. In the following, the notation, e.g., $({\varvec{\Lambda}}^{(2 - 1)} )_{(e:ab,c,d)}$ indicates an element of ${\varvec{\Lambda}}^{(2 - 1)}$ corresponding to the e-th row and the column denoted by “ab, c, d” which corresponds to $({\mathbf{M}})_{ab} \,\,(a \ge b),\,\,\,\partial \bar{l}/\partial ({\varvec{\uptheta}}_{0} )_{c}$ and $\partial \bar{l}/\partial ({\varvec{\uptheta}}_{0} )_{d}$ in ${\mathbf{l}}_{0}^{(2 - 1)}$. The notation, e.g., $\sum\nolimits_{(g,\,h)}^{(2)} {}$ indicates the summation of two terms exchanging g and h; $\sum\limits_{a \ge b}^{{}} {( \cdot )} \equiv \sum\limits_{b = 1}^{q} {\sum\limits_{a = b}^{q} {( \cdot )} }$; and $\lambda^{ab} = ({\varvec{\Lambda}}^{ - 1} )_{ab}$.

• 1.
  $$ \begin{aligned} {\text{E}}_{\text{T}} \left\{ {\frac{1}{2}\frac{{\partial^{2} \bar{l}^{*} }}{{(\partial {\varvec{\uptheta}}_{0}^{\prime} )^{ \langle 2 \rangle } }}({\hat{\varvec{\uptheta }}}_{\text{ML}} - {\varvec{\uptheta}}_{0} )^{ \langle 2 \rangle } } \right\} \hfill \\ = \frac{1}{2}n^{ - 1} {\text{vec}}^{{\prime}} ({\varvec{\Lambda}})n{\text{E}}_{\text{T}} \{ ({\varvec{\Lambda}}^{(1)} {\mathbf{l}}_{0}^{(1)} )^{ \langle 2 \rangle } \} \hfill \\ \,\,\, + \frac{1}{2}n^{ - 2} {\text{vec}}^{{\prime}} ({\varvec{\Lambda}})\bigg[ {2n^{2} } {\text{E}}_{\text{T}} \{ ({\varvec{\Lambda}}^{(2)} {\mathbf{l}}_{0}^{(2)} ) \otimes ({\varvec{\Lambda}}^{(1)} {\mathbf{l}}_{0}^{(1)} )\} \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, + 2n^{2} {\text{E}}_{\text{T}} \{ ({\varvec{\Lambda}}^{(3)} {\mathbf{l}}_{0}^{(3)} ) \otimes ({\varvec{\Lambda}}^{(1)} {\mathbf{l}}_{0}^{(1)} )\} + n^{2} {{\text{E}}_{\text{T}} \{ ({\varvec{\Lambda}}^{(2)} {\mathbf{l}}_{0}^{(2)} )^{ \langle 2 \rangle } \} } \bigg] \hfill \\ \,\,\, + O(n^{ - 3} ) \hfill \\ \end{aligned} $$

  (132)

$$ \begin{aligned} = \frac{{n^{ - 1} }}{2}{\text{tr}}({\varvec{\Gamma \Lambda }}^{ - 1} ) \hfill \\ + n^{ - 2} \sum\limits_{a,b = 1}^{q} {\lambda_{ab} } \mathop {\bigg[ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } }\limits_{({\text{A}})} \sum\limits_{c \ge d} {\sum\limits_{e,f = 1}^{q} {n^{2} } } {\text{E}}_{\text{T}} \{ m_{cd} ({\mathbf{l}}_{0}^{(1)} )_{e} ({\mathbf{l}}_{0}^{(1)} )_{f} \} ({\varvec{\Lambda}}^{(2 - 1)} )_{(a\colon\,\,cd,e)} ( - \lambda^{bf} ) \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, + \sum\limits_{c,d,e = 1}^{q} {n^{2} } {\text{E}}_{\text{T}} \{ ({\mathbf{l}}_{0}^{(1)} )_{c} ({\mathbf{l}}_{0}^{(1)} )_{d} ({\mathbf{l}}_{0}^{(1)} )_{e} \} ({\varvec{\Lambda}}^{(2 - 2)} )_{(a\colon\,\,\,c,d)} ( - \lambda^{be} ) \hfill \\ \end{aligned} $$

$$ \begin{aligned} + \sum\limits_{c \ge d} {\sum\limits_{e \ge f} {\sum\limits_{g,h = 1}^{q} {({\varvec{\Lambda}}^{(3 - 1)} )_{(a\colon\,\,cd,ef,g)} } } } ( - \lambda^{bh} )\,\bigg[ {n\,{\text{acov}}\{ m_{cd} ,m_{ef} \} \gamma_{gh} } \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, + \sum\limits_{(g,h)}^{2} {n\,{\text{acov}}\{ m_{cd} ,({\mathbf{l}}_{0}^{(1)} )_{g} \} } {n\,{\text{acov}}\{ m_{ef} ,({\mathbf{l}}_{0}^{(1)} )_{h} \} } \bigg] \hfill \\ + \sum\limits_{c \ge d} {\sum\limits_{e,f,g = 1}^{q} {({\varvec{\Lambda}}^{(3 - 2)} )_{(a\colon\,\,cd,e,f)} } } ( - \lambda^{bg} )\sum\limits_{(e,f,g)}^{3} {n\,{\text{acov}}\{ m_{cd} ,({\mathbf{l}}_{0}^{(1)} )_{e} \} } \gamma_{fg} \hfill \\ + \sum\limits_{c,d,e,f,g,h = 1}^{q} {({\varvec{\Lambda}}^{(3 - 3)} )_{(a\colon\,\,cde,f,g,h)} } ( - \lambda^{bh} )\sum\limits_{(f,g,h)}^{3} {n\,{\text{acov}}\{ ({\mathbf{J}}_{0}^{(3)} )_{(c,d,e)} ,({\mathbf{l}}_{0}^{(1)} )_{f} \} } \gamma_{gh} \hfill \\ + \sum\limits_{c,d,e,f = 1}^{q} {({\varvec{\Lambda}}^{(3 - 4)} )_{(a\colon\,\,c,d,e)} } ( - \lambda^{bf} )(\gamma_{cd} \gamma_{ef} + \gamma_{ce} \gamma_{df} + \gamma_{cf} \gamma_{de} ) \hfill \\ \end{aligned} $$

$$ \begin{aligned} + \frac{1}{2}\mathop {\left[ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.}\limits_{({\text{B}})} \sum\limits_{c \ge d} {\sum\limits_{e \ge f} {\sum\limits_{g,h = 1}^{q} {({\varvec{\Lambda}}^{(2 - 1)} )_{(a\colon\,\,cd,g)} } } } ({\varvec{\Lambda}}^{(2 - 1)} )_{(b\colon\,\,ef,h)} \left[ {n\,{\text{acov}}(m_{cd} ,m_{ef} )} \right.\gamma_{gh} \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, + \sum\limits_{(g,h)}^{2} {n\,{\text{acov\{ }}m_{cd} ,({\mathbf{l}}_{0}^{(1)} )_{g} \} \left. {n\,{\text{acov\{ }}m_{ef} ,({\mathbf{l}}_{0}^{(1)} )_{h} \} } \right]} \hfill \\ + 2\sum\limits_{c \ge d} {\sum\limits_{e,f,g = 1}^{q} {({\varvec{\Lambda}}^{(2 - 1)} )_{(a\colon\,\,cd,e)} } } ({\varvec{\Lambda}}^{(2 - 2)} )_{(b\colon\,\,f,g)} \sum\limits_{(e,f,g)}^{3} {n\,{\text{acov\{ }}m_{cd} ,({\mathbf{l}}_{0}^{(1)} )_{e} \} } \gamma_{fg} \hfill \\ + \sum\limits_{c,d,e,f = 1}^{q} {({\varvec{\Lambda}}^{(2 - 2)} )_{(a\colon\,\,c,d)} ({\varvec{\Lambda}}^{(2 - 2)} )_{(b\colon\,\,e,f)} } (\gamma_{cd} \gamma_{ef} + \gamma_{ce} \gamma_{df} + \gamma_{cf} \gamma_{de} )\mathop {\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right]}\limits_{{({\text{B}})}} \mathop {\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right]}\limits_{{({\text{A}})}} + O(n^{ - 3} ), \hfill \\ \end{aligned} $$

where, e.g., $\mathop [\limits_{{({\text{A}})}} \cdot \mathop ]\limits_{{({\text{A}})}}$ is for ease of finding correspondence.

• 2.
  $$ \begin{aligned} {\text{E}}_{\text{T}} \left\{ {\frac{1}{6}\frac{{\partial^{3} \bar{l}^{*} }}{{(\partial {\varvec{\uptheta}}_{0}^{\prime} )^{ \langle 3 \rangle } }}({\hat{\varvec{\uptheta }}}_{\text{ML}} - {\varvec{\uptheta}}_{0} )^{ \langle 3 \rangle } } \right\} \hfill \\ = n^{ - 2} \frac{{\partial^{3} \bar{l}^{*} }}{{(\partial {\varvec{\uptheta}}_{0}^{\prime} )^{ \langle 3 \rangle } }}\left[ {\frac{{n^{2} }}{6}{\text{E}}_{\text{T}} \{ ({\varvec{\Lambda}}^{(1)} {\mathbf{l}}_{0}^{(1)} )^{ \langle 3 \rangle } \} + \frac{{n^{2} }}{2}{\text{E}}_{\text{T}} \{ ({\varvec{\Lambda}}^{(2)} {\mathbf{l}}_{0}^{(2)} ) \otimes ({\varvec{\Lambda}}^{(1)} {\mathbf{l}}_{0}^{(1)} )^{ \langle 2 \rangle } \} } \right] \hfill \\ \,\,\, + O(n^{ - 3} ) \hfill \\ \end{aligned} $$

  (133)

$$ \begin{aligned} = n^{ - 2} \sum\limits_{a,b,c = 1}^{q} {\frac{{\partial^{3} \bar{l}^{*} }}{{\partial \theta_{0a} \partial \theta_{0b} \partial \theta_{0c} }}} \mathop {\left[ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.}\limits_{{({\text{A}})}} \frac{1}{6}\sum\limits_{d,e,f = 1}^{q} {( - \lambda^{ad} )( - \lambda^{be} )( - \lambda^{cf} )} \hfill \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, \times n^{2} {\text{E}}_{\text{T}} \{ ({\mathbf{l}}_{0}^{(1)} )_{d} ({\mathbf{l}}_{0}^{(1)} )_{e} ({\mathbf{l}}_{0}^{(1)} )_{f} \} \hfill \\ + \frac{1}{2}\mathop {\left[ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.}\limits_{{({\text{B}})}} \sum\limits_{d \ge e} {\sum\limits_{f,g,h = 1}^{q} {({\varvec{\Lambda}}^{(2 - 1)} )_{(a\colon\,\,de,f)} } } ( - \lambda^{bg} )( - \lambda^{ch} )\sum\limits_{(f,g,h)}^{3} {n\,{\text{acov}}\{ m_{de} ,({\mathbf{l}}_{0}^{(1)} )_{f} \} } \gamma_{gh} \hfill \\ \,\,\, + \sum\limits_{d,e,f,g = 1}^{q} {({\varvec{\Lambda}}^{(2 - 2)} )_{(a\colon\,\,d,e)} } ( - \lambda^{bf} )( - \lambda^{cg} )(\gamma_{de} \gamma_{fg} + \gamma_{df} \gamma_{eg} + \gamma_{dg} \gamma_{ef} )\mathop {\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right]}\limits_{({\text{B}})} \mathop {\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right]}\limits_{({\text{A}})} \,\,\, + O(n^{ - 3} ). \hfill \\ \end{aligned} $$

• 3.
  $$ \begin{aligned} {\text{E}}_{\text{T}} \left\{ {\frac{1}{24}\frac{{\partial^{4} \bar{l}^{*} }}{{(\partial {\varvec{\uptheta}}_{0}^{\prime} )^{ \langle 4 \rangle } }}({\hat{\varvec{\uptheta }}}_{\text{ML}} - {\varvec{\uptheta}}_{0} )^{ \langle 4 \rangle } } \right\} \hfill \\ \,\,\,\,\,\,\,\, = \frac{{n^{ - 2} }}{8}{\text{vec}}^{{\prime}} ({\varvec{\Lambda}}^{ - 1} {\varvec{\Gamma \Lambda }}^{ - 1} )\frac{{\partial^{4} \bar{l}^{*} }}{{(\partial {\varvec{\uptheta}}_{0} )^{ \langle 2 \rangle } (\partial {\varvec{\uptheta}}_{0}^{\prime} )^{ \langle 2 \rangle } }}\,{\text{vec}}({\varvec{\Lambda}}^{ - 1} {\varvec{\Gamma \Lambda }}^{ - 1} )\,\, +\, O(n^{ - 3} ). \hfill \\ \end{aligned} $$

  (134)

In 1, 2 and 3, when the model is true ${\text{E}}_{\text{T}} ( \cdot ) = {\text{E}}_{{\theta_{0} }} ( \cdot )$ and $- {\varvec{\Lambda}} = {\varvec{\Gamma}} = {\mathbf{I}}_{0}$. Especially, the term of order $O(n^{ - 1} )$ becomes

$$ \frac{{n^{ - 1} }}{2}{\text{tr}}({\varvec{\Gamma \Lambda }}^{ - 1} ) = - n^{ - 1} \frac{q}{2} . $$

(135)

That is, the expectation is asymptotically smaller than $\bar{l}^{*} ({\varvec{\uptheta}}_{0} ) = \bar{l}_{0}^{*}$ by $n^{ - 1} q/2$ up to this order.