Industrial Growth and Productivity Change in German Cities: A Multilevel Investigation

Abstract

The role of productivity change and city-specific characteristics on economic growth are analyzed for German cities. Productivity change is measured by the Malmquist index and its components, which are estimated by non-parametric data envelopment analysis. The nested structure as well as the interaction between industries within cities and over time is accounted for by estimating multilevel models. It is shown that there are differences for industrial growth for different cities and years. Therefore, the use of multilevel models is required. Schumpeter’s creative destruction is found to hold for efficiency change on industrial growth. Efficiency change measures the catching-up to the best practice production function, reducing both value added growth and employment growth. Technological progress shifts the best practice production function and leads only to a rise in value added growth and not in employment growth. The estimations indicate a converging growth of urban industrial value added while employment growth diverges.

Appendices

Appendix 1: List of Cities Included (Table 12)

Table 12 Cities included with average population

Appendix 2: Multilevel Model Estimation

In general and in the formulation of Pinheiro and Bates (2000) a three level model with two levels of random effects is written as

$$\displaystyle{ y_{\mathit{ijt}} =\boldsymbol{ X}_{\mathit{ijt}}\boldsymbol{\beta }_{\mathit{ijt}} +\boldsymbol{ Z}_{\mathit{ij},t}\boldsymbol{b}_{\mathit{ij}} +\boldsymbol{ Z}_{\mathit{ijt}}\boldsymbol{b}_{\mathit{ijt}} + e_{\mathit{ijk}}, }$$

(37)

with $i = 1,\ldots,N$, $j = 1,\ldots,n$, and $t = 2,\ldots,T$, and $\boldsymbol{b}_{\mathit{ij}} \sim N\left (\mathbf{0},\boldsymbol{\Sigma }_{1}\right )$, $\boldsymbol{b}_{\mathit{ijt}} \sim N\left (\mathbf{0},\boldsymbol{\Sigma }_{2}\right )$, $\boldsymbol{e}_{\mathit{ijk}} \sim N\left (0,\sigma ^{2}\boldsymbol{I}\right )$. For simplification the number observations is the same for every level and group so that no observation is missing and it does not vary by lower level groups. In the mixed or random effects literature Eq. ( 37) is written in vector notation for all i as

$$\displaystyle{ \boldsymbol{y}_{\mathit{jt}} =\boldsymbol{ X}_{\mathit{jt}}\boldsymbol{\beta }_{\mathit{jt}} +\boldsymbol{ Z}_{j,t}\boldsymbol{b}_{j} +\boldsymbol{ Z}_{\mathit{jt}}\boldsymbol{b}_{\mathit{jt}} +\boldsymbol{ e}_{\mathit{jt}}. }$$

(38)

Equation ( 37) and accordingly Eq. ( 37) incorporate $\boldsymbol{X}_{\mathit{jt}}$ the regressor matrix for the vector of the p fixed effects $\boldsymbol{\beta }_{\mathit{jt}}$, $\boldsymbol{Z}_{j,t}$ the regressor matrix for the random effects $\boldsymbol{b}_{j}$ of the second level, and $\boldsymbol{Z}_{\mathit{jt}}$ the regressor matrix for the random effect $\boldsymbol{b}_{\mathit{jt}}$ of the third level. The variance-covariance matrices $\boldsymbol{\Sigma }_{l}$ for l = 1, 2 and in each of the two levels of random effects have to be symmetric and positive definite and can be expressed as $\sigma ^{2}\boldsymbol{D}_{l}$ with σ ² the variance of the error term and $\boldsymbol{D}_{l}$ a scaled variance-covariance matrix for the random effects of level l.

The estimation procedure is developed from the simple model with one level of random effects to two levels of random effects and can be extended by further levels of random effects.

For one level of random effects with l = 1 the calculation is performed as follows. The general model equation without the third level denoted with t or the second level of random effects is in vector notation

$$\displaystyle{ y_{\mathit{ij}} =\boldsymbol{ X}_{\mathit{ij}}\boldsymbol{\beta }_{\mathit{ij}} +\boldsymbol{ Z}_{\mathit{ij}}\boldsymbol{b}_{\mathit{ij}} + e_{\mathit{ij}}, }$$

(39)

for $i = 1,\ldots,N$, $j = 1,\ldots,n$, and $\boldsymbol{X}_{\mathit{ij}}$ the $\left (N \cdot n \times p\right )$ regressor matrix for the $\left (p \times 1\right )$ vector of fixed effects $\boldsymbol{\beta }_{\mathit{ij}}$, $\boldsymbol{Z}_{\mathit{ij}}$ is the $\left (N \cdot n \times q\right )$ regressor matrix for the q random effects $\boldsymbol{b}_{\mathit{ij}}$. In notation for all i as vector it follows

$$\displaystyle{ \boldsymbol{y}_{j} =\boldsymbol{ X}_{j}\boldsymbol{\beta }_{j} +\boldsymbol{ Z}_{j}\boldsymbol{b}_{j} +\boldsymbol{ e}_{j}, }$$

(40)

for $j = 1,\ldots,n$. As Lindstrom and Bates (1988) show in general without restriction on the error term structure $\boldsymbol{e}_{j} \sim N\left (\mathbf{0},\sigma ^{2}\boldsymbol{\Lambda }\right )$ where $\boldsymbol{\Lambda }$ is of size N × N and does not have to be the identity matrix $\boldsymbol{I}$

$$\displaystyle{y_{j}\vert \boldsymbol{b}_{j} \sim N\left (\boldsymbol{X}_{j}\boldsymbol{\beta }_{j} +\boldsymbol{ Z}_{j}\boldsymbol{b}_{j},\sigma ^{2}\boldsymbol{\Lambda }_{ j}\right ),\,j = 1,\,\ldots,\,n.}$$

For all j, it becomes in vector notation

$$\displaystyle{\boldsymbol{y}\vert \boldsymbol{b} \sim N\left (\boldsymbol{X}\boldsymbol{\beta } +\boldsymbol{ Z}\boldsymbol{b},\sigma ^{2}\boldsymbol{\Lambda }\right )}$$

with$\boldsymbol{Z} =\mathrm{ diag}\left (\boldsymbol{Z}_{1},\boldsymbol{Z}_{2},\ldots,\boldsymbol{Z}_{n}\right )$, $\boldsymbol{\Lambda } =\mathrm{ diag}\left (\boldsymbol{\Lambda }_{1},\boldsymbol{\Lambda }_{2},\ldots,\boldsymbol{\Lambda }_{n}\right )$ and $\boldsymbol{b} \sim N\left (0,\sigma ^{2}\boldsymbol{\Sigma }\right )$

$$\displaystyle{ \boldsymbol{y} \sim N\left (\boldsymbol{X}\boldsymbol{\beta },\boldsymbol{D}\right ),\boldsymbol{D} =\sigma ^{2}\left (\boldsymbol{Z}\boldsymbol{\Sigma }\boldsymbol{Z}^{{\prime}} +\boldsymbol{ \Lambda }\right ) }$$

(41)

The likelihood function is

$$\displaystyle{ L\left (\boldsymbol{\beta },\boldsymbol{\theta },\sigma ^{2}\vert \boldsymbol{y}\right ) =\prod _{ j=1}^{n}p\left (y_{ j}\vert \boldsymbol{\beta },\boldsymbol{\theta },\sigma ^{2}\right ). }$$

(42)

In Eq. ( 42) $\boldsymbol{\theta }$ contains the unique elements of $\boldsymbol{\Sigma }$ and the parameters in $\boldsymbol{\Lambda }$ which are the variance components without exact specification (Harville 1977; Lindstrom and Bates 1990). Because $\boldsymbol{b}_{j}$ and $\boldsymbol{e}_{j}$ are independent, as Eq. ( 41) indicates, Eq. ( 42) results in

$$\displaystyle\begin{array}{rcl} L\left (\boldsymbol{\beta },\boldsymbol{\theta },\sigma ^{2}\vert \boldsymbol{y}\right )& =& \prod _{ j=1}^{n}\int p\left (y_{ j}\vert \boldsymbol{b}_{j},\boldsymbol{\beta },\sigma ^{2}\right )p\left (\boldsymbol{b}_{ j}\vert \boldsymbol{\theta },\sigma ^{2}\right )d\boldsymbol{b}_{ j} \\ & =& \prod _{j=1}^{n}\int \frac{\exp (-\left \Vert \boldsymbol{y}_{j} -\boldsymbol{ X}_{j}\boldsymbol{\beta }_{j} + \mathbf{Z}_{j}\boldsymbol{b}_{j}\right \Vert ^{2}/2\sigma ^{2})} {\left (2\pi \sigma ^{2}\right )^{N/2}} \\ & & \times \frac{\exp \left (-\boldsymbol{b}_{j}^{{\prime}}\boldsymbol{D}^{-1}\boldsymbol{b}_{j}/2\sigma ^{2}\right )} {\left (2\pi \sigma ^{2}\right )^{q/2}\sqrt{\left \vert \boldsymbol{D} \right \vert }} d\boldsymbol{b}_{j} \\ & =& \prod _{j=1}^{n} \frac{1} {\sqrt{\left (2\pi \sigma ^{2 } \right ) ^{N/2}}} \\ & & \times \int \frac{\exp \left [\frac{-1} {2\sigma ^{2}} \left (\left \Vert y_{j} -\boldsymbol{ X}_{j}\boldsymbol{\beta } -\boldsymbol{ Z}_{j}\boldsymbol{b}_{j}\right \Vert ^{2} +\boldsymbol{ b}_{j}^{{\prime}}\boldsymbol{D}^{-1}\boldsymbol{b}_{j}\right )\right ]} {\left (2\pi \sigma ^{2}\right )^{q/2}\sqrt{\left \vert \boldsymbol{D} \right \vert }} d\boldsymbol{b}_{j} \\ & =& \prod _{j=1}^{n} \frac{1} {\sqrt{\left (2\pi \sigma ^{2 } \right ) ^{N/2}}} \\ & & \times \int \frac{\exp \left [\frac{-1} {2\sigma ^{2}} \left (\left \Vert y_{j} -\boldsymbol{ X}_{j}\boldsymbol{\beta } -\boldsymbol{ Z}_{j}\boldsymbol{b}_{j}\right \Vert ^{2} -\left \Vert \Delta \boldsymbol{b}_{j}\right \Vert ^{2}\right )\right ]} {\left (2\pi \sigma ^{2}\right )^{q/2}\mathsf{abs}\left \vert \Delta \right \vert ^{-1}} d\boldsymbol{b}_{j} \\ & =& \prod _{j=1}^{n} \frac{\mathsf{abs}\left \vert \Delta \right \vert } {\sqrt{\left (2\pi \sigma ^{2 } \right ) ^{N/2}}} \\ & & \times \int \frac{\exp \left [\frac{-1} {2\sigma ^{2}} \left (\left \Vert \tilde{\boldsymbol{y}}_{j} -\boldsymbol{\tilde{ X}}_{j}\boldsymbol{\beta } -\boldsymbol{\tilde{ Z}}_{j}\boldsymbol{b}_{j}\right \Vert ^{2}\right )\right ]} {\left (2\pi \sigma ^{2}\right )^{q/2}} d\boldsymbol{b}_{j}, {}\end{array}$$

(43)

with $\tilde{\boldsymbol{y}}_{j} = \left [\begin{array}{c} \boldsymbol{y}_{j} \\ \mathbf{0} \end{array} \right ],\tilde{\boldsymbol{X}}_{j} = \left [\begin{array}{c} \boldsymbol{X}_{j} \\ \mathbf{0}\end{array} \right ],\tilde{\boldsymbol{Z}}_{j} = \left [\begin{array}{c} \boldsymbol{Z}_{j} \\ \boldsymbol{\Delta }\end{array} \right ]$ as pseudo data, where $\boldsymbol{\Delta }$ a relative precision factor as the Cholesky factor of $\boldsymbol{D}^{-1}$, since $\boldsymbol{b}_{j}^{{\prime}}\boldsymbol{D}^{-1}\boldsymbol{b}_{j} = \left \Vert \boldsymbol{\bigtriangleup }\boldsymbol{b}_{j}\right \Vert ^{2} = \left \Vert \mathbf{0} -\mathbf{0}\boldsymbol{\beta } -\boldsymbol{\bigtriangleup }\boldsymbol{b}_{j}\right \Vert ^{2}$ and therefore $\boldsymbol{D}^{-1} = \Delta ^{{\prime}}\Delta$ (Lindstrom and Bates 1990).

So the exponent is the sum of squared residuals ($\left \Vert a\right \Vert = \sqrt{a^{{\prime} } a}$ as the norm of a matrix). Equation ( 43) clearly points out that the maximization of the log-likelihood requires the minimization of the quadratic norm within the exponential function within the integral. This quadratic norm includes the quadratic error terms and is therefore similar to other least squares problems except that the mean of the random effects have to be zero. To solve that least squares problem numerically the orthogonal-triangular decomposition of rectangular matrices is preferred since it provides stable and efficient results by reducing the condition, i.e. the complexity of $\boldsymbol{X}_{j}$ and $\boldsymbol{Z}_{j}$. The orthogonal-triangular decomposition uses is the QR-decomposition, with $\tilde{\boldsymbol{Z}}_{j} =\boldsymbol{ Q}_{(j)}\left [\begin{array}{c} \boldsymbol{R}_{11(j)} \\ \mathbf{0} \end{array} \right ]$, where $\boldsymbol{Q}_{(j)}$ is a $\left (N + q\right ) \times \left (N + q\right )$ orthogonal matrix $\left (Q_{(j)}^{{\prime}} = Q_{(j)}^{-1}\right )$ and $\boldsymbol{R}_{11(j)}$ is an upper-triangular $\left (q \times q\right )$ matrix. This decomposition can be performed for every real matrix but in the case for positive elements in $\boldsymbol{R}_{11(j)}$ have to be invertible, so $\tilde{\boldsymbol{Z}}_{j}$ has to have full rank as for OLS regression there must not be any linear dependency structure within the random variables. Also $\tilde{\boldsymbol{X}}_{j} =\boldsymbol{ Q}_{(j)}\left [\begin{array}{c} \boldsymbol{R}_{10(j)} \\ \boldsymbol{R}_{00(j)} \end{array} \right ]$ and $\tilde{\boldsymbol{y}}_{j} =\boldsymbol{ Q}_{(j)}\left [\begin{array}{c} \boldsymbol{c}_{1(j)} \\ \boldsymbol{c}_{0(j)} \end{array} \right ]$. Therefore, it is also possible to orthogonal triangular decomposition (QR) of an augmented matrix

$$\displaystyle{\left [\begin{array}{ccc} \boldsymbol{Z}_{j}&\boldsymbol{X}_{j}&\boldsymbol{y}_{j} \\ \boldsymbol{\Delta } & \mathbf{0} & \mathbf{0} \end{array} \right ] = \left (\begin{array}{ccc} \tilde{\boldsymbol{Z}}_{j}&\tilde{\boldsymbol{X}}_{j}&\tilde{\boldsymbol{y}}_{j} \end{array} \right ) =\boldsymbol{ Q}_{(j)}\left [\begin{array}{ccc} \boldsymbol{R}_{11(j)} & \boldsymbol{R}_{10(j)} & \boldsymbol{c}_{1(j)} \\ \mathbf{0} &\boldsymbol{R}_{00(j)} & \boldsymbol{c}_{0(j)} \end{array} \right ]}$$

or

$$\displaystyle{\boldsymbol{Q}_{(j)}^{-1}\left (\begin{array}{ccc} \tilde{\boldsymbol{Z}}_{j}&\tilde{\boldsymbol{X}}_{j}&\tilde{\boldsymbol{y}}_{j} \end{array} \right ) = \left [\begin{array}{ccc} \boldsymbol{R}_{11(j)} & \boldsymbol{R}_{10(j)} & \boldsymbol{c}_{1(j)} \\ \mathbf{0} &\boldsymbol{R}_{00(j)} & \boldsymbol{c}_{0(j)} \end{array} \right ].}$$

The exponent in Eq. ( 43) becomes

$$\displaystyle\begin{array}{rcl} \left \Vert \tilde{\boldsymbol{y}}_{j} -\tilde{\boldsymbol{ X}}_{j}\boldsymbol{\beta } -\tilde{\boldsymbol{ Z}}_{j}\boldsymbol{b}_{j}\right \Vert ^{2}& =& \left \Vert \boldsymbol{Q}_{ (j)}^{{\prime}}\left (\tilde{\boldsymbol{y}}_{ j} -\tilde{\boldsymbol{ X}}_{j}\boldsymbol{\beta } -\tilde{\boldsymbol{ Z}}_{j}\boldsymbol{b}_{j}\right )\right \Vert ^{2} {}\\ & =& \left \Vert \boldsymbol{c}_{1(j)} -\boldsymbol{ R}_{10(j)}\boldsymbol{\beta } -\boldsymbol{ R}_{11(j)}\boldsymbol{b}_{j}\right \Vert ^{2} + \left \Vert \boldsymbol{c}_{ 0(j)} -\boldsymbol{ R}_{00(j)}\boldsymbol{\beta }\right \Vert. {}\\ \end{array}$$

Thus the integral in Eq. ( 43) can be expressed as

$$\displaystyle{ \exp \left [\frac{\left \Vert \boldsymbol{c}_{0(j)} -\boldsymbol{ R}_{00(j)}\boldsymbol{\beta }\right \Vert ^{2}} {-2\sigma ^{2}} \right ]\int \frac{\exp \left [\frac{-1} {2\pi \sigma ^{2}} \left (\left \Vert \boldsymbol{c}_{1(j)} -\boldsymbol{ R}_{10(j)}\boldsymbol{\beta } -\boldsymbol{ R}_{11(j)}\boldsymbol{b}_{j}\right \Vert ^{2}\right )\right ]} {\left (2\pi \sigma ^{2}\right )^{q/2}} d\boldsymbol{b}_{j}. }$$

(44)

Note because $\boldsymbol{R}_{11(j)}$ is a non-singular, Bates and Pinheiro construct the following variable

$\boldsymbol{\phi }_{j} = \left (\boldsymbol{c}_{1(j)} -\boldsymbol{ R}_{10(j)}\boldsymbol{\beta } - n\boldsymbol{R}_{11(j)}\boldsymbol{b}_{1}\right )/\sigma$ with $d\boldsymbol{\phi }_{j} =\sigma ^{-q}\mathrm{abs}\vert \boldsymbol{R}_{11(j)}\vert d\boldsymbol{b}_{j}$ to easily eliminate the integral. The integral expressed in Eq. ( 44) is

$$\displaystyle\begin{array}{rcl} & & \int \frac{\exp \left [\frac{-1} {2\pi \sigma ^{2}} \left (\left \Vert \boldsymbol{c}_{1(j)} - n\boldsymbol{R}_{10(j)}\boldsymbol{\beta } -\boldsymbol{ R}_{11(j)}\boldsymbol{b}_{j}\right \Vert ^{2}\right )\right ]} {\left (2\pi \sigma ^{2}\right )^{q/2}} d\boldsymbol{b}_{j} {}\\ & & \quad = \frac{1} {\mathrm{abs}\vert \boldsymbol{R}_{11(j)}\vert }\int \frac{\exp \left (-\left \Vert \boldsymbol{\phi }_{j}\right \Vert ^{2}/2\right )} {\left (2\pi \right )^{q/2}} d\boldsymbol{\phi }_{j} {}\\ & & \quad =\mathrm{ abs}\vert \boldsymbol{R}_{11(j)}\vert ^{-1} {}\\ \end{array}$$

because the integral is over a standard normal distribution, which is unity over the whole range.

And because the determinant of $\boldsymbol{R}_{11(j)}$ is the sum of its diagonal elements since it is an upper-triangular matrix by construction of QR decomposition. So altogether the likelihood function becomes

$$\displaystyle{L\left (\boldsymbol{\beta },\boldsymbol{\theta },\sigma ^{2}\vert \boldsymbol{y}\right ) =\prod _{ j=1}^{n}\frac{\exp \left [\frac{\left \Vert \boldsymbol{c}_{0j}-\boldsymbol{R}_{00(j)}\boldsymbol{\beta }\right \Vert ^{2}} {-2\sigma ^{2}} \right ]} {\sqrt{\left (2\pi \sigma ^{2 } \right ) ^{N } \vert \boldsymbol{D}\vert }}\mathrm{abs}\vert \boldsymbol{R}_{11(j)}\vert ^{-1}.}$$

A further QR decomposition can be performed by

$$\displaystyle{\left [\begin{array}{cc} \boldsymbol{R}_{00(1)} & \boldsymbol{c}_{0(1)}\\ \vdots & \vdots \\ \boldsymbol{R}_{00(M)} & \boldsymbol{c}_{0(M)} \end{array} \right ] =\boldsymbol{ Q}_{0}\left [\begin{array}{cc} \boldsymbol{R}_{00} & \boldsymbol{c}_{0} \\ \mathbf{0} &\boldsymbol{c}_{-1} \end{array} \right ]}$$

to

$$\displaystyle\begin{array}{rcl} L\left (\boldsymbol{\beta },\boldsymbol{\theta },\sigma ^{2}\vert \boldsymbol{y}\right )& =& \left (2\pi \sigma ^{2}\right )^{-N_{M}/2}\exp \left (\frac{\left \Vert \boldsymbol{c}_{-1}\right \Vert ^{2} + \left \Vert \boldsymbol{c}_{ 0} -\boldsymbol{ R}_{00}\boldsymbol{\beta }\right \Vert ^{2}} {-n2\sigma ^{2}} \right ) {}\\ & & \times \prod _{j=1}^{n}\mathrm{abs}\left ( \frac{\vert \boldsymbol{\Delta }\vert } {\vert \boldsymbol{R}_{11(j)}\vert }\right ) {}\\ \end{array}$$

with $N_{n} =\sum _{ j=1}^{n}N = n \cdot N$ and $1/\sqrt{\vert \boldsymbol{D}\vert } =\mathrm{ abs}\vert \boldsymbol{\Delta }\vert$. The estimate of fixed effects $\boldsymbol{\beta }$ follows from $\left \Vert \boldsymbol{c}_{0} -\boldsymbol{ R}_{00}\boldsymbol{\beta }\right \Vert ^{2}$ and is

$$\displaystyle{\hat{\boldsymbol{\beta }}=\boldsymbol{ R}_{00}^{-1}\boldsymbol{c}_{ 0}}$$

and

$$\displaystyle{\boldsymbol{\sigma }^{2} = \left \Vert \boldsymbol{c}_{ -1}\right \Vert ^{2}/N_{ n}.}$$

Maximum likelihood estimates are then performed by setting an estimate for $\boldsymbol{\theta }$. The random effects are evaluated by

$$\displaystyle{\hat{\boldsymbol{b}}_{j}\left (\boldsymbol{\theta }\right ) =\boldsymbol{ R}_{11(j)}^{-1}\left (\boldsymbol{c}_{ 1j} -\boldsymbol{ R}_{10(j)}\hat{\boldsymbol{\beta }}\left (\boldsymbol{\theta }\right )\right ).}$$

This is the best linear unbiased predictor for the random effects, where $\boldsymbol{\theta }=\hat{\boldsymbol{\theta }}$ as the maximum likelihood estimate.

Lindstrom and Bates (1988, 1990) show the computation for full maximum likelihood and restricted maximum likelihood estimation. Since the maximum likelihood estimation does not account for the loss in degrees of freedom (N _M − p) the estimators are generally downward biased for example if the estimator for the variance component is $\theta _{i}\left (N_{n} - p\right )/N$ its bias is θ _i p∕N _n (Harville 1977). The estimation is therefore performed with the restricted maximum likelihood estimation (REML) sometimes also called residual maximum likelihood which accounts for the degrees of freedom but results in incomparable results if the number of parameters differ. The restricted form as Laird and Ware (1982) and Ware (1985)

$$\displaystyle{ L_{R}\left (\boldsymbol{\theta },\sigma ^{2}\vert \boldsymbol{y}\right ) =\int L\left (\boldsymbol{\beta },\boldsymbol{\theta },\sigma ^{2}\vert \boldsymbol{y}\right )d\boldsymbol{\beta } }$$

(45)

logarithm

$$\displaystyle\begin{array}{rcl} l_{R}\left (\boldsymbol{\theta },\sigma ^{2}\vert \boldsymbol{y}\right )& =& \log L_{ R}\left (\boldsymbol{\theta },\sigma ^{2}\vert \boldsymbol{y}\right ) {}\\ & =& -\frac{N_{n} - p} {2} \log \left (2\pi \sigma ^{2}\right ) -\frac{\left \Vert \boldsymbol{c}_{-1}\right \Vert ^{2}} {2\sigma ^{2}} -\log \mathrm{ abs}\vert \boldsymbol{R}_{00}\vert {}\\ & & +\sum _{j=1}^{n}\log \mathrm{abs}\left ( \frac{\vert \boldsymbol{\Delta }\vert } {\vert \boldsymbol{R}_{11(j)}\vert }\right ). {}\\ \end{array}$$

As the result, the conditional estimate for $\boldsymbol{\beta }$ is

$$\displaystyle{\hat{\boldsymbol{\beta }}=\boldsymbol{ R}_{00}^{-1}\boldsymbol{c}_{ 0}}$$

as the same as in the unconditional case but with $\boldsymbol{R}_{00}^{-1}$ different due to different $\boldsymbol{\Delta }$ and σ ²

$$\displaystyle{\hat{\sigma }_{R}^{2}\left (\boldsymbol{\theta }\right ) = \left \Vert c_{ -1}\right \Vert ^{2}/\left (N_{ t} - p\right ).}$$

So the restricted log-likelihood is

$$\displaystyle\begin{array}{rcl} l_{R}\left (\boldsymbol{\theta }\vert \boldsymbol{y}\right )& =& l_{R}\left (\boldsymbol{\theta },\hat{\sigma }_{RE}^{2}\left (\boldsymbol{\theta }\right )\vert \boldsymbol{y}\right ) {}\\ & =& \mathrm{const} -\left (N_{n} - p\right )\log \left \Vert \boldsymbol{c}_{-1}\right \Vert -\log \mathrm{ abs}\vert \boldsymbol{R}_{00}\vert +\sum _{ j=1}^{n}\log \mathrm{abs}\left ( \frac{\vert \boldsymbol{\Delta }\vert } {\vert \boldsymbol{R}_{11(j)}\vert }\right ). {}\\ \end{array}$$

In both cases the variance of the fixed effect coefficients is

$$\displaystyle{\mathrm{Var}\left (\hat{\boldsymbol{\beta }}\right ) =\hat{\sigma } ^{2}\boldsymbol{R}_{ 00}^{-1}\left (\boldsymbol{R}_{ 00}^{-1}\right )^{{\prime}}.}$$

The integral or respectively the sum becomes clear as soon as we rewrite the likelihood function for one level of random effects in Eq. ( 42) for two levels of random effects namely in my example the city level $j = 1,\ldots,n$ which is nested within the time level $t = 1,\ldots,T$, it becomes

$$\displaystyle\begin{array}{rcl} L\left (\boldsymbol{\beta },\boldsymbol{\theta }_{1},\boldsymbol{\theta }_{2},\sigma ^{2}\vert \boldsymbol{y}\right )& =& \prod _{ t=1}^{T}\int \prod _{ j=1}^{n}\left [\int p\left (\boldsymbol{y}_{\mathit{ jt}}\vert \boldsymbol{b}_{\mathit{jt}},\boldsymbol{b}_{it},\boldsymbol{\beta },\sigma ^{2}\right )p\left (\boldsymbol{b}_{\mathit{ jt}}\vert \boldsymbol{\theta }_{2},\sigma ^{2}\right )d\boldsymbol{b}_{\mathit{ jt}}\right ] \\ & & \times p\left (\boldsymbol{b}_{t}\vert \boldsymbol{\theta }_{1},\sigma ^{2}\right )d\boldsymbol{b}_{ t}. {}\end{array}$$

(46)

Decomposition is constructed similar to the case with one level of random effects

$$\displaystyle\begin{array}{rcl} \left [\begin{array}{cccc} \boldsymbol{Z}_{\mathit{jt}} & \boldsymbol{Z}_{j,t}&\boldsymbol{X}_{\mathit{jt}} & \boldsymbol{y}_{\mathit{jt}} \\ \boldsymbol{\Delta }_{2} & \mathbf{0} & \mathbf{0} & \mathbf{0} \end{array} \right ]& =& \boldsymbol{Q}_{\mathit{jt}}\left [\begin{array}{cccc} \boldsymbol{R}_{22(\mathit{jt})} & \boldsymbol{R}_{21(\mathit{jt})} & \boldsymbol{R}_{20(\mathit{jt})} & \boldsymbol{c}_{2(\mathit{jt})} \\ \mathbf{0} &\boldsymbol{R}_{11(\mathit{jt})} & \boldsymbol{R}_{10(\mathit{jt})} & \boldsymbol{c}_{1(\mathit{jt})} \end{array} \right ],\, {}\\ j& =& 1,\,\ldots,\,n,\,t = 1,\,\ldots,\,T {}\\ \end{array}$$

decomposition for that

$$\displaystyle{\left [\begin{array}{ccc} \boldsymbol{R}_{11(1t)} & \boldsymbol{R}_{10(1t)} & \boldsymbol{c}_{1(1t)}\\ \vdots & \vdots & \vdots \\ \boldsymbol{R}_{11(\mathit{Mt})} & \boldsymbol{R}_{1(\mathit{Mt})} & \boldsymbol{c}_{1\mathit{Mt})} \\ \boldsymbol{\Delta }_{1} & \mathbf{0} & \mathbf{0} \end{array} \right ] = Q_{(i)}\left [\begin{array}{ccc} \boldsymbol{R}_{11(t)} & \boldsymbol{R}_{10(t)} & \boldsymbol{c}_{1(t)} \\ 0 &\boldsymbol{R}_{00(t)} & \boldsymbol{c}_{0(t)} \end{array} \right ]}$$

the profiled log-likelihood becomes

$$\displaystyle\begin{array}{rcl} l_{R}\left (\boldsymbol{\theta }_{1},\boldsymbol{\theta }_{2}\vert \boldsymbol{y}\right )& =& \log L_{R}\left (\hat{\boldsymbol{\beta }}_{R}\left (\boldsymbol{\theta }_{1},\boldsymbol{\theta }_{2}\right ),\boldsymbol{\theta }_{1},\boldsymbol{\theta }_{2},\hat{\sigma }_{R}^{2}\left (\boldsymbol{\theta }_{ 1},\boldsymbol{\theta }_{2}\right )\vert \boldsymbol{y}\right ) {}\\ & =& \mathrm{const} -\left (N_{T} - np\right )\log \left \Vert \boldsymbol{c}_{-1}\right \Vert -\log \mathrm{ abs}\vert \boldsymbol{R}_{00}\vert {}\\ & & +\sum _{t=1}^{T}\log \mathrm{abs}\left ( \frac{\vert \boldsymbol{\Delta }_{1}\vert } {\vert \boldsymbol{R}_{11(t)}\vert }\right ) +\sum _{ t=1}^{T}\sum _{ j=1}^{n}\log \mathrm{abs}\left ( \frac{\vert \boldsymbol{\Delta }_{2}\vert } {\vert \boldsymbol{R}_{22(\mathit{jt})}\vert }\right ), {}\\ \end{array}$$

with $N_{T} = N \cdot n \cdot T$ the total number of observations. Compared to the two level model, the three level model just adds the last addend for the nested higher level.

The solution is straight forward according to one level estimation.

Multilevel models are solved by EM algorithm, which is an iteration of two steps, namely the expectation and maximization (Laird et al. 1987). The data are fitted to the model within the expectation step by estimating the fixed effects, random effects, and the pseudo data ($\tilde{\boldsymbol{y}}_{j}$, $\tilde{\boldsymbol{X}}_{j}$, and $\tilde{\boldsymbol{Z}}_{j}$) to the current values of variance components $\hat{\boldsymbol{\theta }}$. The maximization step fits the parameter $\boldsymbol{\theta }$ of the model to the data by maximizing the likelihood to achieve new variance component parameters $\hat{\boldsymbol{\theta }}$ for the expectation step (Laird and Ware 1982; Lindstrom and Bates 1988).

As described in Laird and Ware (1982) and Lindstrom and Bates (1990) it starts by setting an initial value for $\boldsymbol{\theta }$ within the maximization-step. The error term depends on those variance components in $\boldsymbol{\hat{\theta }}$ which is straightforward $\boldsymbol{e}_{j} =\boldsymbol{ y}_{j} -\boldsymbol{ X}_{j}\boldsymbol{\beta }_{j}\left (\hat{\boldsymbol{\theta }}\right ) -\boldsymbol{ Z}_{j}\boldsymbol{b}_{j}\left (\hat{\boldsymbol{\theta }}\right )$. The expectation-step consists of estimation of the variance components namely for the error terms and the random effects, they basically are presented as in Laird and Ware (1982)

$$\displaystyle{ \mathsf{E}\left (\sum _{j=1}^{n}\boldsymbol{e}_{ j}^{T}\boldsymbol{e}_{ j}\mid \boldsymbol{y}_{j},\hat{\boldsymbol{\theta }}\right ) =\sum _{ j=1}^{n}\boldsymbol{e}_{ j}^{T}\left (\hat{\boldsymbol{\theta }}\right )\boldsymbol{e}_{ j}\left (\hat{\boldsymbol{\theta }}\right ) + \mathsf{tr}\mathsf{\,var}\left (\boldsymbol{e}_{j}\mid \boldsymbol{y}_{j},\hat{\boldsymbol{\theta }}\right ) }$$

(47)

and

$$\displaystyle{ \mathsf{E}\left (\sum _{j=1}^{n}\boldsymbol{b}_{ j}\boldsymbol{b}_{j}^{T}\mid \boldsymbol{y}_{ j},\hat{\boldsymbol{\theta }}\right ) =\sum _{ j=1}^{n}\boldsymbol{b}_{ j}\left (\hat{\boldsymbol{\theta }}\right )\boldsymbol{b}_{j}^{T}\left (\hat{\boldsymbol{\theta }}\right ) + \mathsf{var}\left (\boldsymbol{b}_{ j}\mid \boldsymbol{y}_{j},\hat{\boldsymbol{\theta }}\right ). }$$

(48)

The maximization steps then use the log-nlikelihood function depending on whether estimating by maximum likelihood or restricted maximum likelihood as presented above or in Lindstrom and Bates (1990) for both estimation in general and with computational improvements in Laird et al. (1987) as implemented in current software to achieve faster convergence.

Appendix 3: Residual Plots for Employment Growth at City and Time Level (Figs. 4 and 5)

Fig. 4

Residual plot for employment growth at city level

Fig. 5

Residual plot for employment growth at time level