Population-based absolute risk estimation with survey data

Abstract

Absolute risk is the probability that a cause-specific event occurs in a given time interval in the presence of competing events. We present methods to estimate population-based absolute risk from a complex survey cohort that can accommodate multiple exposure-specific competing risks. The hazard function for each event type consists of an individualized relative risk multiplied by a baseline hazard function, which is modeled nonparametrically or parametrically with a piecewise exponential model. An influence method is used to derive a Taylor-linearized variance estimate for the absolute risk estimates. We introduce novel measures of the cause-specific influences that can guide modeling choices for the competing event components of the model. To illustrate our methodology, we build and validate cause-specific absolute risk models for cardiovascular and cancer deaths using data from the National Health and Nutrition Examination Survey. Our applications demonstrate the usefulness of survey-based risk prediction models for predicting health outcomes and quantifying the potential impact of disease prevention programs at the population level.

Appendix

Derivatives and Taylor deviates for the piecewise exponential hazard function

To simplify the notation, we express the absolute risk estimate of (4) in a more compact form,

$$\begin{aligned} \hat{\pi }(\tau _{0{n_0}},\tau _{1n_1}; \varvec{x}) = \sum _{q={n_0}}^{n_1} \hat{S}(\tau _q) A_q (1-B_q) \end{aligned}$$

where

$$\begin{aligned} A_q&= \frac{\hat{\lambda }_0^{(1)} (\tau _q) \exp ( \hat{\varvec{\beta }}^{(1)^{\prime }}\varvec{x}^{(1)})}{\sum _m \hat{\lambda }_0^{(m)}(\tau _q) \exp (\hat{\varvec{\beta }}^{(m)^{\prime }}\varvec{x}^{(m)} )},\\ B_q&= \exp \left\{ - \sum _m \hat{\lambda }_0^{(m)} (\tau _q) \exp (\hat{\varvec{\beta }}^{(m)^{\prime }}\varvec{x}^{(m)}) (\tau _{1q} - \tau _{0q} )\right\} , \end{aligned}$$

and $\hat{S}(\tau _q)$ is as defined in Eq. (5). Then the deviates for $\hat{\pi }(\tau _{0{n_0}},\tau _{1n_1}; \varvec{x})$ are

$$\begin{aligned} \begin{array}{ll} \Delta _{ijk} \lbrace \hat{\pi }(\tau _{0{n_0}},\tau _{1n_1}; \varvec{x})\rbrace =&{} \sum \limits _{q={n_0}}^{n_1} \left[ \hat{S}(\tau _q) (1-B_q) \Delta _{ijk} \lbrace A_q \rbrace \right. \\ &{}\left. + A_q (1-B_q) \Delta _{ijk} \lbrace \hat{S}(\tau _q) \rbrace - A_q \hat{S}(\tau _q) \Delta _{ijk} \lbrace B_q \rbrace \right] . \end{array} \end{aligned}$$

Taking each component in turn, the deviates for $A_q$ are

$$\begin{aligned} \Delta _{ijk} \lbrace A_q \rbrace = T_{q}^{-1} \Delta _{ijk} \lbrace T^{(1)}_{q} \rbrace - \frac{T^{(1)}_{q}}{T_{q}^2} \sum _{m=1}^M \Delta _{ijk} \lbrace T^{(m)}_{q} \rbrace \end{aligned}$$

where $T^{(m)}_{q} = \hat{\lambda }_0^{(m)} (\tau _q) \exp ( \hat{\varvec{\beta }}^{(m)^{\prime }}\varvec{x}^{(m)})$ and $T_{q} = \sum _{m=1}^M T^{(m)}_{q}$, with deviates

$$\begin{aligned} \Delta _{ijk} \lbrace T^{(m)}_{q} \rbrace = \exp (\hat{\varvec{\beta }}^{(m)^{\prime }}\varvec{x}^{(m)})\Delta _{ijk} \lbrace \hat{\lambda }_0^{(m)} (\tau _q) \rbrace + \varvec{x}^{(m)} T^{(m)}_{q} \Delta _{ijk} \lbrace \hat{\varvec{\beta }}^{(m)} \rbrace . \end{aligned}$$

The deviates for $B_q$ are

$$\begin{aligned} \Delta _{ijk} \lbrace B_q \rbrace = - \sum _{{m}=1}^M \left[ \exp (\hat{\varvec{\beta }}^{(m)^{\prime }}\varvec{x}^{(m)}) (\tau _{1q} - \tau _{0q} ) B_q\left( \varvec{x}^{(m)} \hat{\lambda }_0^{(m)} (\tau _q) \Delta _{ijk} \lbrace \hat{\varvec{\beta }}^{(m)^{\prime }} \rbrace + \Delta _{ijk} \lbrace \hat{\lambda }_0^{(m)} (\tau _q) \rbrace \right) \right] \end{aligned}$$

For $q>n_0$, we note that $\hat{S}(\tau _q) = \prod _{l=n_0}^{q-1} B_l$ so that

$$\begin{aligned} \Delta _{ijk} \lbrace \hat{S}(\tau _q) \rbrace = \hat{S}(\tau _q) \sum _{l=n_0}^{q-1} B_l^{-1} \Delta _{ijk} \lbrace B_l \rbrace , \end{aligned}$$

and $\Delta _{ijk} \lbrace \hat{S}(\tau _q) \rbrace$ is zero when $q=n_0$.

The Taylor deviates for $A_q$, $B_q$ and $\hat{S}(\tau _q)$ are each functions of $\hat{\lambda }_0^{(m)}$ and $\hat{\varvec{\beta }}^{(m)}$. For $\hat{\lambda }_0^{(m)}$, we have

$$\begin{aligned} \Delta _{ijk} \lbrace \hat{\lambda }_0^{(m)} (\tau _q) \rbrace = D^{(m)}(\tau _q)^{-1} \left[ w_{ijk} \delta ^{(m)}_{ijk} (t_{ijk}) I(\tau _{0q} \le t_{ijk} < \tau _{1q}) - \hat{\lambda }_0^{(m)} (\tau _q) \Delta _{ijk} \lbrace D^{(m)}(\tau _q) \rbrace \right] \end{aligned}$$

where

$$\begin{aligned} \Delta _{ijk} \lbrace D^{(m)}(\tau _q) \rbrace&= \mathcal A _{ijk}(\tau _q) w_{ijk} \exp ( \hat{\varvec{\beta }}^{(m)^{\prime }} \varvec{x}^{(m)}_{ijk} )\nonumber \\&+ \left[ \sum _{i,j,k} \varvec{x}^{(m)}_{ijk} \mathcal A _{ijk}(\tau _q) w_{ijk} \exp ( \hat{\varvec{\beta }}^{(m)^{\prime }}\varvec{x}^{(m)}_{ijk} ) \right] \Delta _{ijk} \lbrace \hat{\varvec{\beta }}^{(m)} \rbrace . \nonumber \\ \end{aligned}$$

(12)

with $\mathcal A _{ijk}(\tau _q)$ defined in Eq. (6). The Taylor deviates for each $\hat{\varvec{\beta }}^{(m)}$ are

$$\begin{aligned} \Delta _{ijk} \lbrace \hat{\varvec{\beta }}^{(m)} \rbrace = \mathcal H (\hat{\varvec{\beta }}^{(m)} )^{-1} w_{ijk} \delta ^{(m)}_{ijk} (t_{ijk}) \lbrace \varvec{x}^{(m)}_{ijk} - \bar{\varvec{H}}(\hat{\varvec{\beta }}^{(m)},t_{ijk}) \rbrace \end{aligned}$$

(13)

where $\mathcal H (\hat{\varvec{\beta }}^{(m)})$ is the second partial derivative of the pseudo-likelihood,

$$\begin{aligned} \mathcal H (\hat{\varvec{\beta }}^{(m)})&= - \left[ \sum _{i,j,k} \varvec{x}^{(m)}_{ijk}{\varvec{x}^{(m)}_{ijk}}^{\prime } h^{(m)}_{ijk} \right] + \bar{\varvec{H}}(\hat{\varvec{\beta }}^{(m)},t_{ijk}) \bar{\varvec{H}}(\hat{\varvec{\beta }}^{(m)},t_{ijk})^{\prime } ,\\ h^{(m)}_{ijk}&= w_{ijk} y_{ijk}(t) \exp ( \hat{\varvec{\beta }}^{(m)^{\prime }} \varvec{x}^{(m)}_{ijk} ) / \sum _{i,j,k} w_{ijk} y_{ijk}(t) \exp ( \hat{\varvec{\beta }}^{(m)^{\prime }} \varvec{x}^{(m)}_{ijk} ), \end{aligned}$$

and $\bar{\varvec{H}}(\hat{\varvec{\beta }}^{(m)},t_{ijk})$ is defined in Eq. (3). Thus, the deviates for $\hat{\varvec{\beta }}^{(m)}$ are equivalent to the per-observation update in a Newton–Raphson optimization algorithm where the objective function is the weighted pseudo-likelihood of the Cox regression model.

Derivatives and Taylor deviates for the semiparametric hazard function

Denote the $N^{(m)}$ ordered observed event times occurring within $[t_0,t_1)$ for the $m$th cause as $u^{(m)}_1 < u^{(m)}_2 < \ldots < u^{(m)}_{N^{(m)}}$. In terms of these event times, Eq. (1) becomes

$$\begin{aligned} \hat{\pi }(t_0,t_1; \varvec{x})&= \sum \limits _{i=1}^{N^{(1)}} \exp (\hat{\varvec{\beta }}^{(1)^{\prime }}\varvec{x}^{(1)} )\hat{\lambda }_0^{(1)}(u^{(1)}_i) \prod _{m=1}^M \left( \frac{\hat{S}_0^{(m)}(u^{(1)}_i)}{\hat{S}_0^{(m)}(u^{(1)}_1)} \right) ^{\exp ( \hat{\varvec{\beta }}^{(m)^{\prime }}\varvec{x}^{(m)} )} \nonumber \\&= \sum \limits _{i=1}^{N^{(1)}} \hat{p}(u^{(1)}_i). \end{aligned}$$

(14)

As with the piecewise model, we determine the derivative and deviates for each component of (14). For the $\hat{\varvec{\beta }}^{(m)}$, the derivate is

$$\begin{aligned} \frac{\partial \hat{\pi }(t_0,t_1;\varvec{x}) }{\partial \hat{\varvec{\beta }}^{(m)}} = \varvec{x}^{(m)} \left[ \hat{\pi }(t_0,t_1;\varvec{x}) + \exp ( \hat{\varvec{\beta }}^{(m)^{\prime }}\varvec{x}^{(m)} ) \sum _{i=1}^{N^{(1)}} \log \left( \hat{S}_0^{(m)}(u^{(1)}_i)/\hat{S}_0^{(m)}(u^{(1)}_1) \right) \hat{p}(u^{(1)}_i) \right] , \end{aligned}$$

when $m=1$ and

$$\begin{aligned} \frac{\partial \hat{\pi }(t_0,t_1;\varvec{x}) }{\partial \hat{\varvec{\beta }}^{(m)}} = \varvec{x}^{(m)} \exp ( \hat{\varvec{\beta }}^{(m)^{\prime }}\varvec{x}^{(m)} ) \sum _{i=1}^{N^{(1)}} \log \left( \hat{S}_0^{(m)}(u^{(1)}_i)/\hat{S}_0^{(m)}(u^{(1)}_1) \right) \hat{p}(u^{(1)}_i) \end{aligned}$$

for competing causes. The Taylor deviates for each $\hat{\varvec{\beta }}^{(m)}$ are the same as given by Eq. (13) of the piecewise model.

The derivatives for the baseline hazard components are

$$\begin{aligned} \frac{\partial \hat{\pi }(t_0,t_1; \varvec{x})}{\partial \hat{\lambda }_0^{(1)}(u^{(1)}_i)} = \hat{\lambda }_0^{(1)}(u^{(1)}_i)^{-1} \hat{p}(u^{(1)}_i). \end{aligned}$$

The Taylor deviates for the baseline hazard of cause $m$ at observed event time $t$ are

$$\begin{aligned} \Delta _{ijk} \lbrace \hat{\lambda }_0^{(m)} (t) \rbrace = \frac{\partial \hat{\lambda }_0^{(m)} (t)}{\partial N^{(m)}(t)} \Delta _{ijk} \lbrace N^{(m)}(t) \rbrace + \frac{\partial \hat{\lambda }_0^{(m)} (t)}{\partial Y^{(m)}(t)} \Delta _{ijk} \lbrace Y^{(m)}(t) \rbrace , \end{aligned}$$

where

$$\begin{aligned} N^{(m)}(t) = \sum _{i,j,k} w_{ijk} y_{ijk}(t) \delta ^{(m)}_{ijk}(t) \end{aligned}$$

and

$$\begin{aligned} Y^{(m)}(t) = \sum _{i,j,k} w_{ijk} y_{ijk}(t) \exp ( \hat{\varvec{\beta }}^{(m)^{\prime }} \varvec{x}^{(m)}_{ijk} ). \end{aligned}$$

In terms of these quantities, the Taylor deviates are

$$\begin{aligned} \Delta _{ijk} \lbrace \hat{\lambda }_0^{(m)} (t) \rbrace = Y^{(m)}(t)^{-1} (w_{ijk} y_{ijk}(t) \delta ^{(m)}_{ijk}(t) - \hat{\lambda }_0^{(m)}(t) \Delta _{ijk} \lbrace Y^{(m)}(t) \rbrace ) \end{aligned}$$

with

$$\begin{aligned} \begin{array}{ll} \Delta _{ijk} \lbrace Y^{(m)}(t) \rbrace =&{} w_{ijk} y_{ijk}(t) \exp ( \hat{\varvec{\beta }}^{(m)^{\prime }} \varvec{x}^{(m)}_{ijk} ) \\ &{}+ \left[ \sum \limits _{i,j,k} \varvec{x}_{ijk} w_{ijk} y_{ijk}(t) \exp ( \hat{\varvec{\beta }}^{(m)^{\prime }} \varvec{x}^{(m)}_{ijk} ) \right] \Delta _{ijk} \lbrace \hat{\varvec{\beta }}^{(m)} \rbrace . \end{array} \end{aligned}$$

We note that the hazard deviates for the piecewise and semiparametric model in Eq. (12) are equivalent when each interval of the piecewise model contains exactly one observed event time.

The final components are the survival functions. The derivatives for each $\hat{S}_0^{(m)}(u^{(1)}_j)$ are

$$\begin{aligned} \frac{\partial \hat{\pi }(t_0,t_1;\varvec{x})}{\partial \hat{S}_0^{(m)}(u^{(1)}_j)} = \exp ( \hat{\varvec{\beta }}^{(m)^{\prime }}\varvec{x}^{(m)} ) \hat{S}_0^{(m)}(u^{(1)}_j)^{-1} \hat{p}(u^{(1)}_j). \end{aligned}$$

From the semiparametric estimate of Eq. (7), the Taylor deviates for the baseline survival up to time $u^{(1)}_j$ for the $m$th risk type are

$$\begin{aligned} \Delta _{ijk} \lbrace \hat{S}_0^{(m)} (u^{(1)}_{j}) \rbrace = -\hat{S}_0^{(m)} (u^{(1)}_{j}) \sum _{u^{(m)}_{q}\le u^{(1)}_j} \Delta _{ijk} \lbrace \hat{\lambda }_0^{(m)}(u^{(m)}_{q}) \rbrace . \end{aligned}$$

Combining these results, the expression for the Taylor deviates of $\hat{\pi }(t_0,t_1;\varvec{x})$ are

$$\begin{aligned} \Delta _{ijk} \lbrace \hat{\pi }(t_0,t_1; \varvec{x}) \rbrace&= \sum _{m=1}^M \frac{\hat{\pi }(t_0,t_1; \varvec{x})}{\partial \hat{\varvec{\beta }}^{(m)}} \Delta _{ijk} \lbrace \hat{\varvec{\beta }}^{(m)} \rbrace + \sum _{{q}=1}^{N^{(1)}} \frac{\hat{\pi }(t_0,t_1; \varvec{x})}{\partial \hat{\lambda }_0^{(1)} (u^{(1)}_{q}) } \Delta _{ijk} \lbrace \hat{\lambda }_0^{(1)} (u^{(1)}_l) \rbrace \\&+ \sum _{{q}=1}^{N^{(1)}} \sum _{m=1}^M \frac{\hat{\pi }(t_0,t_1; \varvec{x})}{\partial \hat{S}_0^{(m)} (u^{(1)}_{q})} \Delta _{ijk} \lbrace \hat{S}_0^{(m)} (u^{(1)}_{q}) \rbrace . \end{aligned}$$