Correction for misclassification of caries experience in the absence of internal validation data

Abstract

Objectives

To quantify the effects of risk factors and/or determinants on disease occurrence, it is important that the risk factors as well as the variable that measures the disease outcome are recorded with the least error as possible. When investigating the factors that influence a binary outcome, a logistic regression model is often fitted under the assumption that the data are collected without error. However, most categorical outcomes (e.g., caries experience) are accompanied by misclassification and this needs to be accounted for. The aim of this research was to adjust for binary outcome misclassification using an external validation study when investigating factors influencing caries experience in schoolchildren.

Materials and methods

Data from the Signal Tandmobiel® study were used. A total of 500 children from the main and 148 from the validation study were included in the analysis. Regression models (with several covariates) for sensitivity and specificity were used to adjust for misclassification in the main data.

Results

The use of sensitivity and specificity modeled as functions of several covariates resulted in a better correction compared to using point estimates of sensitivity and specificity. Age, geographical location of the school to which the child belongs, dentition type, tooth type, and surface type were significantly associated with the prevalence of caries experience.

Conclusions

Sensitivity and specificity calculated based on an external validation study may resemble those obtained from an internal study if conditioned on a rich set of covariates.

Clinical relevance

Main data can be corrected for misclassification using information obtained from an external validation study when a rich set of covariates is recorded during calibration.

Appendix

Multilevel model assuming error-free CE data

Let Y _{M, stme} be the true CE score of surface s, (s = 1, …, n _t ) nested in tooth t = 1, …, n _m , which is nested in child/mouth m = 1, …, N _M according to examiner e, (e = 1, …, n _e ) in the main study. The model uses ${\pi_{\mathrm{stme}}}=Pr\left( {{Y_{\mathrm{M},\mathrm{stme}}}=1\left| {\boldsymbol{\upbeta}, {{\mathbf{x}}_{\mathrm{stme}}},{u_{\mathrm{m}}},\ {u_{\mathrm{tm}}},{u_{\mathrm{e}}}} \right.} \right)$, which is the true conditional probability for CE on surface s nested in tooth t in mouth m by examiner e. The multilevel logistic model for the true main data is then given by:

$$ \mathrm{logit}\left( {{\pi_{\mathrm{stme}}}} \right)=\mathbf{x}_{\mathrm{stme}}^T\boldsymbol{\upbeta} +{u_{\mathrm{m}}}+{u_{\mathrm{tm}}}+{u_{\mathrm{e}}} $$

where x _stme represents the risk factors and/or determinants, β is a vector of regression coefficients and it quantifies the effect of the risk factors/or determinants. The quantities u _m, u _tm, and u _e are random intercepts at mouth, tooth, and examiner level and they are independently distributed with mean zero and variances $\sigma_{\mathrm{m}}^2,\sigma_{\mathrm{tm}}^2,\;\mathrm{and}\;\sigma_{\mathrm{e}}^2$, at mouth, tooth (nested in mouth), and examiner level, respectively, i.e., (u _m, u _tm, u _e) ~ N(0, D) where $\mathbf{D}=\mathrm{diag}\left( {\sigma_{\mathrm{m}}^2,\sigma_{\mathrm{tm}}^2,\sigma_{\mathrm{e}}^2} \right)$. They take into account the clustering of teeth within mouths, surfaces within teeth, and an examiner recording many surfaces, respectively.

Dealing with external validation data

There are two types of misclassification, i.e., differential and non-differential. Non-differential misclassification occurs when the misclassification does not depend on determinants [19, 20]. Differential misclassification occurs when misclassification is different in boys and girls.

If scoring in the main and validation studies is done by the same fallible dental examiners, then either the misclassification probabilities are the same in the two studies or the misclassification process is differential (misclassification depending on covariates), but given a rich set of covariates (e.g., gender, dentition type, tooth type, and surface type as for the ST validation study), they become the same in the two studies. If the misclassification probabilities between the two studies are equal, then the external validation data can be immediately used to correct for misclassification in the main model. However, if these misclassification probabilities do differ between the main data and validation data, then the misclassification process is differential and, conditional on a rich set of covariates, the scoring in the two data sets may become identical.

Consider an external validation data set as in the ST study. Assume P ₁ and P ₂ are the misclassification processes in the main study and validation study, respectively. Suppose that subjects are well characterized by a rich covariate vector, z, then under the settings described above, our claim is that often ${P_1}\left( {{Y^{*}}\left| {Y,z} \right.} \right)={P_2}\left( {{Y^{*}}\left| {Y,z} \right.} \right)$ when ${P_1}\left( {{Y^{*}}\left| Y \right.} \right)\ne {P_2}\left( {{Y^{*}}\left| Y \right.} \right)$. Inequality of the (assumed non-differential) misclassification process P ₁ and P ₂ occurs because f ₁(z)  ≠  f ₂(z) in ${P_j}\left( {{Y^{*}}\left| Y \right.} \right)=\int {{P_j}\left( {{Y^{*}}\left| {Y,\mathbf{z}} \right.} \right){f_j}\left( \mathbf{z} \right)d\left( \mathbf{z} \right)}$ for j = 1,2. As a result, the misclassification probabilities become identical given z.

Multilevel model for cross-sectional CE data adjusting for misclassification

Using the estimates of parameters for logistic models of SE and SP from validation data, say α and η, a corrected multilevel logistic model for the main observed data uses $\pi_{\mathrm{stme}}^{*}=Pr\left( {Y_{\mathrm{M},\mathrm{stme}}^{*}=1\left| {\beta, {{\mathbf{x}}_{\mathrm{stme}}},{u_{\mathrm{m}}},\ {u_{\mathrm{tm}}},{u_{\mathrm{e}}},\boldsymbol{\upalpha}, \boldsymbol{\upeta}, \mathbf{z}} \right.} \right)$, which is the observed (corrected for misclassification) probability for CE on surface s nested in tooth t in mouth m from the main data set given x _stme and z, a vector of covariates from the main data and validation data, respectively, and random effects u _m, u _tm, and u _e and estimates of SE and SP α and η, respectively. This processing was done in one joint model, i.e., a model that encompasses the estimation of SE and SP and at the same time correcting for misclassification. The corrected model is given by:

$$ \pi_{\mathrm{stme}}^{*}=\left( {1-{\tau_{00 }}} \right)+\left[ {{\tau_{11 }}+{\tau_{00 }}-1} \right]\left[ {{g^{-1 }}\left( {x_{\mathrm{stme}}^T\boldsymbol{\upbeta} +{u_{\mathrm{m}}}+{u_{\mathrm{tm}}}+{u_{\mathrm{e}}}} \right)} \right] $$

where τ ₁₁ = τ ₁₁(z) and τ ₀₀ = τ ₀₀(z) are the differential SE and SP.

Bayesian estimation approach

The posterior summary measures of the parameters are obtained using a sampling approach called the Markov Chain Monte Carlo (MCMC) approach [21]. Here, non-informative or vague priors were used which express that there is no prior information on the parameters. For this purpose, JAGS 3.1.0 [22] software was used. Three MCMC chains were run, each for 100,000 iterations for each model. The convergence of these MCMC chains was checked using the CODA package (see [23]) in R. In particular, the Gelman and Rubin diagnostics measure $\widehat{\mathrm{R}}$ was used and this value was close to 1 for all the parameters, which means there was no evidence against convergence. Finally, a sensitivity analysis on the model corrected for misclassification was performed. Specifically, a sensitivity analysis was performed by changing the prior distributions for fixed effects. This was done in order to check whether the model was robust to some perturbations.