The inappropriateness of conventional use of the correlation coefficient in assessing validity and reliability of dietary assessment methods

Abstract

Accuracy and precision of nutritional data are crucial in estimating effects in nutritional epidemiology. Because it is known that such data are usually flawed, studies have been designed to estimate both the validity of diet assessment methods in measuring “true” diet and the reliability of these methods in providing nutrient data that are at least reproducible. In these studies, validity and reliability have often been gauged by computing correlation coefficients between two or more estimates of diet and testing the coefficeint's departure from 0. We propose that the correlation coefficient may be inappropriate in these studies as a measure of association. If correlation coefficients are presented, we suggest that one should also present confidence intervals and test the departure of the coefficient from approximately 1 rather than 0. We have examined this approach using dietary data from various studies. We have computed 95% confidence intervals of the correlation coefficients and have tested H_o:rho=0.95 as an approximation of rho=1.00. In all of the studies selected, comparisons produced correlation coefficients statistically significantly different from both 0.95 and 0. Due to the dependence of the correlation coefficient on factors unique to individual studies, it is recommended that other techniques be used to assess agreement between nutrient scores derived in reliability or validation studies. Viable options include linear regression, analyses of the standard deviations of the differences between scores, and examinations of the intraclass correlation coefficient.