Abstract
Objective
To evaluate the reproducibility and validity of the food-frequency questionnaire (FFQ) used in the California Teachers Study (CTS) cohort and to use this data to quantify the effects of correcting nutrient-breast cancer relative risks for measurement error.
Methods
One hundred and ninety five CTS cohort members participated in a 10-month dietary validation study that included four 24-h dietary recalls and pre- and post-study FFQs. Shrout–Fleiss intraclass correlations for reproducibility were computed. Under several standard assumptions concerning the correlations of errors in the FFQs and 24-h recalls, we calculated energy-adjusted deattenuated Pearson correlations for validity and tested for differences in validity according to a number of demographic and other risk factors. For each nutrient, we compared the performance of the FFQ versus the 24-h recalls, estimating the number of days of recalls that give equivalent information about true intake as does a single FFQ. Finally, the effects of adjustment for measurement error on risk estimates were evaluated in 44,423 postmenopausal cohort members, 1,544 of whom developed breast cancer during seven years of follow-up. Relative risks (RR) and confidence intervals (CI) were calculated using Cox proportional hazards with and without correction for measurement error.
Results
Reproducibility correlations for the nutrients ranged from 0.60 to 0.87. With a few exceptions, validity correlations were reasonably high (range: 0.55–0.85), including r = 0.74 for alcohol. Performance of the FFQ differed by age for percent of calories from fat and by body mass index and hormone therapy use for alcohol consumption. For most nutrients examined, our FFQ is comparable to two to six recalls for each subject in capturing true intake. In the measurement error-adjusted risk analyses, corrected RRs were within 13% of uncorrected values for all nutrients examined except for linoleic acid. For alcohol consumption the corrected RR (per 20 g/1,000 kcal/d) was 1.36 (95% CI: 1.03–1.51) compared to the uncorrected estimate of 1.25 (95% CI: 1.10–1.42).
Conclusion
The FFQ dietary assessment used in the CTS is reproducible and valid for all nutrients except the unsaturated fatty acids. Correcting relative risk estimates for measurement error resulted in relatively small changes in the associations between the majority of nutrients and the risk of postmenopausal breast cancer.
Similar content being viewed by others
References
Stram DO, Hankin JH, Wilkens LR et al (2000) Calibration of the dietary questionnaire for a multiethnic cohort in Hawaii and Los Angeles. Am J Epidemiol 151:358–370
Munger RG, Folsom AR, Kushi LH, Kaye SA, Sellers TA (1992) Dietary assessment of older Iowa women with a food frequency questionnaire: nutrient intake, reproducibility, and comparison with 24-hour dietary recall interviews. Am J Epidemiol 136:192–200
Smith-Warner S, Spiegelman D, Ritz J et al (2006) Methods for pooling of results of epidemiologic studies: the pooling project of prospective studies of diet and cancer. Am J Epidemiol 163:1053–1064
Willett WC, Sampson L, Browne ML et al (1988) The use of a self-administered questionnaire to assess diet four years in the past. Am J Epidemiol 127:188–199
Willett WC (1998) Nutritional epidemiology. Oxford University Press, New York
Spiegelman D, McDermott A, Rosner B (1997) Regression calibration method for correcting measurement-error bias in nutritional epidemiology. Am J Clin Nutr 65:1179S–1186S
Bernstein L, Anton-Culver H, Deapen D et al (2002) High breast cancer rates among California teachers: results from the California Teachers Study Cohort. Cancer Causes Control 13:625–635
Block G, Hartman AM, Dresser CM, Carroll MD, Gannon J, Gardner L (1986) A data-based approach to diet questionnaire design and testing. Am J Epidemiol 124:453–469
Block G, Woods M, Potosky A, Clifford C (1990) Validation of a self-administered diet history questionnaire using multiple diet records. J Clin Epidemiol 43:1327–1335
Block G, Thompson FE, Hartman AM, Larkin FA, Guire KE (1992) Comparison of two dietary questionnaires validated against multiple dietary records collected during a 1-year period. J Am Diet Assoc 92:686–693
Kumanyika SK, Mauger D, Mitchell DC, Phillips B, Smiciklas-Wright H, Palmer JR (2003) Relative validity of food frequency questionnaire nutrient estimates in the Black Women’s Health Study. Ann Epidemiol 13:111–118
Potischman N, Carroll RJ, Iturria SJ et al (1999) Comparison of the 60- and 100-item NCI-block questionnaires with validation data. Nutr Cancer 34:70–75
Subar AF, Thompson FE, Kipnis V et al (2001) Comparative validation of the Block, Willett, and National Cancer Institute food frequency questionnaires: the Eating at America’s Table Study. Am J Epidemiol 154:1089–1099
Horn-Ross PL, Hoggatt KJ, West DW et al (2002) Recent diet and breast cancer risk: the California Teachers Study (USA). Cancer Causes Control 13:407–415
Horn-Ross PL, Barnes S, Lee VS et al (2006) Reliability and validity of an assessment of usual phytoestrogen consumption (United States). Cancer Causes Control 17:85–93
Kelsey J, Thompson W, Evans A (1986) Methods in observational epidemiology. Oxford, New York
Breslow NE, Day NE (1980) Statistical methods in cancer research, vol. I—the analysis of case–control studies. International Agency for Research on Cancer, Lyon
Shrout P, Fleiss J (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420–428
Rosner B, Spiegelman D, Willett WC (1990) Correction of logistic regression relative risk estimates and confidence intervals for measurement error: the case of multiple covariates measured with error. Am J Epidemiol 132:734–745
Kroke A, Klipstein-Grobusch K, Voss S et al (1999) Validation of a self-administered food-frequency questionnaire administered in the European Prospective Investigation into Cancer and Nutrition (EPIC) Study: comparison of energy, protein, and macronutrient intakes estimated with the doubly labeled water, urinary nitrogen, and repeated 24-h dietary recall methods. Am J Clin Nutr 70:439–447
Kipnis V, Subar AF, Midthune D et al (2003) Structure of dietary measurement error: results of the OPEN Biomarker Study. Am J Epidemiol 158:14–21
Schatzkin A, Kipnis V, Carroll RJ et al (2003) A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: results from the biomarker-based Observing Protein and Energy Nutrition (OPEN) study. Int J Epidemiol 32:1054–1062
Coates R, Monteilh C (1997) Assessments of food-frequency questionnaires in minority populations. Am J Clin Nutr 65:S1108–S1115
Kristal AR, Feng Z, Coates RJ, Oberman A, George V (1997) Associations of race/ethnicity, education, and dietary intervention with the validity and reliability of a food frequency questionnaire. The Women’s Health Trial Feasibility Study in Minority Populations. Am J Epidemiol 146:856–869
Acknowledgments
The authors would like to thank Dr. Esther John for her work in compiling the vitamin D nutrient data and the members of the CTS Steering Committee, who are responsible for the formation and maintenance of the cohort within which this study was conducted, but are not authors on the current article: Hoda Anton-Culver, Rosemary Cress, Dennis Deapen, David Peel, Rich Pinder, Ronald K. Ross (deceased), Dee W. West, William E. Wright, Giske Ursin, and Argyrios Ziogas. This research was supported by Grant R01 CA77398 from the National Cancer Institute and contract 97-10500 from the California Breast Cancer Research Fund. The funding sources did not contribute to the design or conduct of the study, nor to the writing or submission of this manuscript. The collection of cancer incidence data used in this study was supported by the California Department of Health Services as part of the statewide cancer reporting program mandated by California Health and Safety Code Section 103885; the National Cancer Institute’s Surveillance, Epidemiology and End Results Program under contract N01-PC-35136 awarded to the Northern California Cancer Center, contract N01-PC-35139 awarded to the University of Southern California, and contract N02-PC-15105 awarded to the Public Health Institute; and the Centers for Disease Control and Prevention’s National Program of Cancer Registries, under agreement #U55/CCR921930-02 awarded to the Public Health Institute. The ideas and opinions expressed herein are those of the authors and endorsement by the State of California, Department of Health Services, the National Cancer Institute, and the Centers for Disease Control and Prevention or their contractors and subcontractors is not intended nor should be inferred.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
Estimation of the number of days, n, of 24-h recalls a FFQ report is worth
We define n by equating the correlation, r, between the true nutrient intake and the FFQ with the correlation between true nutrient intake and n 24-h recalls (which increases as n increases). Note that this is equivalent to equating Var(True Intake|FFQ value) to Var(True Intake|n 24-h recalls) since 1 − r2 is equal to Var(True|Estimated)/Var(True), i.e., it is the “unexplained” portion of the variance of True not captured by the estimate. If we assume that all variables are distributed normally then we can compute both Var(True Intake|FFQ value) as well as Var(True Intake|n 24-h recalls) by a standard variance components analysis.
Let Xi be true intake of the nutrient, then by fitting a random intercept model (as in SAS proc mixed) to the four recalls (Rij, j = 1,4, i = 1, N) per subject, then Rij is estimated by:
where εij is the random error in our measurement. We can estimate Var(X) as the variance of the random intercept and Var(ε) as the residual variance (i.e., the variance of one 24-h recall given true intake). By an elementary application of conditional probability for normal random variables, and defining \( \ifmmode\expandafter\bar\else\expandafter\=\fi{R}_{i} \) = Rij/n, we have
Note also that by fitting a second-random intercept model
and equating Xi = a + bFFQi + Wi (so that the random intercept W is the portion of true X unexplained by the regression on the FFQ value), we can estimate Var(X|FFQ) ≡ Var(W) as the variance of the random intercept term. Thus to estimate the number of days of 24-h recalls that one FFQ is “worth” we equate Var(W) = Var(X|\( \ifmmode\expandafter\bar\else\expandafter\=\fi{R}_{i} \)) = Var(X) − Var(X)2/[Var(X) + Var(ε)/n] and solve for n as:
Since the square of the corrected correlation between FFQ and the 24-h recalls can be written as:
(this follows since X − W is the portion of X that is “explained” by the FFQ and r2 is always the ratio of the variance of the “explained” portion of X to the total variance of X)
we have
which is identical to Eq. 1.
Rights and permissions
About this article
Cite this article
Horn-Ross, P.L., Lee, V.S., Collins, C.N. et al. Dietary assessment in the California Teachers Study: reproducibility and validity. Cancer Causes Control 19, 595–603 (2008). https://doi.org/10.1007/s10552-008-9124-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10552-008-9124-y