Skip to main content
SpringerLink
Log in

Systematic self-report bias in health data: impact on estimating cross-sectional and treatment effects

  • Published:
Health Services and Outcomes Research Methodology Aims and scope Submit manuscript

Abstract

This paper examines the effect of systematic self-report bias, the non-random deviation between the self-reported and true values of the same measure. This bias may be constant or variable, and can mislead empirical analyses based on descriptive statistics, program evaluation and instrumental variables estimation. I illustrate these issues with data on self-reported and measured overweight/obesity status, and BMI, height and weight z-scores of public school students in California from 2004 to 2006. I find that the prevalence of overweight/obesity is 2.4–7.6% points lower in self-reported data relative to measured data in the cross-section. A school nutrition policy changed the bias differentially in the treatment and control groups so that program evaluations could find spurious positive or null impacts of the intervention. Potential channels for this effect include improved information and stigma.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. The prevalence of overweight is measured as the percentage of individuals above a specific cutoff in the distribution of body-mass index (BMI). BMI is calculated as (weight in kg)/(height in m)\(^2\). For children the cutoffs vary by age and gender (Vidmar et al. 2004).

  2. The CHKS is based on the California Student Survey along with items from the Youth Risk Behavioral Survey (YRBS). It is required of districts that accept funds through the federal Title IV Safe and Drug-Free Schools and Communities (SDFSC) or the state Tobacco-Use Prevention Education (TUPE) programs.

  3. Since 2001 the PFT is administered annually between February 1 and May 30 but the actual survey dates are not available in the data. I use April 1 as midpoint in determining the age cutoffs. I also only consider students in “pure” middle (offering at most grades 6–8) and high schools (offering grades 9 and higher). Most Californian schools meet these criteria. Less than 0.5% of the data are biologically implausible by the CDC standards, and the results reported below are similar for the full sample.

  4. The cut-off is recommended by the Childhood Obesity Working Group of the International Obesity Taskforce (Vidmar et al. 2004). For some students weight is reported twice in the PFT, in the aerobic capacity and the body composition test. Where the reported weights disagree I use the weight from the body composition section. From 2002/3 onward the PFT also reports BMI which I use to infer missing values in weight with help of the height measure. When both weight measures are missing I use this calculated weight. If the composite BMI and the body composition weight are both missing I use the weight from the aerobic capacity test as last resort.

  5. Values for California exclude five other districts that also implemented nutrition guidelines (Samuels et al. 2006).

  6. Hausman specification tests reject the random intercept models but results are comparable with the fixed-effects results shown here. Since clustering on the district level increases the standard errors on the coefficients of interest due to negative intracluster correlation, I report the more conservative unclustered errors for all models.

  7. Since I cannot link the self-reported and measured data for individual students, I am unable to see the effects on height and weight for students near the overweight/obesity cutoff.

  8. This information effect may be particularly important for adolescents since their physique is changing rapidly and truthful reporting of their overweight status would require constant updating.

References

  • Bauhoff, S.: The Effect of School Nutrition Policies in California on Dietary Intake and Obesity: A Synthetic Control Approach. Working Paper (2010)

  • Bound, J., Brown, C., Mathiowetz, N.: Measurement error in survey data. In: Heckman, J., Leamer, E. (eds.) Handbook of Econometrics, vol. 5, chap. 59, pp. 3705–3843. Elsevier, Amsterdam (2001)

    Google Scholar 

  • Brener, N.D., Eaton, D.K., Lowry, R., McManus, T.: The association between weight perception and BMI among high school students. Obes. Res. 12, 1866–1874 (2004)

    Article  PubMed  Google Scholar 

  • Brener, N.D., Mcmanus, T., Galuska, D.A., Lowry, R., Wechsler, H.: Reliability and validity of self-reported height and weight among high school students. J. Adolesc. Health 32, 281–287 (2003)

    Article  PubMed  Google Scholar 

  • Cawley, J.: Rational Addiction, the Consumption of Calories, and Body Weight, Ph.D. thesis (1999)

  • Cawley, J., Meyerhoefer, C., Newhouse, D.L.: The impact of state physical education requirements on youth physical activity and overweight. Health Econ. 16, 1287–1301 (2007)

    Article  PubMed  Google Scholar 

  • CDC: YRBS 2007 Data User Manual. Technical Report, Centers for Disease Control and Prevention (2007)

  • Ezzati, M., Martin, H., Skjold, S., Hoorn, S.V., Murray, C.J.L.: Trends in national and state-level obesity in the USA after correction for self-report bias: analysis of health surveys. J. R. Soc. Med. 99, 250–257 (2006)

    Article  PubMed  Google Scholar 

  • Puhl, R.M., Latner, J.D.: Stigma, obesity, and the health of the nation’s children. Psychol. Bull. 133, 557–580 (2007)

    Article  PubMed  Google Scholar 

  • Puhl, R.M., Latner, J.D.: Weight bias: new science on an significant social problem. Obesity 16, S1–S2 (2008)

    Article  PubMed  Google Scholar 

  • Samuels, S.E., Craypo, L., Boyle, M., Stone-Francisco, S., Schwarte, L.: Improving School Food Environments Through District-Level Policies: Findings from Six California Case Studies. Technical Report, Samuels and Associates (2006)

  • StataCorp: Stata Statistical Software: Release 11 (2009)

  • Strauss, R.S.: Childhood obesity and self-esteem. Pediatrics 105, e15 (2000)

    Article  PubMed  CAS  Google Scholar 

  • Vidmar, S., Carlin, J., Hesketh, K., Cole, T.: Standardizing anthropometric measures in children and adolescents with new functions for egen. Stata J. 4, 50–55 (2004)

    Google Scholar 

Download references

Acknowledgements

I am grateful to Alberto Abadie, Eliana Carranza, David Cutler, Caroline Hoxby, Holger Kern, James O’Malley, Thomas McGuire, Manoj Mohanan, Joseph Newhouse, Alan Zaslavsky and an anonymous referee for helpful suggestions; and to Kiku Annon, Jerry Bailey and Julie Williams for assistance with the PFT and CHKS data.

Author information

Authors and Affiliations

  1. Department of Health Care Policy, Harvard University, Boston, MA, 02115, USA

    Sebastian Bauhoff

Authors
  1. Sebastian Bauhoff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sebastian Bauhoff.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bauhoff, S. Systematic self-report bias in health data: impact on estimating cross-sectional and treatment effects. Health Serv Outcomes Res Method 11, 44–53 (2011). https://doi.org/10.1007/s10742-011-0069-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10742-011-0069-3

Keywords

JEL classification

Search

Navigation