Skip to main content
SpringerLink
Log in

Reporting missing data: a study of selected articles published from 2003–2007

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

Missing data (MD) is prevalent in empirical social science research (Allison, http://books.google.ca/books?id=ZtYArHXjpB8Canddq=missing+dataandprintsec=frontcoverandsource=blandots=ziTBvDHxWbandsig=Uiy83xDbWSO4HeBSth0Q_oZIrQandhl=enandei=q6A7Su25ApnIM-eYlccOandsa=Xandoi=book_resultandct=resultandresnum, 2001; Rubin in J Am Stat Assoc 91:473–489, 1996). Furthermore, inconsistencies in the reporting and treatment of MD have been documented (Peugh and Enders in Rev Educ Res 74:525–556, 2004). The goal of the current project was to examine how MD was reported and treated in 68 studies issued from a refereed educational journal. It was observed that a fifth of the quantitative articles reviewed actually treated MD, using mainly deletion methods and only 10% of the explicit articles explained MD. Overall, MD reported or inferred averaged about 33% of cases and was commonly associated with missing subjects. Only three articles provided the actual percentages of MD. The results suggest that there may be a lack of consistency in how MD is reported and treated and that studies may remain deficient in how MD is addressed. The importance of the study was to signal tendencies in reporting and treating MD, to provide tentative explanations for such trends, to propose hypotheses for future empirical studies in the field, and to offer an initial set of practical guidelines for including addressing, explaining, and treating MD in educational research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abraham W.T., Russell D.W.: Missing data: a review of current methods and applications in epidemiological research. Curr. Opin. Psychiatry 17, 315–321 (2004)

    Article  Google Scholar 

  • Acock A.C.: Working with missing values. Marriage Fam. 67, 1012–1028 (2005)

    Article  Google Scholar 

  • Allison P.D.: Multiple imputation for missing data. Sociol. Methods Res. 28, 301–309 (2000)

    Article  Google Scholar 

  • Allison, P.D.: Missing data: quantitative applications in the social sciences. http://books.google.ca/books?id=ZtYArHXjpB8Canddq=missing+dataandprintsec=frontcoverandsource=blandots=ziTBvDHxWbandsig=Uiy83xDbWSO4HeBSth0Q_oZIrQandhl=enandei=q6A7Su25ApnIM-eYlccOandsa=Xandoi=book_resultandct=resultandresnum (2001). Accessed 22 June 2008

  • Braund M.: Bridging work and its role in improving progression and continuity: an example from science education. Br. Educ. Res. J. 33, 905–926 (2007)

    Article  Google Scholar 

  • Brown C.H.: Asymptotic comparison of missing data procedures for estimating factor loadings. Psychometika 48, 129–145 (1983)

    Article  Google Scholar 

  • Brown M., Askew M., Millet A., Rhodes V.: The key role of educational research in the development and evaluation of the national numeracy strategy. Br. Educ. Res. J. 29, 655–667 (2003)

    Article  Google Scholar 

  • Carpenter, J., Kenward, M.: Missing value jargon. http://www.lshtm.ac.uk/msu/missingdata/jargon_web/jargon_web.htm (2005). Accessed 8 Sep 2008

  • Collins L.M., Schafer J.L., Kam C.: MA comparison of inclusive and restrictive strategies in modern missing data procedures. Psychol. Methods 6, 330–351 (2001)

    Article  Google Scholar 

  • Connolly P.: The effects of social class and ethnicity on gender differences in GCSE attainment: a secondary analysis of the youth cohort study of England and Wales 1997–2001. Br. Educ. Res. J. 32, 3–21 (2006)

    Article  Google Scholar 

  • Croninger R.G., Douglas K.M.: Missing data and institutional research. In: Umbach, P.D. (ed.) Emerging Issues: New Directions for Institutional Research # 127, Jossey-Bass, San Francisco (2005)

    Google Scholar 

  • De Fraine B., Van Damme J., Van Landeghem G., Opdenakker C.M., Onghena P.: The effect of schools and classes on language achievement. Br. Educ. Res. J. 29, 841–859 (2003)

    Article  Google Scholar 

  • Dockrell J.E., Shield B.M.: Acoustical barriers in classrooms: the impact of noise on performance in the classroom. Br. Educ. Res. J. 32, 509–525 (2006)

    Article  Google Scholar 

  • Donders A.R.T., van der Heijden G.J.M.G., Stijnen T., Moons K.G.M.: Review: a gentle introduction to imputation of missing values. J. Clin. Epidemiol. 59, 1087–1091 (2006)

    Article  Google Scholar 

  • Ducharme, M.K., Licklider, B.L., Mathes, W.A., Vannatta, R.A.: Conceptual and analysis criteria: A process for identifying quality educational research. Des Moines, IA: FINE-First in the Nation in Education, Iowa’s Educational Research Foundation (1995)

  • Forsyth A., Furlong A.: Access to higher education and disadvantaged young people. Br. Educ. Res. J. 29, 205–225 (2003)

    Article  Google Scholar 

  • Goddard R., O’Brien P., Goddard M.: Work environment predictors of beginning teacher burnout. Br. Educ. Res. J. 32, 857–874 (2006)

    Article  Google Scholar 

  • Hallam S., Ireson J.: Safety in stereotypes? The impact of gender and ‘race’ on young people’s perceptions of their post-compulsory education and labor market opportunities. Br. Educ. Res. J. 32, 583–599 (2006)

    Article  Google Scholar 

  • Hurd S., Mangan J., Adnet N.: Are secondary schools spending enough on books?. Br. Educ. Res. J. 31, 239–255 (2005)

    Article  Google Scholar 

  • Ireson J., Hallam S., Hurley C.: What are the effects of ability grouping on GCSE attainment?. Br. Educ. Res. J. 31, 443–458 (2005)

    Article  Google Scholar 

  • Kaufman C.J.: The application of logical imputation to household measurement. J. Mark. Res. Soc. 30, 453–466 (1988)

    Google Scholar 

  • Kidger J.: Measuring effectiveness in school sex education-methodological dilemmas in researching an intervention involving young mothers?. Br. Educ. Res. J. 32, 291–305 (2006)

    Article  Google Scholar 

  • Lerkkanen M.K., Rasku-putonen H., Aunola K., Nurmi J.E.: Predicting reading performance during the first and the second year of primary school?. Br. Educ. Res. J. 30, 67–92 (2004)

    Article  Google Scholar 

  • Lindsay G., Lewis A.: An evaluation of the use of accredited baseline assessment schemes in England?. Br. Educ. Res. J. 29, 149–167 (2003)

    Article  Google Scholar 

  • Ludbrook J.: Outlying observations and missing values: How should they be handled?. Clin. Exp. Pharmacol. Physiol. 35, 670–678 (2008)

    Article  Google Scholar 

  • Marseden E.: Can educational experiments both test a theory and inform practice?. Br. Educ. Res. J. 33, 545–588 (2007)

    Google Scholar 

  • McKnight P.E., McKnight K.M., Sidani S., Figuerdo A.J.: Missing Data: A Gentle Introduction. Guildford Press, New York (2007)

    Google Scholar 

  • Mercer N., Dawes L., Wegerif R., Sams C.: Reasoning as a scientist: ways of helping children to use language to learn science. Br. Educ. Res. J. 30, 359–377 (2004)

    Article  Google Scholar 

  • Opdenakker M.C., Van Damme J.: Do school context, student composition and school leadership affect school practice and outcomes in secondary education?. Br. Educ. Res. J. 33, 179–206 (2007)

    Article  Google Scholar 

  • Peugh J.L., Enders C.K.: Missing data in educational research: a review of reporting practices and suggestions for improvement. Rev. Educ. Res. 74, 525–556 (2004)

    Article  Google Scholar 

  • Poirier D.J., Rund P.A.: Diagnostic testing in missing data models. Int. Econ. Rev. 24, 537–546 (1983)

    Article  Google Scholar 

  • Pustjens H., Van de Gaer E., Van Damme J., Onghena P., Van Landeghem G.: The short-term and the long-term effect of primary schools and classes in mathematics and language achievement scores. Br. Educ. Res. J. 33, 419–440 (2007)

    Article  Google Scholar 

  • Rees G., Baon S., Boyask R., Taylor C.: Research-capacity building, professional learning and the social practices of educational research. Br. Educ. Res. J. 33, 681–701 (2007)

    Article  Google Scholar 

  • Richardson J.T.E.: Students’ perceptions of academic quality and approaches to studying in distance education. Br. Educ. Res. J. 31, 7–27 (2005)

    Article  Google Scholar 

  • Riddell S., Tinklin T., Wilson A.: New labor, social justice and disabled students in higher education. Br. Educ. Res. J. 31, 623–643 (2005)

    Article  Google Scholar 

  • Roth P.L.: Missing data: a conceptual review for applied psychologists. Pers. Psychol. 47, 537–560 (1994)

    Article  Google Scholar 

  • Rousseau, M.: L’impact des méthodes de traitement des valeurs manquantes sur les qualités psychométriques d’échelles de mesure de type Likert. Unpublished Doctoral Thesis, Université Laval, Faculté des sciences de l’éducation (2006)

  • Rubin D.B.: Inference and missing data. Biometrika 63, 581–592 (1976)

    Article  Google Scholar 

  • Rubin D.B.: Multiple imputation for 18+ years. J. Am. Stat. Assoc. 91, 473–489 (1996)

    Google Scholar 

  • SAS.: USA SAS Inc., Cary (2008)

  • SPSS for Windows, Rel. 11.0.1. SPSS Inc., Chicago (2001)

  • Strand S.: Comparing predictive validity of reasoning tests and national end of key stage 2 tests: Which tests are the ‘best’?. Br. Educ. Res. J. 32, 209–225 (2006)

    Article  Google Scholar 

  • Tinklin T.: Gender differences and high attainment. Br. Educ. Res. J. 29, 307–325 (2003)

    Article  Google Scholar 

  • Van De Werfhorst H.G., Sullivan A., Cheng S.Y.: Social class, ability and choice of subject in secondary and tertiary education in Britain. Br. Educ. Res. J. 29, 41–62 (2003)

    Article  Google Scholar 

  • Van der Kamp L.J.T., Bijleveld C.C.J.H.: Methodological issues in longitudinal research. In: Bijleveld, C.C.J.H., Van der Kamp, L.J.T. (eds) Longitudinal Data Analysis, pp. 1–45. Sage Publication Ltd., London (1998)

    Google Scholar 

  • Wilkinson, L., Task Force on Statistical Inference.: Statistical methods in Psychology Journals. Am. Psychol. 54, 594–604 (1999)

  • William D., Bartholomew H.: It’s not which school but which se you’re in that matters: the influence of ability grouping practices on student progress in mathematics. Br. Educ. Res. J. 30, 279–293 (2004)

    Article  Google Scholar 

  • Wisniewski S.R., Leon A.C., Otto M.W., Trivedi M.H.: Prevention of missing data in clinical research studies. Biol. Psychiatry 59, 997–1000 (2006)

    Article  Google Scholar 

  • Wood A.M., White I.R., Thompson S.G.: Are missing outcome data adequately handled? A review of published randomized controlled trials in major medical journals. Clin. Trials 1, 368–376 (2004)

    Article  Google Scholar 

  • Yuan K.H., Bentler P.M.: Three likelihood-based methods for mean and covariance structure analysis with non-normal missing data. In: Becker, M., Sobel, M. (eds) Sociological Methodology 2000, Blackwell, Maiden (2000)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universite Laval, Laval, QC, Canada

    Michel Rousseau, Marielle Simon, Richard Bertrand & Krystal Hachey

Authors
  1. Michel Rousseau
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marielle Simon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Richard Bertrand
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Krystal Hachey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michel Rousseau.

Additional information

Dr. Jeffrey D. Kromrey acted as a reviewer for the NCME conference 2009.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rousseau, M., Simon, M., Bertrand, R. et al. Reporting missing data: a study of selected articles published from 2003–2007. Qual Quant 46, 1393–1406 (2012). https://doi.org/10.1007/s11135-011-9452-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-011-9452-y

Keywords

Search

Navigation