Skip to main content
SpringerLink
Log in

Meta-analysis of survey data: application to health services research

  • Published:
Health Services and Outcomes Research Methodology Aims and scope Submit manuscript

Abstract

Traditionally, meta-analysis methods have been developed and used to combine data from several independent clinical trials as well as observational studies, but have not been as widely used in survey research. This paper describes the steps in conducting such a meta-analysis of surveys, to obtain a single summary estimate from a combination of individual-level and summary data. The methods are applied in the context of a project aimed at obtaining an estimate of the prevalence of use of electronic health records.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • AAPOR. Code of Professional Ethics and Practices (2005)

  • American Association for Public Opinion Research.: Best Practices for Survey and Public Opinion Research (1997)

  • Armitage, P.: Inference and decision in clinical trials. J. Clin. Epidemiol. 42, 293–299 (1989)

    Article  PubMed  CAS  Google Scholar 

  • Audet, A.M., Doty, M., Peugh, J., Shamasdin, J., Zapert, K., Schoenbaum, S.: Information technologies: When will they make it into physicians’ black bags? Med. Gen. Med. 6(4), 2 (2004)

    Google Scholar 

  • Berlin, J.A., Rennie, D.: Measuring the quality of trials: the quality of quality scales. JAMA 282(11), 1083–1085 (1999)

    Article  PubMed  CAS  Google Scholar 

  • Blumenthal, D., DesRoches, C., Donelan, K., Ferris, T., Jha, A., Kaushal, R., Rao, S., Rosenbaum S., Shield, A.: Health information technology in the United States: The information base for progress. Robert Wood Foundation (2006)

  • Burt, C.W., Hing, E., Woodwell, D.: Electronic medical record use by office-based physicians: United States, 2005. National Center for Health Statistics, Hyattsville, MD. Available at: http://www.cdc.gov/nchs/products/pubs/pubd/hestats/electronic/electronic.htm. Accessed April 16, 2008

  • Buuren, S.V., Eyres, S., Tennant, A., Hopman-Rock, M.: Improving comparability of existing data by response conversion. JOS 21, 53–72 (2005)

    Google Scholar 

  • Chalmers, T.C.: Problems induced by meta-analyses. Stat. Med. 10, 971–980 (1991)

    Article  PubMed  CAS  Google Scholar 

  • Cochran, W.G.: Sampling Techniques. John Wiley & Sons, New York, NY (1977)

    Google Scholar 

  • Cooper, H., Hedges, L. (eds.): The Handbook of Research Synthesis. Russell Sage Foundation (1994)

  • DerSimonian, R., Laird, N.: Meta-analysis in clinical trials. Control. Clin. Trials 7, 177–188 (1986)

    Article  PubMed  CAS  Google Scholar 

  • Devereaux, P.J., Choi, P.T., Lacchetti, C., Weaver, B., Schünemann, H.J., Haines, T., Lavis, J.N., Grant, B.J., Haslam, D.R., Bhandari, M., Sullivan, T., Cook, D.J., Walter, S.D., Meade, M., Khan, H., Bhatnagar, N., Guyatt, G.H.: A systematic review and meta-analysis of studies comparing mortality rates of private for-profit and private not-for-profit hospitals. CMAJ 166(11), 1399–1406 (2002)

    Google Scholar 

  • Egger, M., Smith, G.D.: Meta-analysis bias in location and selection of studies. BMJ 316, 61–66 (1998)

    PubMed  CAS  Google Scholar 

  • Elliott, M.R., Davis, W.W.: Obtaining cancer risk factor prevalence estimates in small areas: combining data from two surveys. JRSS Appl. Stat. 54(3), 595–609 (2005)

    Google Scholar 

  • Fuller, W.A, Burmeister, L.F.: Estimators for samples selected from two overlapping frames. Proceedings of the Social Statistics Section, American Statistical Association, 245–249 (1972)

  • Gans, D.N., Kralewski, J.E., Hammons, T., Dowd, B.: Medical groups’ adoption of electronic health records and information systems. Health Aff. 24, 1323–1333 (2005)

    Article  Google Scholar 

  • Graham, D.: Analysis of Variance. In: Armitage, P., Colton, T. (eds.) Encyclopedia of Biostatistics, vol. 1, pp. 2574–2576. Wiley, New York (1998)

  • Groves, R.M., Dillman, D.A., Eltinge, J.L., Little, R.J.A. (eds.): Survey Nonresponse. John Wiley & Sons, New York (2002)

    Google Scholar 

  • Hartley, H.O.: Multiple frame surveys. Proceedings of the Social Statistics Section, American Statistical Association, 203–206 (1962)

  • Hartley, H.O.: Multiple frame methodology and selected applications. Sankhya C 36, 99–118 (1974)

    Google Scholar 

  • Higgins, J.P.T., Thompson, S.G.: Quantifying heterogeneity in a meta-analysis. Stat. Med. 21, 1539–1558 (2002)

    Article  PubMed  Google Scholar 

  • Higgins, J.P.T., Thompson, S.G., Deeks, J.J., Altman, D.G.: Measuring inconsistency in meta-analysis. BMJ 327, 557–560 (2003)

    Article  PubMed  Google Scholar 

  • Jha, A.J, Ferris, T.G., Donelan, K., Desroches, C., Shields, A., Rosenbaum, S., Blumenthal, D.: How common are electronic health records in the United States? A summary of the evidence. Health Affairs Web Exclusive, October 11 (2006)

  • Jüni, P., Witschi, A., Bloch, R., Egger M.: The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 282, 1054–1060 (1999)

    Article  PubMed  Google Scholar 

  • Kadane, J.: Some statistical problems in merging data files. Compendium of Tax Research, U.S. Department of the Treasury, 159–171 (1978) (Reprinted in JOS 17(3), 423–433 (2001))

  • Korn, E.L., Graubard, B.I.: Analysis of Health Surveys. John Wiley & Sons, New York, NY (1999)

    Google Scholar 

  • Lambert, P.C., Sutton, A.J., Abrams, K.R., Jones, D.R.: A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. J. Clin. Epidemiol. 55, 86–94 (2002)

    Article  PubMed  CAS  Google Scholar 

  • Lan, G.K.K., Hu, M., Cappelleri, J.C.: Applying the law of iterated logarithm to cumulative meta-analysis of a continuous endpoint. Stat. Sin. 13, 1135–1145 (2003)

    Google Scholar 

  • Lau, J., Antman, E.M., Jimenez-Silva, J., Kupelnick, B., Mosteller, F., Chalmers, T.C.: Cumulative meta-analysis of therapeutic trials for myocardial infarction. NEJM 327, 248–254 (1992)

    PubMed  CAS  Google Scholar 

  • Lau, J., Schmid, C.H., Chalmers, T.C.: Cumulative meta-analysis of clinical trials builds evidence for exemplary medical care. J. Clin. Epidemiol. 48, 45–57 (1995)

    Article  PubMed  CAS  Google Scholar 

  • Lohr, S.L., Rao, J.N.K.: Inference from dual frame surveys. JASA 99, 271–280 (2000)

    Google Scholar 

  • Loomis, G.A., Ries, J.S., Saywell, R.M. Jr., Thakker, N.R.: If electronic medical records are so great, why aren’t family physicians using them? J. Fam. Pract. 51(7), 636–641 (2002)

    PubMed  Google Scholar 

  • Lund, R.E.: Estimators in multiple frame surveys. Proceedings of the Social Statistics Section, American Statistical Association, 282–288 (1968)

  • Moriarty, C., Scheuren, F.: Statistical matching: a paradigm for assessing the uncertainty in the procedure. JOS 17(3), 407–422 (2001)

    Google Scholar 

  • Rodgers, W.L.: An evaluation of statistical matching. J. Bus. Econ. Stat. 2, 91–102 (1984)

    Article  Google Scholar 

  • Rubin, D.B.: Multiple Imputation for Nonresponse in Surveys. John Wiley & Sons, New York (1987)

    Google Scholar 

  • Schenker, N., Gentleman, J.F., Rose, D., Hing, E., Schimizu, I.: Combining estimates from complementary surveys: a case study using prevalence estimates from national health surveys of households and nursing homes. Public Health Rep. 117, 393–407 (2002)

    PubMed  Google Scholar 

  • Schenker, N., Raghunathan, T.E.: Combining information from multiple surveys to enhance estimation of measures of health. Stat. Med. 26(8), 1802–1811 (2007)

    Article  PubMed  Google Scholar 

  • Scheuren, F.: What is a survey? www.whatisasurvey.info. Accessed April 16, 2008

  • Schmid, C.H., Landa, M., Jafar, T.H., Giatras, I., Karim, T., Reddy, M., Stark, P.C., Levey, A.S. for the Angiotensin-Converting Enzyme Inhibition in Progressive Renal Disease (AIPRD) Study Group.: Constructing a database of individual clinical trials for longitudinal analysis. Control. Clin. Trials 24, 324–340 (2003)

    Google Scholar 

  • Schmid, C.H., Stark, P.C., Berlin, J.A., Landais, P., Lau, J.: Meta-regression detected associations between heterogeneous treatment effects and study-level, but not patient-level, factors. J. Clin. Epidemiol. 57, 683–697 (2004)

    Article  PubMed  Google Scholar 

  • Stroup, D.F., Berlin, J.A., Morton, S.C., Olkin, I., Williamson, G.D., Rennie, D., Moher, D., Becker, B.J., Sipe, T.A., Thacker, S.T. for the Meta-analysis of Observational Studies in Epidemiology (MOOSE) Group.: Meta-analysis of observation studies in epidemiology: a proposal for reporting. JAMA 283(15), 2008–2012 (2000)

    Google Scholar 

  • Terry, K.: Exclusive survey: Doctors and EHRs. Medical Economics (2005). Available at: http://www.memag.com/memag/article/articleDetail.jsp?id=143144&pageID=143141&sk=&date=. Accessed April 16, 2008

  • U.S. Dept. of Commerce, Bureau of the Census. Census Quality Survey Public Use Data File, 2001: [United States] [Computer file]. ICPSR release. Washington, DC: U.S. Dept. of Commerce, Bureau of the Census [producer], 2003. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], (2004)

  • Willis, G.B.: Cognitive Interviewing: A Tool for Improving Questionnaire Design. Sage Publications, Thousand Oaks, CA (2005)

    Google Scholar 

  • Zaslavsky, A.M., Shaul, J.A., Zaborski, L.B., Cioffi, M.J., Cleary, P.D.: Combining health plan performance indicators into simpler composite measures. Health Care Financ. Rev. 23(4), 101–115 (2002)

    PubMed  Google Scholar 

Download references

Acknowledgments

The authors thank Dr. David Blumenthal, Dr. Karen Donelan, Dr. Catherine DesRoches, and the HIT Adoption Initiative team for their help. We also thank Catherine W. Burt and Esther S. Hing from NCHS for providing us with the estimates for the group practices using the 2005 NAMCS data. S.R.R. and D.M.F. were supported by a contract from the Office of the National Coordinator for Health Information Technology and the Robert Wood Johnson Foundation. Funding: The project was supported by the Office of the National Coordinator for Health Information Technology, Washington, D.C. and the Robert Wood Johnson Foundation, Princeton, N.J.

Author information

Authors and Affiliations

  1. MGH Biostatistics Center and The Institute for Health Policy, MGH, 50 Staniford Street, Suite 560, Boston, MA, 02114, USA

    Sowmya R. Rao

  2. National Cancer Institute, Bethesda, MD, USA

    Barry I. Graubard

  3. Tufts Sackler School of Graduate Biomedical Sciences, Boston, MA, USA

    Christopher H. Schmid

  4. RTI International, Research Triangle Park, NC, USA

    Sally C. Morton

  5. Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA

    Thomas A. Louis

  6. Department of Health Care Policy, Harvard Medical School, Boston, MA, USA

    Alan M. Zaslavsky

  7. MGH Biostatistics Center, MGH, Boston, MA, USA

    Dianne M. Finkelstein

Authors
  1. Sowmya R. Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Barry I. Graubard
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christopher H. Schmid
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sally C. Morton
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Thomas A. Louis
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alan M. Zaslavsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dianne M. Finkelstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sowmya R. Rao.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rao, S.R., Graubard, B.I., Schmid, C.H. et al. Meta-analysis of survey data: application to health services research. Health Serv Outcomes Res Method 8, 98–114 (2008). https://doi.org/10.1007/s10742-008-0032-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10742-008-0032-0

Keywords

Search

Navigation