Abstract
The bivariate model has become a de facto standard in diagnostic meta-analysis. Complex iterative algorithms are needed to fit the model, and thus a meta-analysis of diagnostic accuracy data is much aided by appropriate software packages. Also, graphical methods ease exploration, interpretation, and communication in the context of a diagnostic meta-analysis. This chapter reviews existing software and discusses the relative merits of general packages and specialized packages for DTA meta-analysis. The use of software for diagnostic meta-analysis and especially fitting the bivariate model is illustrated with a sample workflow in the open-source statistical framework R. Some ways to extend the bivariate model and software for the case of multiple cutoff values per primary study are discussed.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
We mention in passing that RevMan can plot SROC curves if suitable output is supplied from other packages.
- 2.
Example code for other packages can typically be found in the references in Table 12.1 or in the technical documentation.
- 3.
The package can be installed by typing install.packages(“mada”) at an R-prompt. After this, the package only needs to load once in an R session with library(mada). The most current (development) version of mada is found at http://r-forge.r-project.org/projects/mada/. Some additional functionality of mada is explained in the package vignette that is automatically installed with the package and can be accessed by typing vignette(“mada”) at an R-prompt.
- 4.
Note that the subset function is a convenient way in R to form subsets.
- 5.
Note that for special cases like a binary covariate, plotting SROC curves for the parameters corresponding to each of both levels of the covariates is meaningful. For an example, see Meyer, Frings, Rücker, and Hellwig [68].
- 6.
Note that brms’s syntax is very similar to lme4’s so that the sample code below can be adapted. For similar lme4 code, also consult Partlett and Takwoingi [24].
- 7.
CAMAN is also the backbone for the implementation of the semiparametric mixture approach of Schlattmann, Verba, Dewey, and Walther [69], which extends the bivariate model.
- 8.
At the time of writing, code is found on V. Dukic’s homepage: http://amath.colorado.edu/faculty/vdukic/software/ROC.html
References
The Nordic Cochrane Centre. Review manager (RevMan) [Computer Program], version 5.3. The Cochrane Collaboration, Copenhagen; 2014.
Schwarzer G. meta: an R package for meta-analysis. R News. 2007;7:40–5.
Glas A, Lijmer J, Prins M, Bonsel G, Bossuyt P. The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol. 2003;56:1129–35.
Reitsma J, Glas A, Rutjes A, Scholten R, Bossuyt P, Zwinderman A. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58:982–90.
Leeflang M, Deeks J, Gatsonis C, Bossuyt P. Systematic reviews of diagnostic test accuracy. Ann Intern Med. 2008;149:889–97.
Macaskill P, Gatsonis C, Deeks J, Harbord R, Takwoingi Y. Chapter 10: analysing and presenting results. In: Deeks J, Bossuyt P, Gatsonis C, editors. Cochrane handbook for systematic reviews of diagnostic test accuracy version 1.0. The Cochrane Collaboration; 2010. Available from http://srdta.cochrane.org/. Accessed 29 June 2018.
Rutter C, Gatsonis C. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. 2001;20:2865–84.
Harbord R, Deeks J, Egger M, Whiting P, Sterne J. A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2007;8:239–51.
DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177–88.
Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36:1–48. Available from http://www.jstatsoft.org/v36/i03/. Accessed 29 June 2018.
Gasparrini A, Armstrong B, Kenward M. Multivariate meta-analysis for non-linear and other multi-parameter associations. Stat Med. 2012;31:3821–39.
R Core Team. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2017. Available from https://www.R-project.org/. Accessed 29 June 2018.
Schwarzer G, Carpenter J, Rücker G. Meta-analysis with r. New York: Springer; 2015. UseR!
Liu Z, Yao Z, Li C, Liu X, Chen H, Gao C. A step-by-step guide to the systematic review and meta-analysis of diagnostic and prognostic test accuracy evaluations. Br J Cancer. 2013;108:2299–303.
Kim K, Lee J, Choi S, Huh J, Park S. Systematic review and meta-analysis of studies evaluating diagnostic test accuracy: a practical review for clinical researchers-part I. General guidance and tips. Korean J Radiol. 2015;16:1175–87.
Lee J, Kim K, Choi S, Huh J, Park S. Systematic review and meta-analysis of studies evaluating diagnostic test accuracy: a practical review for clinical researchers-part II. Statistical methods of meta-analysis. Korean J Radiol. 2015;16:1188–96.
Nikoloulopoulos A. CopulaREMADA: copula mixed effect models for bivariate and trivariate meta-analysis of diagnostic test accuracy studies. R package version 1.0. 2016. Available from https://CRAN.R-project.org/package=CopulaREMADA. Accessed 29 June 2018.
Schiller I, Dendukuri N. HSROC: joint meta-analysis of diagnostic test sensitivity and specificity with or without a gold standard reference test. R package version 2.1.8; 2015.
Verde PE. bamdit: bayesian meta-analysis of diagnostic test data. R package version 3.1.0. 2017. Available from https://CRAN.R-project.org/package=bamdit. Accessed 29 June 2018.
Lunn D, Thomas A, Best N, Spiegelhalter D. WinBUGS—a Bayesian modelling framework: concepts, structure, and extensibility. Stat Comput. 2000;10:325–37.
Lunn D, Spiegelhalter D, Thomas A, Best N. The BUGS project: evolution, critique and future directions. Stat Med. 2009;28:3049–67.
Plummer M. rjags: Bayesian graphical models using MCMC. R package version 4-6. 2016. Retrieved from https://CRAN.R-project.org/package=rjags. Accessed 29 June 2018.
Bürkner P-C. brms: an R package for Bayesian multilevel models using Stan. J Stat Softw. 2017;80:1–28.
Partlett C, Takwoingi Y. Meta-analysis of test accuracy studies in R: a summary of user-written programs and step-by-step guide to using glmer. Version 1.0. 2016. Available from http://methods.cochrane.org/sdt/. Accessed 29 June 2018.
Doebler P. mada: meta-analysis of diagnostic accuracy. R package version 0.5.7. 2015. Available from https://CRAN.R-project.org/package=mada. Accessed 29 June 2018.
Guo J, Riebler A. meta4diag: meta-analysis for diagnostic test studies. R package version 2.0.5. 2016. Available from https://CRAN.R-project.org/package=meta4diag. Accessed 29 June 2018.
Huang H. Metatron: meta-analysis for classification data and correction to imperfect reference. R package version 0.1-1. 2014. Available from https://CRAN.R-project.org/package=Metatron. Accessed 29 June 2018.
Botella J, Huang H, Suero M. Multinomial tree models for assessing the status of the reference in studies of the accuracy of tools for binary classification. Front Psychol. 2013;4:694.
Charlton C, Rasbash J, Browne W, Healy M, Cameron, B. Mlwin [computer program] version 3.00. Centre for Multilevel Modelling, University of Bristol; 2017.
Chu H, Cole S. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol. 2006;59:1331–2.
Arends L, Hamza T, Van Houwelingen J, Heijenbrok-Kal M, Hunink M, Stijnen T. Bivariate random effects meta-analysis of ROC curves. Med Decis Mak. 2008;28:621–38.
Menke J. Bivariate random-effects meta-analysis of sensitivity and specificity with sas proc glimmix. Methods Inf Med. 2010;49:54–64.
Takwoingi Y, Deeks J. METADAS: an SAS macro for meta-analysis of diagnostic accuracy studies, version 1.3. Computer program; 2011.
Rabe-Hesketh S, Skrondal A, Pickles A. Generalized multilevel structural equation modeling. Psychometrika. 2004;69:167–90.
Harbord R, Whiting P. Metandi: meta-analysis of diagnostic accuracy using hierarchical logistic regression. Stata J. 2010;9:211–29.
Takwoingi Y. Meta-analysis of test accuracy studies in Stata: a bivariate model approach. Version 1.1. 2016. Available from http://methods.cochrane.org/sdt/. Accessed 29 June 2018.
Dwamena B. midas: Stata module for meta-analytical integration of diagnostic accuracy studies. 2007. Available from: http://econpapers.repec.org/software/bocbocode/s456880.htm. Accessed 29 June 2018.
Patrick D, Cheadle A, Thompson D, Diehr P, Koepsell T, Kinne S. The validity of self-reported smoking: a review and meta-analysis. Am J Public Health. 1994;84:1086–93.
Wickham H, Bryan J. readxl: read excel files. R package version 1.0.0. 2017. Available from https://CRAN.R-project.org/package=readxl. Accessed 29 June 2018.
Phillips B, Stewart L, Sutton A. ‘cross hairs’ plots for diagnostic meta-analysis. Res Synth Methods. 2010;1:308–15.
Vogelgesang F, Schlattmann P, Dewey M. The evaluation of bivariate mixed models in meta-analyses of diagnostic accuracy studies with SAS, Stata and R. Methods Inf Med. 2018;57:111–9.
Doebler P, Holling H, Böhning D. A mixed model approach to meta-analysis of diagnostic studies with binary test outcome. Psychol Methods. 2012;17:418–36.
Karrasch S, Linde K, Rücker G, Sommer H, Karsch-Völk M, Kleijnen J, et al. Accuracy of FENO for diagnosing asthma: a systematic review. Thorax. 2017;72:109–16.
Brown H, Prescott R. Applied mixed models in medicine. 3rd ed. Hoboken: John Wiley & Sons; 2015.
Demidenko E. Mixed models: theory and applications. Hoboken: John Wiley & Sons; 2013.
Chu H, Nie L, Cole S, Poole C. Meta-analysis of diagnostic accuracy studies accounting for disease prevalence: alternative parameterizations and model selection. Stat Med. 2009;28:2384–99.
Hoyer A, Kuss O. Meta-analysis of diagnostic tests accounting for disease prevalence: a new model using trivariate copulas. Stat Med. 2015;34:1912–24.
Nikoloulopoulos A. A vine copula mixed effect model for trivariate meta-analysis of diagnostic test accuracy studies accounting for disease prevalence. Stat Methods Med Res. 2017;26:2270–86.
Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed-effects models using lme4. J Stat Softw. 2015;67:1–48.
Moses L, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med. 1993;12:1293–316.
Holling H, Böhning W, Böhning D. Meta-analysis of diagnostic studies based upon SROC-curves: a mixed model approach using the Lehmann family. Stat Model. 2012;12:347–75.
Holling H, Böhning W, Böhning D. Likelihood-based clustering of meta-analytic SROC curves. Psychometrika. 2012;77:106–26.
Schlattmann P, Höhne J, Verba M. CAMAN: finite mixture models and meta-analysis tools–based on C.A.MAN. R package version 0.74. 2016. Available from https://CRAN.R-project.org/package=CAMAN. Accessed 29 June 2018.
Doebler P, Holling H. Meta-analysis of diagnostic accuracy and ROC curves with covariate adjusted semiparametric mixtures. Psychometrika. 2015;80:1084–104.
Charoensawat S, Böhning W, Böhning D, Holling H. Meta-analysis and meta-modelling for diagnostic problems. BMC Med Res Methodol. 2014;14:56.
Rücker G, Schumacher M. Summary ROC curve based on a weighted Youden index for selecting an optimal cut point in meta-analysis of diagnostic accuracy. Stat Med. 2010;29:3069–78.
Steinhauser S, Schumacher M, Rücker G. Modelling multiple thresholds in meta-analysis of diagnostic test accuracy studies. BMC Med Res Methodol. 2016;16:97.
Levis B, Benedetti A, Levis A, Ioannidis J, Shrier I, Cuijpers P, et al. Selective cutoff reporting in studies of diagnostic test accuracy: a comparison of conventional and individual-patient-data meta-analyses of the Patient Health Questionnaire-9 depression screening tool. Am J Epidemiol. 2017;185:954–64.
Dukic V, Gatsonis C. Meta-analysis of diagnostic test accuracy assessment studies with varying number of thresholds. Biometrics. 2003;59:936–46.
Hamza T, Arends L, van Houwelingen H, Stijnen T. Multivariate random effects meta-analysis of diagnostic tests with multiple thresholds. BMC Med Res Methodol. 2009;9:73.
Putter H, Fiocco M, Stijnen T. Meta-analysis of diagnostic test accuracy studies with multiple thresholds using survival methods. Biom J. 2010;52:95–110.
Simoneau G, Levis B, Cuijpers P, Ioannidis J, Patten S, Shrier I, et al. A comparison of bivariate, multivariate random-effects, and Poisson correlated gamma-frailty models to meta-analyze individual patient data of ordinal scale diagnostic tests. Biom J. 2017;59:1317.
Riley R, Ahmed I, Ensor J, Takwoingi Y, Kirkham A, Morris R, et al. Meta-analysis of test accuracy studies: an exploratory method for investigating the impact of missing thresholds. Syst Rev. 2015;4:12.
Hoyer A, Hirt S, Kuss O. Meta-analysis of full ROC curves using bivariate time-to-event models for interval-censored data. Res Synth Methods. 2018;9:62–72.
Martínez-Camblor P. Fully non-parametric receiver operating characteristic curve estimation for random-effects meta-analysis. Stat Methods Med Res. 2017;26:5–20.
Riley R, Takwoingi Y, Trikalinos T, Guha A, Biswas A, Ensor J, et al. Meta-analysis of test accuracy studies with multiple and missing thresholds: a multivariate-normal model. J Biomet Biostat. 2014;5:196.
Deeks J, Wisniewski S, Davenport C. Chapter 4: guide to the contents of a Cochrane Diagnostic Test Accuracy Protocol. In: Deeks J, Bossuyt P, Gatsonis C, editors Cochrane handbook for systematic reviews of diagnostic test accuracy version 1.0.0. 2013. Available from http://srdta.cochrane.org/. Accessed 29 June 2018.
Meyer P, Frings L, Rücker G, Hellwig S. 18F-FDG PET in parkinsonism: differential diagnosis and cognitive impairment in Parkinson’s disease. J Nucl Med. 2017;58:1888.
Schlattmann P, Verba M, Dewey M, Walther M. Mixture models in diagnostic meta-analyses—clustering summary receiver operating characteristic curves accounted for heterogeneity and correlation. J Clin Epidemiol. 2015;68:61–72.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Doebler, P., Bürkner, PC., Rücker, G. (2018). Statistical Packages for Diagnostic Meta-Analysis and Their Application. In: Biondi-Zoccai, G. (eds) Diagnostic Meta-Analysis. Springer, Cham. https://doi.org/10.1007/978-3-319-78966-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-78966-8_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-78965-1
Online ISBN: 978-3-319-78966-8
eBook Packages: MedicineMedicine (R0)