Meta-analysis as Judgment Aggregation

Conference paper
Part of the The European Philosophy of Science Association Proceedings book series (EPSP, volume 1)

Abstract

For several decades now, a new inductive method, meta-analysis, is all the rage in social and medical sciences. Meta-analyses, that is, statistical reviews of the results of primary studies concerning a test hypothesis, set new standards of excellence on what counts as strong evidence. In the current prevailing mood in medical and behavioural sciences, it is only a properly conducted, up-to-date meta-analysis that licenses detachment of hypotheses from the host of evidential claims made in individual studies, which claims may be inconclusive or contradictory with each other. My goal in this chapter is to see the extent to which judgment aggregation methods subsume meta-analytic ones. To this end, I derive a generalized version of the classical Condorcet Jury Theorem, and I contend that one can model at least some meta-analytic procedures using this theorem.

Keywords

Rational Agent Primary Study Primary Research Rational Choice Theory Background Assumption 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Cartwright, Nancy. 2007. Hunting causes and using them. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  2. Cooper, Harris, and L.V. Hedges, eds. 1994. The handbook of research synthesis. New York: Russell Sage Foundation.Google Scholar
  3. Dietrich, Franz, and Christian List. 2004. A model of jury decisions where all jurors have the same evidence. Synthese 142(2): 175–202.CrossRefGoogle Scholar
  4. Donoho, David L. 2000. High-dimensional data analysis: The curses and blessings of dimensionality. Lecture delivered at the American Mathematical Society Conference, “Mathematical Challenges of the Twenty-first Century”. http://www-stat.stanford.edu/~donoho/Lectures/AMS2000/AMS2000.html
  5. Duval, S., and R. Tweedie. 2000. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56: 455–463.CrossRefGoogle Scholar
  6. Fiske, Donald W. 1983. The meta-analytic revolution in outcome research. Journal of Consulting and Clinical Psychology 51(1): 65–70.CrossRefGoogle Scholar
  7. Haenni, R., and S. Hartmann. 2006. Modeling partially reliable information sources: A general approach based on Dempster-Shafer theory. Information Fusion 7: 361–379.CrossRefGoogle Scholar
  8. Hammersley, Martyn. 2001. On ‘systematic’ reviews of research literatures: A ‘narrative’ response to Evans&Benefield. British Educational Research Journal 27(5): 543–554.CrossRefGoogle Scholar
  9. Hawthorne, James, unpublished manuscript (circulated beginning 2001). Voting in search of the public good: The probabilistic logic of majority judgments. http://faculty-staff.ou.edu/H/James.A.Hawthorne-1/Hawthorne-{}-Jury-Theorems.pdf
  10. Hedges, Larry V., and Ingram Olkin. 1985. Statistical methods for meta-analysis. San Diego, CA: Academic.Google Scholar
  11. Higgins, J.P.T., and S. Green, eds. 2009. Cochrane handbook for systematic reviews of interventions Version 5.0.2 [updated September 2009]. The Cochrane Collaboration, 2009. http://www.cochrane-handbook.org.
  12. Hunt, Morton. 1997. How science takes stock: The story of meta-analysis. New York: Russell Sage Foundation.Google Scholar
  13. Hunter, J.E., and F.L. Schmidt. 1990. Methods of meta-analysis. Newbury Park: Sage.Google Scholar
  14. Ioannidis, John P.A. 2005. Contradicted and initially stronger effects in highly cited clinical research. JAMA 294(2): 218–228.CrossRefGoogle Scholar
  15. Lewis, D. 1980. A subjectivist’s guide to objective chance. In Studies in inductive logic and probability, Vol. II, ed. Richard C. Jeffrey. Berkeley, CA, and Los Angeles: University of California Press.Google Scholar
  16. Light, Richard J., and David B. Pillemer. 1984. Summing up: The science of reviewing research. Cambridge, MA, and London: Harvard University Press.Google Scholar
  17. Resnick, Sidney I. 1998. A probability path. Basel: Birkhauser Verlag AG.Google Scholar
  18. Rubin, Donald B. 1992. Meta-analysis: Literature synthesis or effect-size surface estimation? Journal of Educational and Behavioral Statistics 17: 363–374.CrossRefGoogle Scholar
  19. Sackett, David L. et al. 2000. Evidence-based medicine. Edinburgh: Churchill Livingstone.Google Scholar
  20. Schmidt, Frank L. 1996. Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods 1(2): 115–129.CrossRefGoogle Scholar
  21. Smith, Mary L., and Gene V. Glass. 1977. Meta-analysis of psychotherapy outcome studies. American Psychologist 32(9): 752–760.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2012

Authors and Affiliations

  1. 1.Philosophy DepartmentBoğaziçi UniversityIstanbulTurkey

Personalised recommendations