Skip to main content
SpringerLink
Log in

negative results in social science

  • Research Article
  • Published:
European Political Science Aims and scope Submit manuscript

Abstract

Do academic publication standards reflect or determine research results? The article proposes minimal criteria for distinguishing useful ‘unpublishable’ results from low-quality research, and argues that the virtues of negative results have been overlooked. We consider the fate these results have suffered thus far, review arguments for and against their publication and introduce a new initiative – a journal to disseminate negative results and advance debate on their recognition and use.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. We draw particular inspiration from the typologisation of negative results in computer science in Prechelt (1997).

  2. Debates on grounded theory, data mining and exploratory data analysis (Begg and Berlin, 1988; Simon, 1988) must be reserved for a future article.

  3. We are indebted to such pioneering efforts in medical, computer and other sciences such as the Journal of Negative Results in Ecology & Evolutionary Biology (www.jnr-eeb.org), the Journal of Negative Results in Biomedicine (www.jnrbm.com) and the Forum for Negative Results of the Journal of Universal Computer Science (www.jucs.org).

  4. A summary of the survey results is available on the website www.jspurc.org and from the authors upon request.

  5. www.jspurc.org.

References

  • Adcock, R. and Collier, D. (2001) ‘Measurement validity: a shared standard for qualitative and quantitative research’, American Political Science Review 95: 529–546.

    Article  Google Scholar 

  • Baert, P. (2005) ‘Towards a pragmatist-inspired philosophy of social science’, Acta Sociologica 48 (3): 191–203.

    Article  Google Scholar 

  • Bartels, L.M. (1997) ‘Specification uncertainty and model averaging’, American Journal of Political Science 41: 641–674.

    Article  Google Scholar 

  • Beck, N. and Katz, J.N. (1995) ‘What to do (and not to do) with time-series cross-section data’, American Political Science Review 89: 634–647.

    Article  Google Scholar 

  • Beck, N. and Katz, J.N. (1996) ‘Nuisance vs. substance: specifying and estimating time-series cross-section models’, Political Analysis 6: 1–34.

    Article  Google Scholar 

  • Begg, C.B. and Berlin, J.A. (1988) ‘Publication bias: a problem in interpreting medical data’, Journal of the Royal Statistical Society 151 (Series A): 419–463.

    Article  Google Scholar 

  • Begley, S. (2006) ‘New journals bet ‘negative results’ save time, money’’, Wall Street Journal, 15 September.

  • Boruch, R. (2006) ‘The null hypothesis is not called that for nothing: statistical tests in randomized experiments’, Journal of Experimental Criminology, forthcoming.

  • Breslow, J.M. (2006) ‘A glance at The Journal of Spurious Correlations: a new journal’, The Chronicle of Higher Education, 2 October, http://chronicle.com/temp/reprint.php?id=bjtfvwlb77y9hz1fldwjlzr3y3sqzh8b.

  • Bryant, F.B. and Wortman, P.M. (1984) ‘Methodological Issues in the Meta-Analysis of Quasi-Experiments’, in W.H. Yeaton and P. M. Wortman (eds.) Issues in Data Synthesis: New Directions for Program Evaluation, San Francisco: Jossey-Bass, pp. 5–24.

    Google Scholar 

  • Cartwright, N. (2004) ‘Well-ordered science: evidence for use’, Paper presented at the Biennial Meeting of the Philosophy of Science Association, 18–20 November, Austin, USA.

  • Cartwright, N. (2006) ‘Causal claims: warranting them and using them’, Paper presented at the Universidad del Pais Vasco Ninth Summer School, July 10–16, Madrid.

  • Fölster, S. (1995) ‘The perils of peer review in economics and other sciences’, Journal of Evolutionary Economics 5: 43–57.

    Article  Google Scholar 

  • Gans, J.S. and Shepherd, G.B. (1994) ‘How are the mighty fallen: rejected classic articles by leading economists’, Journal of Economic Perspectives 8 (1): 165–179.

    Article  Google Scholar 

  • Gelman, A. and Stern, H. (2005) ‘The difference between ‘significant’ and ‘not significant’ is not itself statistically significant, http://www.stat.columbia.edu/~gelman/research/unpublished/signifrev.pdf.

  • Gerber, A.S., Green, D.P. and Nickerson, D. (2001) ‘Testing for publication bias in political science’, Political Analysis 9 (4): 385–392.

    Article  Google Scholar 

  • Gerber, A.S. and Malhotra, N. (2006) ‘Can political science literature be believed? a study of publication bias in the APSR and AJPS’, Paper presented at the Annual Meeting of the Midwest Political Science Association, 12–15 April, Chicago.

  • Granger, C.W.J. and Jeon, Y. (2004) ‘Thick modeling’, Economic Modeling 21: 323–343.

    Article  Google Scholar 

  • King, G. and Zeng, L. (2006) ‘The dangers of extreme counterfactuals’, Political Analysis 14 (2): 131–159.

    Article  Google Scholar 

  • Kittel, B. and Winner, H. (2005) ‘How reliable is pooled analysis in political economy? The globalization-welfare state nexus revisited’, European Journal of Political Research 44: 269–293.

    Article  Google Scholar 

  • Kotze, J., Johnson, C., O’Hara, R., Vepsäläinen, K. and Fowler, M. (2004) ‘Editorial: the journal of negative results in ecology and evolutionary biology’, Journal of Negative Results - Ecology & Evolutionary Biology 1: 1–5.

    Google Scholar 

  • Laumann, E., Gagnon, J.H., Michael, R.T. and Michaels, S. (1994) The Social Organization of Sexuality in the United States, Chicago: University of Chicago Press.

    Google Scholar 

  • MacCoun, R.J. (1998) ‘Biases in the interpretation and use of research results’, Annual Review of Psychology 49: 259–287.

    Article  Google Scholar 

  • Mahoney, M.J. (1977) ‘Publication prejudices: an experimental study of confirmatory bias in the peer review system’, Cognitive Therapy and Research 1: 161–175.

    Article  Google Scholar 

  • Prechelt, L. (1997) ‘Why we need an explicit forum for negative results’, Journal of Universal Computer Science 3 (9): 1074–1083.

    Google Scholar 

  • Presser, S. and Stinson, L. (1998) ‘Data collection mode and social desirability bias in self-reported religious attendance’, American Sociological Review 63: 137–145.

    Article  Google Scholar 

  • Scargle, J.D. (2000) ‘Publication bias: the ‘file-drawer’ problem in scientific inference’, Journal of Scientific Exploration 14 (1): 91–106.

    Google Scholar 

  • Scheines, R. (2002) ‘Computation and causation’, MetaPhilosophy 33 (1): 158–180.

    Article  Google Scholar 

  • Sigelman, L. (1999) ‘Publication bias reconsidered’, Political Analysis 8: 201–210.

    Article  Google Scholar 

  • Simon, R. (1988) ‘Comment on Begg and Berlin’, Journal of the Royal Statistical Society 151 (Series A): 419–463.

    Google Scholar 

  • Skloot, R. (2006) ‘Publication probity’, New York Times Magazine, 10 December, http://www.nytimes.com/2006/12/10/magazine, 10 December, 4. html?ex=1323406800&en=b2948892b7417c1b&ei=5090&partner=rssuerland&emc=rss.

  • Thaler, R.H. (1987–2001) ‘Anomalies’, series in The Journal of Economic Perspectives.

  • Tourrangeau, R. and Smith, T.W. (1996) ‘Asking sensitive questions: the impact of data collection mode, question format, and question context’, The Public Opinion Quarterly 60: 275–304.

    Article  Google Scholar 

  • Travis, G.D.L. and Collins, H.M. (1991) ‘New light on old boys: cognitive and institutional particularism in the peer review system’, Science, Technology, and Human Values 16: 322–341.

    Article  Google Scholar 

  • Wilson, S.E. and Butler, D.M. (2007) ‘A lot more to do: the promise and peril of panel data in political science’, Political Analysis, forthcoming.

  • Zamora Bonilla, J . (2002) ‘Scientific inference and the pursuit of fame: a contractarian approach’, Philosophy of Science 69: 300–323.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Political Science, University of Helsinki, P.O. Box 54 (Unioninkatu 37), FIN-00014, Finland

    David Lehrer

  2. Research Department, European Trade Union Institute, Blv. du Roi Albert II, 5 box 4, Bruxelles, B-1210, Belgium

    Janine Leschke

  3. Max Planck Institute for Demographic Research, Konrad-Zuse-Strasse 1, Rostock, D-18057, Germany

    Stefan Lhachimi

  4. National Centre for Sustainable Development, Dr. Burghelea 36, Bucharest, RO-73102, Romania

    Ana Vasiliu

  5. Department of Political Science, University of Tübingen, Melanchthonstr 36, Tübingen, D-72074, Germany

    Brigitte Weiffen

Authors
  1. David Lehrer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Janine Leschke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefan Lhachimi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ana Vasiliu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Brigitte Weiffen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brigitte Weiffen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lehrer, D., Leschke, J., Lhachimi, S. et al. negative results in social science. Eur Polit Sci 6, 51–68 (2007). https://doi.org/10.1057/palgrave.eps.2210114

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/palgrave.eps.2210114

Keywords

Search

Navigation