Skip to main content
SpringerLink
Log in

Fundamental Issues in Evaluating the Impact of Interventions: Sources and Control of Bias

  • Chapter
  • First Online:
Principles of Research Methodology

Abstract

The ability to draw valid inferences from data is the cornerstone of research and provides the basis for understanding the new knowledge that research results represent.

Internal validity reflects the extent to which a manipulated variable can be shown to account for changes in a dependent variable. It is indispensable for interpreting the experiment.

Ten common threats to internal validity include selection bias, history effects, maturation effects, testing effects, instrumentation effects, statistical regression, experimental mortality, interaction of these factors, experimenter bias, and subject expectancy effects.

Four threats to external validity (generalizability) are reactive effects of testing, interactive effects of selection and treatment, reactive effects of experimental arrangements, and multiple treatment interference. A variety of research designs can be used to evaluate interventions. Each differs in its adequacy for ensuring that valid inferences are made about effects and generalizability.

The poorest for controlling threats to internal validity are termed “pre-experimental designs.” These lack adequate control groups. The strongest are termed “true-experimental designs.” They incorporate control groups to which subjects have been randomly allocated but may suffer from lack of generalizability. Quasi-experimental designs represent a good compromise when randomization is not possible.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Campbell DT, Stanley JC. Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin Company; 1963.

    Google Scholar 

  2. Cook TD, Campbell DT. Quasi-experimentation: design and analysis for field settings. Chicago: Rand McNally; 1979.

    Google Scholar 

  3. Kim SYH, Holloway RG, Frank S, Beck CA, Zimmerman C, Wilson MA, Kieburtz K. Volunteering for early phase gene transfer research in Parkinson disease. Neurology. 2006;66:1010–5.

    Article  PubMed  CAS  Google Scholar 

  4. Woodward SH, Stegman WK, Pavao JR, Arsenault NJ, Hartl TL, Drescher KD, Weaver C. Self-selection bias in sleep and psychophysiological studies of posttraumatic stress disorder. J Trauma Stress. 2007;20: 619–23.

    Article  PubMed  Google Scholar 

  5. Lanfear DE, Jones PG, Cresci S, Tang F, Rathore SS, Spertus JA. Factors influencing patient willingness to participate in genetic research after a myocardial infarction. Genome Med. 2011;3:39.

    Article  PubMed  Google Scholar 

  6. McCuen RH. The elements of academic research. New York: ASCE Publications; 1996.

    Google Scholar 

  7. Rosenthal R. The effect of the experimenter on the results of psychological research. In: Maher BA, editor. Progress in experimental personality research. New York: Academic; 1964.

    Google Scholar 

  8. Mayo E. The human problems of an industrial civilization. New York: Macmillan; 1933.

    Google Scholar 

  9. Saretsky G. The OEO P.C. experiment and the John Henry effect. Phi Delta Kappan. 1972;53:579–81.

    Google Scholar 

  10. Visser RF. Angiographic assessment of patency and reocclusion: preliminary results of the dutch APSAC reocclusion multicenter study (ARMS). Clin Cardiol. 1990;13:45–7.

    Article  Google Scholar 

  11. Wender P, Reimherr F. Buproprion treatment of attention deficit hyperactivity disorders in adults. Am J Psychol. 1990;147:1018–20.

    CAS  Google Scholar 

  12. Bolland J, Ward J, Bolland T. Improved accuracy of estimating food quantities up to 4 weeks after treatment. J Am Diet Assoc. 1990;90:1402–7.

    PubMed  CAS  Google Scholar 

  13. Fisher RA. The arrangement of field experiments. J Min Agric. 1926;33:503–13.

    Google Scholar 

  14. Medical Research Council. Streptomycin treatment of pulmonary tuberculosis. BMJ. 1948;2:769–82.

    Article  Google Scholar 

  15. Berry DA. Adaptive designs: the promise and the caution. J Clin Oncol. 2011;29:606–9.

    Article  PubMed  Google Scholar 

  16. Hu F, Rosenberger WF. The theory of response adaptation in clinical trials. Hoboken: Wiley; 2006.

    Book  Google Scholar 

  17. Heritier SR, Gebski VJ, Keech AC. Inclusion of patients in clinical trial analysis: the intention-to-treat principle. MJA. 2003;179:438–40.

    PubMed  Google Scholar 

  18. Montori VM, Guyatt GH. Intention-to-treat principle. CMAJ. 2001;165:1339–41.

    PubMed  CAS  Google Scholar 

  19. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ. 1999;319:670–4.

    Article  PubMed  CAS  Google Scholar 

  20. Sackett DL, Gent M. Controversy in counting and attributing events in clinical trials. N Engl J Med. 1979;301:1410–2.

    Article  PubMed  CAS  Google Scholar 

  21. Lewis JA, Machin D. Intention to treat—who should use ITT? Br J Cancer. 1993;68:647–50.

    Article  PubMed  CAS  Google Scholar 

  22. Gorbach SL, Morrill-LaBrode A, Woods MN, Dwyer JT, Selles WD, Henderson M, Insull Jr W, Goldman S, Thompson D, Clifford C. Changes in food patterns during a low-fat dietary intervention in women. J Am Diet Assoc. 1990;90:802–9.

    PubMed  CAS  Google Scholar 

  23. B-Blocker Heart Attack Research Group. A randomized trial of propranolol in patients with acute myocardial infarction. JAMA. 1982;247:1707–14.

    Article  Google Scholar 

  24. The International Study Group. In-hospital mortality and clinical course of 20,891 patients with suspected acute myocardial infarction randomized between alteplase and streptokinase with or without heparin. Lancet. 1990;336:71–4.

    Article  Google Scholar 

  25. Stampfer MJ, Buring JE, Willett W, Rosner B, Eberlein K, Hennekens CH. The 2 × 2 factorial design: its application to a randomized trial of aspirin and carotene in U.S. physicians. Stat Med. 1985;4:111–6.

    Article  PubMed  CAS  Google Scholar 

  26. Seabra-Gomes R, Aleixo AM, Adao M, Machado FP, Mendes M, Bruges G, Palos JL. Comparison of the effects of a controlled-release formulation of isosorbide-5-mononitrate and conventional isosorbide dinitrate on exercise performance in men with stable angina pectoris. Am J Cardiol. 1990;65:1308–12.

    Article  PubMed  CAS  Google Scholar 

  27. Kazdin AE. Single case research designs. New York: Oxford University Press; 1982.

    Google Scholar 

  28. Janosky JE, Leininger SL, Hoerger MP, Libkuman TM. Single subject designs in biomedicine. New York: Springer; 2009.

    Book  Google Scholar 

  29. Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin Company; 2002.

    Google Scholar 

  30. Steyn K, Rossouw JE, Jooste PL, Chalton DO, Jordaan ER, Jordaan PC, Steyn M, Swanepoel AS. The intervention effects of a community-based hypertension programme in two rural South African towns: the CORIS Study. S Afr Med J. 1993;83:885–91.

    PubMed  CAS  Google Scholar 

  31. Delate T, Mager DE, Sheth J, Motheral BR. Clinical and financial outcomes associated with a proton pump inhibitor prior-authorization program in a Medicaid population. Am J Manag Care. 2005;11:29–36.

    PubMed  Google Scholar 

  32. Reding GR, Raphelson M. Around–the-clock mobile psychiatric crisis intervention: another effective alternative to psychiatric hospitalization. Commun Ment Health J. 1995;31:179–87.

    Article  CAS  Google Scholar 

  33. Haukoos JS, Hopkins E, Byyny RL, Conroy AA, Silverman M, Eisert S, Thrun M, Wilson M, Boyer B, Heffelfinger JD, Denver ED HIV Opt-Out Study Group. Design and implementation of a controlled clinical trial to evaluate the effectiveness and efficiency of routine opt-out rapid human immunodeficiency virus screening in the emergency department. Acad Emerg Med. 2009;16:800–8.

    Article  PubMed  Google Scholar 

  34. Holder HD, Gruenewald PJ, Ponicki WR, Treno AJ, Grube JW, Saltz RF, Voas RB, Reynolds R, Davis J, Sanchez L, Gaumont G, Roeper PR. Effect of community-based interventions on high risk drinking and alcohol-related injuries. JAMA. 2000;284:2341–7.

    Article  PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Medicine, College of Medicine, SUNY Downstate Medical Center, 450 Clarkson Avenue, 1199, Brooklyn, NY, 11203, USA

    Phyllis G. Supino EdD

Authors
  1. Phyllis G. Supino EdD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Phyllis G. Supino EdD .

Editor information

Editors and Affiliations

  1. , Cardiovascular Medicine, SUNY Downstate Medical Center, Clarkson Avenue, box 1199 450, Brooklyn, 11203, USA

    Phyllis G. Supino

  2. , Cardiovascualr Medicine, SUNY Downstate Medical Center, Clarkson Avenue 450, Brooklyn, 11203, USA

    Jeffrey S. Borer

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Supino, P.G. (2012). Fundamental Issues in Evaluating the Impact of Interventions: Sources and Control of Bias. In: Supino, P., Borer, J. (eds) Principles of Research Methodology. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-3360-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-3360-6_5

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-3359-0

  • Online ISBN: 978-1-4614-3360-6

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation