Journal of General Internal Medicine

, Volume 17, Issue 6, pp 469–476

Determination of the clinical importance of study results

A review
  • Malcolm Man-Son-Hing
  • Andreas Laupacis
  • Keith O’Rourke
  • Frank J. Molnar
  • Jeffery Mahon
  • Karen B. Y. Chan
  • George Wells
Review

Abstract

Formal statistical methods for analyzing clinical trial data are widely accepted by the medical community. Unfortunately, the interpretation and reporting of trial results from the perspective of clinical importance has not received similar emphasis. This imbalance promotes the historical tendency to consider clinical trial results that are statistically significant as also clinically important, and conversely, those with statistically insignificant results as being clinically unimportant. In this paper, we review the present state of knowledge in the determination of the clinical importance of study results. This work also provides a simple, systematic method for determining the clinical importance of study results. It uses the relationship between the point estimate of the treatment effect (with its associated confidence interval) and the estimate of the smallest treatment effect that would lead to a change in a patient’s management. The possible benefits of this approach include enabling clinicians to more easily interpret the results of clinical trials from a clinical perspective, and promoting a more rational approach to the design of prospective clinical trials.

Key Words

clinical importance clinical significance study results minimal clinically important difference review 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Jaeschke R, Singer J, Guyatt GH. Measurement of health status: ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10:407–15.PubMedCrossRefGoogle Scholar
  2. 2.
    Atrial Fibrillation Investigators. Risk factors for stroke and efficacy of antithrombotic therapy in atrial fibrillation. Arch Intern Med. 1994;154:1449–57.CrossRefGoogle Scholar
  3. 3.
    EAFT (European Atrial Fibrillation Study) Group. Secondary prevention in non-rheumatic atrial fibrillation after transient ischemic attack or minor stroke. Lancet. 1993;342:1255–62.Google Scholar
  4. 4.
    Albers GW, Dalen JE, Laupacis A, Manning WJ, Petersen P, Singer DE. Antithrombotic therapy in atrial fibrillation. Chest. 2001;119: 194S-206S.PubMedCrossRefGoogle Scholar
  5. 5.
    Gage BF, Waterman AD, Shannon W, Boechler M, Rich M, Radford MJ. Validation of clinical classification schemes for predicting stroke. JAMA. 2001;285:2864–70.PubMedCrossRefGoogle Scholar
  6. 6.
    Hart RG, Benavente O, McBride R, Pearce LA. Antithrombotic therapy to prevent stroke in patients with atrial fibrillation: a meta-analysis. Ann Intern Med. 1999;131:492–501.PubMedGoogle Scholar
  7. 7.
    Atrial Fibrillation Investigators. The efficacy of aspirin in patients with atrial fibrillation. Arch Intern Med. 1997;157:1237–40.CrossRefGoogle Scholar
  8. 8.
    Segal JB, McNamara RL, Miller MR, et al. Prevention of thromboembolism in atrial fibrillation. A meta-analysis of trials of anticoagulants and antiplatelet drugs. J Gen Intern Med. 2000; 15:56–67.PubMedCrossRefGoogle Scholar
  9. 9.
    Young MJ, Bresnitz EA, Strom BL. Sample size nomograms for interpreting negative clinical studies. Ann Intern Med. 1983;99:248–51.PubMedGoogle Scholar
  10. 10.
    Goodman SN, Berlin JA. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med. 1994;121:200–6.PubMedGoogle Scholar
  11. 11.
    Feinstein AR. Clinical Biostatistics. Saint Louis: CV Mosby Company; 1977:333.Google Scholar
  12. 12.
    Naylor CD, Llewellyn-Thomas HA. Can there be a more patient-centred approach to determining clinically important effect sizes for randomized treatment trials. J Clin Epidemiol. 1994;47:787–95.PubMedCrossRefGoogle Scholar
  13. 13.
    Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med. 1980;302:1109–17.PubMedCrossRefGoogle Scholar
  14. 14.
    Bellamy N, Carrette S, Ford PM, et al. Osteoarthritis antirheumatic drug trials III. Setting the delta for clinical trials—results of a consensus development (Delphi) exercise. J Rheumatol. 1992;20:557–60.Google Scholar
  15. 15.
    Gorelick PB, Born GV, D’Agostino RB, Hanley DF Jr, Moye L, Pepine CJ. Therapeutic benefit. Aspirin revisited in light of the introduction of clopidogrel. Stroke. 1999;30:1716–21.PubMedGoogle Scholar
  16. 16.
    Albers GW, Amarenco P, Easton JD, Sacco RL, Teal P. Antithrombotic and thrombolytic therapy for ischemic stroke. Chest. 2001; 119:300S-20S.PubMedCrossRefGoogle Scholar
  17. 17.
    Llewellyn-Thomas HA, Thiel EC, Clark RM. Patients versus surrogates: whose opinion counts on ethics review panels? Clin Res. 1989;37:501–5.PubMedGoogle Scholar
  18. 18.
    Llewellyn-Thomas HA, McGreal MJ, Thiel EC, Fine S, Erlichman C. Patients’ willingness to enter clinical trials: measuring the association with perceived benefit and preference for decision participation. Soc Sci Med. 1991;32:35–42.PubMedCrossRefGoogle Scholar
  19. 19.
    Man-Son-Hing M, Laupacis A, O’Connor A, et al. Warfarin for atrial fibrillation: the patient’s perspective. Arch Intern Med. 1996;156:1841–8.PubMedCrossRefGoogle Scholar
  20. 20.
    Detsky AS. Using economic analysis to determine the resource consequences of choices made in planning clinical trials. J Chronic Dis. 1985;38:753–65.PubMedCrossRefGoogle Scholar
  21. 21.
    van Walraven C, Mahon JL, Moher D, Bohn C, Laupacis A. Surveying physicians to determine the minimal important difference: implications for sample-size calculations. J Clin Epidemiol. 1999;52:717–23.PubMedCrossRefGoogle Scholar
  22. 22.
    Chan K, Man-Son-Hing M, Molnar FJ, Laupacis A. How well is the clinical importance of study results reported. An assessment of randomized controlled trials. CMAJ. 2001;165:1197–202.PubMedGoogle Scholar
  23. 23.
    Kosinski M, Zhao SZ, Dedhiya S, Osterhaus JT, Ware JE Jr. Determining minimally important changes in generic and disease-specific health-related quality of life questionnaires in clinical trials of rheumatoid arthritis. Arthritis Rheum. 2000;43:1478–87.PubMedCrossRefGoogle Scholar
  24. 24.
    Wells G, Beaton D, Shea B, et al. Minimal clinically important differences: review of methods. J Rheumatol. 2001;28:406–12.PubMedGoogle Scholar
  25. 25.
    Redelmeier DA, Guyatt GH, Goldstein RS. Assessing the minimal important difference in symptoms: a comparison of two techniques. J Clin Epidemiol. 1996;49:1215–9.PubMedCrossRefGoogle Scholar
  26. 26.
    Redelmeier DA, Lorig JP. Assessing the clinical importance of statistical significance: illustration in rheumatology. Arch Intern Med. 1993;153:1337–42.PubMedCrossRefGoogle Scholar
  27. 27.
    Wells GA, Tugwell P, Kraag GR, Baker P, Groh J, Redelmeier D. Minimum important difference between patients with rheumatoid arthritis: the patient’s perspective. J Rheumatol. 1993;20:557–60.PubMedGoogle Scholar
  28. 28.
    Juniper EF, Guyatt GH, Willan A, Griffith LE. Determining a minimal important change in a disease-specific quality of life questionnaire. J Clin Epidemiol. 1994;47:81–7.PubMedCrossRefGoogle Scholar
  29. 29.
    Cohen J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988:19–27.Google Scholar
  30. 30.
    Jacobson NS, Roberts LJ, Berns SB, McGlinchey JB. Methods for defining and determining the clinical significance of treatment effects: description, application, and alternatives. J Consul Clin Psychol. 1999;67:300–307.CrossRefGoogle Scholar
  31. 31.
    Kendall PC, Marrs-Garcia A, Nath SR, Sheldrick RC. Normative comparisons for the evaluation of clinical significance. J Consult Clin Psychol. 1999;67:285–99.PubMedCrossRefGoogle Scholar
  32. 32.
    Anonymous. Significance of significance. N Engl J Med. 1968;278:1232–3.Google Scholar
  33. 33.
    Melton AW. Editorial. J Exp Psychol. 1962;64:553–7.CrossRefGoogle Scholar
  34. 34.
    Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology. A Basic Science for Clinical Medicine, 2nd ed. Boston: Little, Brown and Company; 1991.Google Scholar
  35. 35.
    Freiman J, Chalmers TC, Smith H, Kaubler R. The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. N Engl J Med. 1978;299:690–4.PubMedCrossRefGoogle Scholar
  36. 36.
    Cook DJ, Guyatt GL, Laupacis A, Sackett DL, Goldberg RJ. Clinical recommendations using levels of evidence for antithrombotic agents. Chest. 1995;108:227S-30S.PubMedGoogle Scholar
  37. 37.
    Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ. Users’ guide to the medical literature IX. A method for grading health care recommendations. JAMA. 1995;274:1800–4.PubMedCrossRefGoogle Scholar
  38. 38.
    Moher D, Schulz KF, Altman DG, for the CONSORT Group. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Intern Med. 2001;134:657–62.PubMedGoogle Scholar
  39. 39.
    Mahon J, Laupacis A, Donner A, Wood T. Randomised study of n of 1 trials versus standard practice. BMJ. 1996;312:1069–74.PubMedGoogle Scholar
  40. 40.
    Detsky AS, Sackett DL. When was a ‘negative’ clinical trial big enough. Arch Intern Med. 1985;145:709–12.PubMedCrossRefGoogle Scholar
  41. 41.
    Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: importance of rigorous methods. BMJ. 1996;313:36–9.PubMedGoogle Scholar
  42. 42.
    Dunnett CW, Gent M. Significance testing to establish equivalence between treatments, with special reference to data in the form of 2 × 2 tables. Biometrics. 1977;33:593–602.PubMedCrossRefGoogle Scholar
  43. 43.
    Pitt B, Zannad F, Remme WJ, et al. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. N Engl J Med. 1999;341:709–17.PubMedCrossRefGoogle Scholar
  44. 44.
    Naylor MD, Broten D, Campbell R, et al. Comprehensive discharge planning and home follow-up of hospitalized elders. JAMA. 1999; 281:613–20.PubMedCrossRefGoogle Scholar
  45. 45.
    Dalby DM, Sellors JW, Fraser FD, Fraser C, van Ineveld C, Howard M. Effect of preventive home visits by a nurse on the outcomes of frail elderly people in the community: a randomized controlled trial. CMAJ. 2000;62:497–500.Google Scholar
  46. 46.
    Silverstein FE, Graham DY, Senior JR, et al. Misoprostol reduces serous gastrointestinal complications in patients with rheumatoid arthritis receiving nonsteroidal anti-inflammatory drugs. Ann Intern Med. 1995;123:241–9.PubMedGoogle Scholar
  47. 47.
    Dale JJ, Ruckley CV, Harper DR, Gibson B, Nelson EA, Prescott RJ. Randomised, double blind placebo controlled trial of pentoxifylline in the treatment of venous leg ulcers. BMJ. 1999;319:875–8.PubMedGoogle Scholar
  48. 48.
    GISSI-Prevenzione Investigators. Dietary supplementation with n-3 polyunsaturated fatty acids and vitamin E after myocardial infarction: results of the GISSI-Prevenzione trial. Lancet. 1999; 354:447–55.CrossRefGoogle Scholar
  49. 49.
    Hulley S, Grady D, Bush T, et al. Randomized trial of estrogen for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/progestin Replacement Study (HERS) Research Group. JAMA. 1998;280:605–13.PubMedCrossRefGoogle Scholar
  50. 50.
    Pocock SJ. When to stop a clinical trial. BMJ. 1992;305:235–40.PubMedCrossRefGoogle Scholar
  51. 51.
    Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992;327:248–54.PubMedCrossRefGoogle Scholar
  52. 52.
    Guyatt GH, Sinclair J, Cook DJ, Glasziou P. Users’ guide to the medical literature XVI. How to use a treatment recommendation. JAMA. 1999;281:1836–43.PubMedCrossRefGoogle Scholar

Copyright information

© Society of General Internal Medicine 2002

Authors and Affiliations

  • Malcolm Man-Son-Hing
    • 1
    • 2
    • 3
  • Andreas Laupacis
    • 5
  • Keith O’Rourke
    • 1
  • Frank J. Molnar
    • 1
    • 2
    • 3
  • Jeffery Mahon
    • 6
  • Karen B. Y. Chan
    • 1
  • George Wells
    • 1
    • 4
  1. 1.Received from the Clinical Epidemiology Unit, Ottawa Health Research InstituteOttawa HospitalCanada
  2. 2.the Institute on Health of the Elderly, Sisters of CharityOttawa Health ServiceCanada
  3. 3.the Division of Geriatric MedicineUniversity of OttawaOttawa
  4. 4.the Department of Community Medicine and Epidemiology (GW)University of OttawaOttawa
  5. 5.the Institute for Clinical Evaluative Sciences and Faculty of MedicineUniversity of TorontoToronto
  6. 6.the University of Western OntarioLondonCanada
  7. 7.Ottawa Hospital-Civic CampusGeriatric Assessment UnitOttawaCanada

Personalised recommendations