Quality of Life Research

, Volume 10, Issue 7, pp 571–578 | Cite as

Smallest real difference, a link between reproducibility and responsiveness

  • H. Beckerman
  • M.E. Roebroeck
  • G.J. Lankhorst
  • J.G. Becher
  • P.D. Bezemer
  • A.L.M. Verbeek


The aim of this study is to show the relationship between test-retest reproducibility and responsiveness and to introduce the smallest real difference (SRD) approach, using the sickness impact profile (SIP) in chronic stroke patients as an example. Forty chronic stroke patients were interviewed twice by the same examiner, with a 1-week interval. All patients were interviewed during the qualification period preceding a randomized clinical trial. Test-retest reproducibility has been quantified by the intraclass correlation coefficient (ICC), the standard error of measurement (SEM) and the related smallest real difference (SRD). Responsiveness was defined as the ratio of the clinically relevant change to the SD of the within-stable-subject test-retest differences. The ICC for the total SIP was 0.92, whereas the ICCs for the specified SIP categories varied from 0.63 for the category ‘recreation and pastime’ to 0.88 for the category ‘work’. However, both the SEM and the SRD far more capture the essence of the reproducibility of a measurement instrument. For instance, a total SIP score of an individual patient of 28.3% (which is taken as an example, being the mean score in the study population) should decrease by at least 9.26% or approximately 13 items, before any improvement beyond reproducibility noise can be detected. The responsiveness to change of a health status measurement instrument is closely related to its test-retest reproducibility. This relationship becomes more evident when the SEM and the SRD are used to quantify reproducibility, than when ICC or other correlation coefficients are used.

Outcome measure Reliability Reproducibility Responsiveness Sickness impact profile Stroke 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hopkins WG. Measures of reliability in sports medicine and science. Sports Med 2000; 30: 1–15.PubMedCrossRefGoogle Scholar
  2. 2.
    Streiner DL, Norman GR. Health Measurement Scales. A Practical Guide to Their Development and Use. Oxford: Oxford University Press, 1993.Google Scholar
  3. 3.
    Guyatt G, Walter S, Norman G. Measuring change over time: Assessing the usefulness of evaluative instruments. J Chron Dis 1987; 40: 171–178.PubMedCrossRefGoogle Scholar
  4. 4.
    Bennekom CAM van, Jelles F, Lankhorst GJ, Bouter LM. Responsiveness of the rehabilitation activities profile and the Barthel index. J Clin Epidemiol 1996; 49: 39–44.PubMedCrossRefGoogle Scholar
  5. 5.
    Moertel CG, Hanley JA. The effect of measuring error on the results of therapeutic trials in advanced cancer. Cancer 1976; 38: 388–394.PubMedCrossRefGoogle Scholar
  6. 6.
    Beckerman H, Becher J, Lankhorst GJ, Verbeek ALM. Walking ability of stroke patients: Efficacy of tibial nerve blocking and a polypropylene ankle-foot orthosis. Arch Phys Med Rehabil 1996; 77: 1144–1151.PubMedCrossRefGoogle Scholar
  7. 7.
    Gilson BS, Gilson JS, Bergner M, et al. The sickness impact profile. Development of an outcome measure of health care. Am J Public Health 1975; 65: 1304–1310.PubMedCrossRefGoogle Scholar
  8. 8.
    Bergner M, Bobbitt RA, Kressel S, Pollard WE, Gilson BS, Morris JR. The sickness impact profile: Conceptual formulation and methodology for the development of a health status measure. Int J Health Serv 1976; 6: 393–415.PubMedGoogle Scholar
  9. 9.
    Bergner M, Bobbitt RA, Pollard WE, Martin DP, Gilson BS. The sickness impact profile: Validation of a health status measure. Med Care 1976; 14: 57–67.PubMedGoogle Scholar
  10. 10.
    McMahon LP, Dawborn JK. Subjective quality of life assessment in hemodialysis patients at different levels of hemoglobin following use of recombinant human erythropoietin. Am J Nephrol 1992; 12: 162–169.PubMedCrossRefGoogle Scholar
  11. 11.
    Hazard Munro B, Creamer AM, Haggerty MR, Cooper FS. Effect of relaxation therapy on post-myocardial infarction patients' rehabilitation. Nurs Res 1988; 37: 231–235.Google Scholar
  12. 12.
    Hyde E. Acupressure therapy for morning sickness. A controlled clinical trial. J Nurse-Midwifery 1989; 34: 171–178.PubMedCrossRefGoogle Scholar
  13. 13.
    Pollard WE, Bobbitt RA, Bergner M, Martin DP, Gilson BS. The sickness impact profile: Reliability of a health status measure. Med Care 1976; 14: 146–155.PubMedGoogle Scholar
  14. 14.
    Bergner M, Bobbitt RA, Carter WB, Gilson BS. The sickness impact profile: Development and final revision of a health status measure. Med Care 1981; 19: 787–805.PubMedGoogle Scholar
  15. 15.
    Bruin AF de, Witte LP de, Stevens F, Diederiks JPM. Sickness impact profile: The state of the art of a generic functional status measure. Soc Sci Med 1992; 35: 1003–1014.PubMedCrossRefGoogle Scholar
  16. 16.
    Deyo RA, Diehl AK. Measuring physical and psychosocial function in patients with low back pain. Spine 1983; 8: 635–642.PubMedGoogle Scholar
  17. 17.
    Deyo RA. Comparative validity of the sickness impact profile and shorter scales for functional assessment in low-back pain. Spine 1986; 11: 951–954.PubMedGoogle Scholar
  18. 18.
    Jensen MP, Strom SE, Turner JA, Romano JM. Validity of the sickness impact profile Roland scale as a measure of dysfunction in chronic pain patients. Pain 1992; 50: 157–162.PubMedCrossRefGoogle Scholar
  19. 19.
    Sullivan M, Ahlmen M, Archenholtz B, Svensson G. Measuring health in rheumatic disorders by means of a Swedish version of the sickness impact profile. Scand J Rheumatol 1986; 15: 193–200.PubMedGoogle Scholar
  20. 20.
    Jacobs HM, Touw-Otten FWMM, Melker RA de. The evaluation of changes in functional health status in patients with abdominal complaints. J Clin Epidemiol 1996; 49: 163–171.PubMedCrossRefGoogle Scholar
  21. 21.
    Damiano AM, Patrick DL, Guzman GI, et al. Measurement of health-related quality of life in patients with amyotrophic lateral sclerosis in clinical trials of new therapies. Med Care 1999; 37: 15–26.PubMedCrossRefGoogle Scholar
  22. 22.
    Visser MC, Koudstaal PJ, Erdmann RA, et al. Measuring quality of life in patients with myocardial infarction or stroke: A feasibility study of four questionnaires in The Netherlands. J Epidemiol Commun Health 1995; 49: 513–517.Google Scholar
  23. 23.
    Witte L de, Jacobs H, Horst F van der, Luttik A, Joosten J, Philipsen H. De waarde van de sickness impact profile als maat voor het functioneren van patiënten. Gezondheid en Samenleving 1987; 8: 120–127.Google Scholar
  24. 24.
    Jacobs HM, Luttik A, Touw-Otten FWMM, Kastein M, De Melker RA. Measuring impact of sickness in patients with nonspecific abdominal complaints in a Dutch family practice setting. Med Care 1992; 30: 244–251.PubMedGoogle Scholar
  25. 25.
    Arendzen JH, Duijn H van, Beckmann MKF, Harlaar J, Vogelaar TW, Prevo AJH. Diagnostic blocks of the tibial nerve in spastic hemiparesis. Effects on clinical, electrophysiological and gait parameters. Scand J Rehab Med 1992; 24: 75–81.Google Scholar
  26. 26.
    Shavelson RJ, Webb NM. Generalizability Theory: A Primer. Newbury Park: Sage Publications, 1991.Google Scholar
  27. 27.
    Carter WB, Bobbitt RA, Bergner M, Gilson BS. Validation of an interval scaling: The sickness impact profile. Health Services Res 1976; 517–528.Google Scholar
  28. 28.
    Roebroeck ME, Harlaar J, Lankhorst GJ. The application of the generalizability theory to reliability assessment: An illustration using isometric force measurement. Phys Ther 1993; 73: 386–401.PubMedGoogle Scholar
  29. 29.
    Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures. Statistics and strategies for evaluation. Controlled Clinic Trials 1991; 12: 142S–158S.Google Scholar
  30. 30.
    Stratford PW. Reliability: Consistency or differentiating among subjects? Phys Ther 1989; 69: 299–300.PubMedGoogle Scholar
  31. 31.
    Guyatt GH, Kirschner B, Jaeschke R. Measuring health status: What are the necessary measurement properties? J Clin Epidemiol 1992; 45: 1341–1345.PubMedCrossRefGoogle Scholar
  32. 32.
    Pfennings LEMA, Ploeg HM van der, Cohen L, Polman CH. A comparison of responsiveness indices in multiple sclerosis patients. Qual Life Res 1999; 8: 481–489.PubMedCrossRefGoogle Scholar
  33. 33.
    Brennan RL. Elements of Generalizability Theory. Iowa city: ACT publications, 1983.Google Scholar
  34. 34.
    Crick JE, Brennan RL. Manual for GENOVA: A Generalized Analysis of Variance System. Iowa city: American College Testing Program, 1983.Google Scholar
  35. 35.
    Kramer MS, Feinstein AR. Clinical biostatistics. LIV. The biostatistics of concordance. Clin Pharmacol Ther 1981; 29: 111–123.PubMedCrossRefGoogle Scholar
  36. 36.
    Bartko JJ. On various intraclass correlation reliability coefficients. Psycholog Bull 1976a; 83: 762–765.CrossRefGoogle Scholar
  37. 37.
    Bartko JJ, Carpenter WT. On the methods and theory of reliability. J Nerv Mental Dis 1976; 163: 307–317.CrossRefGoogle Scholar
  38. 38.
    Rankin G, Stokes M. Reliability of assessment tools in rehabilitation: An illustration of appropriate statistical analyses. Clinic Rehabil 1998; 12: 187–199.CrossRefGoogle Scholar
  39. 39.
    Lachs MS. The more things change.... J Clin Epidemiol 1993; 46: 1091–1092.PubMedCrossRefGoogle Scholar
  40. 40.
    Hébert R, Spiegelhalter DJ, Brayne C. Setting the minimal metrically detectable change on disability rating scales. Arch Phys Med Rehabil 1997; 78: 1305–1308.PubMedCrossRefGoogle Scholar
  41. 41.
    Schuling J, Greidanus J, Meyboom-de Jong B. Measuring functional status of stroke patients with the sickness impact profile. Disabil Rehabil 1993; 15: 19–23.PubMedCrossRefGoogle Scholar
  42. 42.
    Stratford P, Solomon P, Binkley J, Finch E, Gill C. Sensitivity of sickness impact profile items to measure change over time in a low back pain patient group. Spine 1993; 18: 1723–1727.PubMedGoogle Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  • H. Beckerman
    • 1
  • M.E. Roebroeck
    • 2
  • G.J. Lankhorst
    • 1
  • J.G. Becher
    • 1
  • P.D. Bezemer
    • 3
  • A.L.M. Verbeek
    • 4
  1. 1.Department of Rehabilitation MedicineUniversity Hospital Vrije UniversiteitAmsterdamThe Netherlands
  2. 2.Institute of Rehabilitation MedicineErasmus UniversityRotterdamThe Netherlands
  3. 3.Department of Clinical Epidemiology and BiostatisticsVrije UniversiteitAmsterdamThe Netherlands
  4. 4.Department of EpidemiologyUniversity of NijmegenThe Netherlands

Personalised recommendations