Abstract
Aims
A study was designed to assess the effects of a standardized instruction videotape on reducing interobserver variability for several commonly used observer-dependent outcome measures.
Methods
During a single day, six rheumatologists independently examined six patients with ankylosing spondylitis (AS) in a predetermined order using a Latin square design, before and after viewing a standardized videotape demonstrating 14 examination techniques. Reliability coefficients were calculated based on the variance components of the analysis of variance (ANOVA) table.
Results
Prestandardization reliability coefficients were <0.80 for three measures. Following standardization 12 reliability coefficients exceeded 0.80. For the majority of measures prestandardization reliability coefficients were high and no further improvement in reliability could be demonstrated. For one measure of cervical extension, but not another, an important and beneficial effect in reliability was noted. It was not possible to achieve adequate reliability in the performance of the chest excursion measurement.
Conclusions
It is usually assumed that serial measurement in clinical trials should be performed by the same assessor because of concern regarding interobserver variability. However, the high levels of prestandardization interobserver reliability observed in this study indicate that for these variables serial measurements in a clinical trial could be made by different assessors, assuming they were equally skilled. This observation has important implications for outcome measurement in AS clinical trials. Although high levels of prestandardization reliability precluded the demonstration of any significant effect, we speculate that the videotape might be effective in training less experienced assessors. Nevertheless, an alternative approach to standardization may be required for the chest excursion measurement. Reductions in observer variability have the potential to diminish sample size requirements for AS antirheumatic drug studies. The use of a videotape to achieve this goal offers cost and convenience advantages over one-on-one training procedures, and this method should be further assessed in a group of less experienced assessors.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bellamy N, Anastassiades TP, Buchanan WW et al. Rheumatoid arthritis antirheumatic drug trials. I. Effects of standardization procedures on observer dependent outcome measures. J Rheumatol. 1991;18:1893–1900.
Bellamy N, Carette S, Ford PM et al. Osteoarthritis antirheumatic drug trials. I. Effects of standardization procedures on observer dependent outcome measures. J Rheumatol. 1992;19:436–43.
Bellamy N, Buchanan WW, Esdaile JM et al. Ankylosing spondylitis antirheumatic drug trials. I. Effects of standardization procedures on observer dependent outcome measures. J Rheumatol. 1991;18:1701–8.
Bellamy N, Bell MJ, Carette S et al. Estimation of observer reliability and sample size calculation parameters for outcome measures in fibromyalgia clinical trials. Inflammopharmacology. 1994;2:345–60.
Laurent MR, Buchanan WW, Bellamy N. Methods, of assessment used in ankylosing spondylitis clinical trials: a review. Br J Rheumatol. 1991;30:326–9.
Ball GV. Ankylosing spondylitis. In: McCarty DJ, ed. Arthritis and Allied Conditions—A Textbook of Rheumatology 11th ed. Philadelphia: Lea & Febiger, 1989:934–43.
Hochberg MC, Chang RW, Dwosh I, Lindsey S, Pincus T, Wolfe F. The American College of Rheumatology 1991 revised criteria for the classification of global functional status in rheumatoid arthritis. Arthritis Rheum. 1992;35:498–502.
US Department of Health and Human Serices Guidelines for the Clinical Evaluation of Anti-Inflammatory and Anti-Rheumatic Drugs (Adults and Children). Public Health Service, Food and Drug Administration, April 1988, p. 38–40.
Guidelines for the Clinical Investigation of Drugs Used In Rheumatic Diseases, European Drug Guidelines Series 5, World Health Organization, Regional Office for Europe, Copenhagen, European League Against Rheumatism, March 1985, p. 12.
Dictionary of the Rheumatic Diseases, Vol. 1: Signs and Symptoms. ARA Glossary Committee, American Rheumatism Association; New York: Contact Press, 1982.
Box GEP, Hunter WG, Hunter JS. Statistics for Experimenters. New York: John Wiley, 1978.
SAS Users Guide. Basics and Statistics Version 5 edition. Cary: SAS Institute Inc., 1985.
Felson DT, Anderson JJ, Boers M et al. The American, College of Rheumatology preliminary core set of disease activity measures for rheumatoid arthritis clinical trials. Arthritis Rheum. 1993;36:729–40.
Altman R, Brandt K, Hochberg M et al. Design and conduct of clinical trials in patients with osteoarthritis: Recommendations from a task force of the Osteoarthritis Research Society. Osteoarthritis Cart. 1996;4:217–43.
Van der Heijde D, Bellamy N, Colin A, Dougados M, Khan A, van der Linden S and the ASAS Working Group. Preliminary core sets for efficacy endpoints in ankylosing spondylitis. J Rheumatol. 1997 [in press].
Bellamy N, Muirden KD, Brooks P, Barroclough D, Tellus MM, Campbell J. A survey of outcome measurement procedures used in routine rheumatology outpatient practice by Australian rheumatologists. Scientific Program of the Australia Rheumatology Association 40th Annual Scientific Conference, Brisbane. May 25–28, 1997, p. 160.
Bellamy N, Kaloni S, Pope J, Coulter K, Campbell J. A survey of outcome measurement procedures used in routine rheumatology outpatient practice by, Canadian rheumatologists. Canadian Rheumatology Association Annual Meeting, Mont-Tremblant, Quebec. February 27–March 1. J Rheumatol. 1997;24:1425.
Bellamy N. Musculoskeletal Clinical Metrology. Dordrecht: Kluwer Academic Publishers; 1993:345–7, 355–6.
Klinkoff AV, Bellamy N, Bombardier C et al. An experiment in reducing interobserver variability of the examination for joint tenderness. J Rheumatol. 1988;15:492–4.
Bellamy N, Muirden KD, Bendrups A et al. Effect of a standardized videotape on observer dependent outcome measures in rheumatoid arthritis. Scientific Program of the Australian Rheumatology Association 40th Annual Scientific Conference, Brisbane. May 25–28, 1997, p. 161.
Bellamy N, Bachmeier, C, Brooks P et al. Effect of a standardized videotape on observer-dependent outcome measures in osteoarthritis. Scientific Program of the Australian Rheumatology Association 40th Annual Scientific Conference, Brisbane. May 25–28, 1997, p. 162.
Bellamy N, Buchbinder R, Hall S et al. Effect of a standardized videotape on observer dependent outcome measures in fibromyalgia. Scientific Program of the Australian Rheumatology Association 40th Annual Scientific Conference, Brisbane. May 25–28, 1997, p. 163.
Pope JE, Baron M, Bellamy N et al. Variability of skin scores and clinical measurements in scleroderma. J Rheumatol. 1995;22:1271–6.
Green S, Buchbinder R, Forbes A, Bellamy N. A standardized protocol for measurement of range of movement of the shoulder using the plurimeter-V inclinometer and assessment of its intrarater and interrater reliability. Arthritis Care Res. 1997 (Submitted).
Bellamy N. Musculoskeletal Clinical Metrology. Dordrecht: Kluwer Academic Publishers; 1993:11–24.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bellamy, N., Muirden, K.D., Boyden, K. et al. Ankylosing spondylitis antirheumatic drug trials: Effects of a standardized instructional viddeotape on the reliability of observer-dependent outcome measures. Inflammopharmacol 5, 297–308 (1997). https://doi.org/10.1007/s10787-997-0007-8
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s10787-997-0007-8