Abstract
Aims
A study was designed to assess the effects of a standardized instructional videotape on reducing interobserver variability for several commonly used observer-dependent outcome measures.
Methods
During a single day, six rheumatologists independently examined six patients with rheumatoid arthritis (RA) in a predetermined order using a Latin square design, before and after viewing a standardized videotape demonstrating 13 examination techniques. Reliability coefficients were calculated based on the variance components of the analysis of variance (ANOVA) table.
Results
Prestandardization reliability coefficients were >0.80 for all measures and remained above 0.80 following standardization.
Conclusions
It is usually assumed that serial measurement in clinical trials should be performed by the same assessor because of concern regarding interobserver variability. However, the high levels of prestandardization interobserver reliability observed in this study indicate that, for these variables, serial measurements in a clinical trial could be made by different assessors, assuming they were equally skilled. This observation has important implications for outcome measurement in RA clinical trials. Although high levels of prestandardization reliability precluded the demonstration of any significant effect, we speculate that the videotape might be effective in training less-experienced assessors. Reductions in observer variability have the potential to diminish sample size requirements for RA antirheumatic drug studies. The use of a videotape to achieve this goal offers cost and convenience advantages over one-on-one training procedures, and this method should be further assessed in less-experienced assessors.
Similar content being viewed by others
References
Bellamy N, Anastassiades TP, Buchanan WW et al. Rheumatoid arthritis antirheumatic drug trials. I. Effects of standardization procedures on observer dependent outcome measures. J Rheumatol. 1991;18:1893–1900.
Bellamy N, Carette S, Ford PM et al. Osteoarthritis antirheumatic drug trials. I. Effects of standardization procedures on observer dependent outcome measures. J Rheumatol. 1992;19:436–43.
Bellamy N, Buchanan WW, Esdaile JM et al. Ankylosing spondylitis antirheumatic drug trials. I. Effects of standardization procedures on observer dependent outcome measures. J Rheumatol. 1991;18:1701–8.
Bellamy N, Bell MJ, Carette S et al. Estimation of observer reliability, and sample size calculation parameters for outcome measures in fibromyalgia clinical trials. Inflammopharmacology. 1994;2:345–60.
Gøtzsche PC. Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis. Controlled Clin Trials. 1989;10:31–56.
Arnett FC, Edworthy SM, Bloch DA et al. The American Rheumatism Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheum. 1988;31:315–24.
Hochberg MC, Chang RW, Dwosh I, Lindsey S, Pincus T, Wolfe F The American College of Rheumatology 1991 revised criteria for the classification of global functional status in rheumatoid arthritis. Arthritis Rheum 1992;35:498–502.
US Department of Health and Human Services Guidelines for the Clinical Evaluation of Anti-Inflammatory and Anti-Rheumatic Drugs (Adults and Children). Public Health Service, Food and Drug Administration, April 1988 p.2–11, 29–35.
World Health Organization Guidelines for the Clinical Investigation of Drugs Used in Rheumatic Diseases, European Drug Guideline Series 5, World Health Organization, Regional Office for Europe, Copenhagen, European League Against Rheumatism, March 1985, pp. 7–11.
Dictionary of the Rheumatic Diseases, Vol. 1: Signs and Symptoms. ARA Glossary Committee, American Rheumatism Association. New York: Contact Press; 1982.
Box GEP, Hunter WG, Hunter JS. Statistics for Experimenters. New York: John Wiley; 1978.
SAS Users Guide. Basics and Statistics, Version 5 edition. Cary: SAS Institute Inc.; 1985.
Felson DT, Anderson JJ, Boers M, Bombardier C, Chernoff M, Fried B et al. The American College of Rheumatology preliminary core set of disease activity measures for rheumatoid arthritis clinical trials. Arthritis Rheum. 1993;36:729–40.
Bellamy N. Musculoskeletal Clinical Metrology. Dordrecht: Kluwer Academic Publishers; 1993:151–67.
Bellamy N, Muirden KD, Brooks P, Barroclough D, Tellus MM, Campbell J. A survey of outcome measurement procedures used in routine rheumatology outpatient practice by Australian rheumatologists. Scientific Program of the Australian Rheumatology Association 40th Annual Scientific Conference, Brisbane. May 25–28, 1997, p. 160.
Bellamy N, Kaloni S, Pope J, Coulter K, Campbell J. A survey of outcome measurement procedures used in routine rheumatology outpatient practice by Canadian rheumatologists. Canadian Rheumatology Association Annual Meeting, Mont-Tremblant, Quebec. February 27–March 1. J Rheumatol. 1997;24:1425.
Bellamy N. Musculoskeletal Clinical Metrology. Dordrecht: Kluwer Academic Publishers; 1993:339–41, 351–2.
Klinkoff AV, Bellamy N, Bombardier C et al. An experiment in reducing interobserver variability of the examination for joint tenderness. J Rheumatol. 1988;15:492–4.
Bellamy N, Muirden KD, Bendrups A et al. Effect of a standardized videotape on observer dependent outcome measures in rheumatoid arthritis. Scientific Program of the Australian Rheumatology Association 40th Annual Scientific Conference, Brisbane. May 25–28, 1997, p. 161.
Bellamy N, Bachmeier C, Brooks P et al. Effect of a standardized videotape on observer-dependent outcome measures in osteoarthritis. Scientific Program of the Australian Rheumatology Association 40th Annual Scientific Conference, Brisbane. May 25–28, 1997, p. 162.
Bellamy N, Muirden KD, Boyden K et al. Effect of a standardized videotape on observer dependent outcome measures in ankylosing spondylitis. Scientific Program of the Australian Rheumatology Association 40th Annual Scientific Conference, Brisbane. May 25–28, 1997, p. 159.
Pope JE, Baron M, Bellamy N et al. Variability of skin scores and clinical measurements in scleroderma. J Rheumatol. 1995;22:1271–6.
Green S, Buchbinder R, Forbes A, Bellamy N. A standardized protocol for measurement of range of movement of the shoulder using the plurimeter-V inclinometer and assessment of its intrarater and interrater reliability. Arthritis Care Res. 1997 (Submitted).
Bellamy N. Musculoskeletal Clinical Metrology. Dordrecht: Kluwer Academic Publishers; 1993:11–24.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bellamy, N., Muirden, K.D., Bendrups, A. et al. Rheumatoid arthritis antirheumatic drug trials: Effects of a standardized instructional videotape on the reliability of observer-dependent outcome measures. Inflammopharmacol 5, 273–284 (1997). https://doi.org/10.1007/s10787-997-0005-x
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s10787-997-0005-x