Abstract
Objective: To assess the inter-rater reliability of 14 explicit indicators of appropriate long-term prescribing.
Method: All available data required for the assessment of 59 long-term prescriptions started during a hospital admission for 25 patients were transcribed from the patients’ medical records. These transcripts were presented in a standardised format and random order to four raters (two doctors and two pharmacists) who used the indicators to judge the appropriateness of each prescription. Debriefing interviews were held with each rater. An a priori level of acceptable agreement between the raters was set at a weighed kappa of 0.70.
Results: There was no apparent difference between pharmacists and doctors for all findings, so data were combined. Two indicators showed poor agreement, three showed moderate agreement, and nine showed substantial or near perfect agreement, exceeding a weighted kappa of 0.70. There was excellent positive agreement as to which prescriptions were judged appropriate by the indicators, but much worse negative agreement as to which prescriptions were judged to be inappropriate. In the interviews, the raters remarked on the difficulty of applying explicit indicators when they routinely made implicit judgements about data in the medical records.
Conclusion: Nine of the indicators achieved the required level of reliability and the negative agreement levels showed that this was the area that required greater improvement in future developments of the indicators. Further work needs to be conducted to investigate ways of the improving instructions on how to make explicit judgements and reducing the need for implicit or subjective assessments.
Similar content being viewed by others
References
CA Oborne R Hooper KC Li CG Swift SHD. Jackson (2002) ArticleTitleAn indicator of appropriate neuroleptic prescribing in nursing homes Age Ageing 31 IssueID6 435–9 Occurrence Handle10.1093/ageing/31.6.435 Occurrence Handle12446288
J Naish P Sturdy P. Toon (1995) ArticleTitleAppropriate prescribing in asthma and its related cost in east London BMJ 310 97–100 Occurrence Handle7833736
MH Beers JG Ouslander I Rollingher DB Reuben J Brooks JC. Beck (1991) ArticleTitleExplicit criteria for determining inappropriate medication use in nursing home residents Arch Intern Med 151 1825–32 Occurrence Handle10.1001/archinte.151.9.1825 Occurrence Handle1888249
DF Thompson NR. Heflin (1987) ArticleTitleFrequency and appropriateness of drug prescribing for unlabeled uses in pediatric patients Am J Hosp Pharm 46 IssueID792 794
Medical Economics. Physician’s Desk Reference. 58th ed. Montvale, NJ: Medical Economics Co Inc., 2004.
Joint Formulary Committee. British National Formulary. British Medical Association and Royal Pharmaceutical Society of Great Britain; 2003.
JT Hanlon KE Schmader GP Samsa M Weinberger KM Uttech IK Lewis et al. (1992) ArticleTitleA method for assessing drug therapy appropriateness J Clin Epidemiol 45 1045–51 Occurrence Handle10.1016/0895-4356(92)90144-C Occurrence Handle1474400
JA Cantrill B Sibbald S. Buetow (1998) ArticleTitleIndicators of the appropriateness of long term prescribing in general practice in the United Kingdom: consensus development, face and content validity, feasibility and reliability Qual Health Care 7 130–5 Occurrence Handle10185138
L Fitzgerald JT Hanlon P Shelton PB Landsman KE Schmader C Pulliam et al. (1997) ArticleTitleReliability of a modified medication appropriateness index in ambulatory older persons Ann Pharmacother 31 543–8 Occurrence Handle9161645
R Kassam L Martin KB. Farris (2003) ArticleTitleReliability of a modified medication appropriateness index in community pharmacies Ann Pharmacother 37 40–6 Occurrence Handle10.1345/aph.1C077 Occurrence Handle12503931
GP Samsa JT Hanlon KE Schmader M Weinberger EC Clipp KM Uttech et al. (1994) ArticleTitleA summated score for the medication appropriateness index: development and assessment of clinimetric properties including content validity J Clin Epidemiol 47 891–6 Occurrence Handle10.1016/0895-4356(94)90192-9 Occurrence Handle7730892
MP Tully JA. Cantrill (2002) ArticleTitleDevelopment of explicit indicators of appropriateness of prescribing (abstract) Pharm World Sci 24 24 Occurrence Handle10.1023/A:1019522921621
N Javed MP. Tully (2002) ArticleTitleContent validity of indicators of the appropriateness of long-term prescribing started during a hospital admission Int J Pharm Pract 10(suppl) R55
MP Tully JA. Cantrill (2003) ArticleTitleThe validity of explicit indicators of appropriateness of long term prescribing Int J Pharm Pract 11(suppl) R49
S Taylor B. Bogdan (1998) Introduction to Qualitative Research Methods A Guide Book and Resource EditionNumber3 John Wiley & Sons New York
G. Dunn (1992) ArticleTitleDesign and analysis of reliability studies Stat Meth Med Res 1 123–57
DV Cicchetti AR. Feinstein (1990) ArticleTitleHigh agreement but low kappa: II. Resolving the paradoxes J Clin Epidemiol 43 551–8 Occurrence Handle10.1016/0895-4356(90)90159-M Occurrence Handle2189948
JR Landis GG. Koch (1977) ArticleTitleThe measurement of observer agreement for categorical data Biometrics 33 159–74 Occurrence Handle843571
P Shelton JT Hanlon PB Landsman M Scott IK Lewis KE Schmader et al. (1997) ArticleTitleReliability of drug utilisation evaluation as an assessment of medication appropriateness Ann Pharmacother 31 533–42 Occurrence Handle9161644
CA Oborne GM Batty V Maskrey CG Swift SHD. Jackson (1997) ArticleTitleDevelopment of prescribing indicators for elderly medical inpatients Br J Clin Pharmacol 43 91–7 Occurrence Handle9056058
M. Berg (1996) ArticleTitlePractices of reading and writing: the constitutive role of the patient record in medical work Soc Health Illness 18 499–524 Occurrence Handle10.1111/1467-9566.ep10939100
M. Berg (1998) ArticleTitleMedical Work and the computer-based patient record: a sociological perspective Meth Inf Med 37 294–301
SA Buetow B Sibbald JA Cantrill S. Halliwell (1997) ArticleTitleAppropriateness in health care: application to prescribing Soc Sci Med 45 261–71 Occurrence Handle10.1016/S0277-9536(96)00342-5 Occurrence Handle9225413
D. Collins (2003) ArticleTitlePretesting survey instruments: an overview of cognitive methods Qual Life Res 12 229–38 Occurrence Handle10.1023/A:1023254226592 Occurrence Handle12769135
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tully, M.P., Cantrill, J.A. Inter-rater Reliability of Explicit Indicators of Prescribing Appropriateness. Pharm World Sci 27, 311–315 (2005). https://doi.org/10.1007/s11096-005-2453-y
Issue Date:
DOI: https://doi.org/10.1007/s11096-005-2453-y