Skip to main content
SpringerLink
Log in

Content Validity and Inter-Rater Reliability of an Instrument to Characterize Unintentional Medication Discrepancies

  • Original Research Article
  • Published:
Drugs & Aging Aims and scope Submit manuscript

Abstract

Background

Medication discrepancies are medication-related problems (MRPs) that frequently occur when patients are transferred between settings of care. Older patients are at high risk for several reasons, including high consumption of medicines, and physical and cognitive deficiencies that can impair the communication process. Most previous studies that have evaluated medication discrepancies used instruments designed for clinical practice, but a well-validated and reliable instrument for clinical research is still lacking. Objectives: The aims of this study were to (i) develop an instrument to characterize medication discrepancies that fulfils quality requirements for classification of MRPs related to continuity of care and (ii) assess its content validity and inter-rater reliability.

Methods

The instrument was developed based on three main inputs: (i) a literature review to collect information about the quality requirements of instruments to characterize MRPs; (ii) another literature review to identify existing instruments to characterize MRPs and, more specifically, medication discrepancies; and (iii) previous experience from a pilot study on Belgian patients discharged from surgical and medical wards. Content validity was assessed using a modified Delphi technique with 11 healthcare professionals. Content validity indexes were calculated. For inter-rater reliability, three pharmacists (one experienced and two naive) were asked to identify and categorize (type and cause of) unintentional medication discrepancies for 21 patients discharged from hospital into the community. The intra-class correlation coefficient was calculated to compare the number of discrepancies identified, and a paradox-resistant index (AC1) was used to determine the inter-rater reliability for the type and cause of the discrepancy.

Results

The instrument had 54 items classified in three sections (type of discrepancy, cause and intervention), with detailed specifications on how to use it. All evaluations relative to content validity met predefined cut-off values, except for two of them. Intra-class correlation coefficients of ≥0.76 and AC1 coefficients of ≥0.89 were found for the number and the type of discrepancies, respectively. Regarding evaluation of the specific causes of medication discrepancies, final AC1 results of ≥0.86 were obtained, except for three items (which had values between 0.62 and 0.79).

Conclusion

The validity and reliability of the instrument developed to assess unintentional medication discrepancies at patient transition from the hospital to the community setting was found to be satisfactory.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Table I
Table II
Table III
Table IV
Table V
Table VI
Fig. A1

Similar content being viewed by others

References

  1. Boockvar KS, Liu S, Goldstein N, et al. Prescribing discrepancies likely to cause adverse drug events after patient transfer. Qual Saf Health Care 2009; 18 (1): 32–6.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  2. Forster AJ, Murff HJ, Peterson JF, et al. Adverse drug events occurring following hospital discharge. J Gen Intern Med 2005; 20 (4): 317–23.

    Article  PubMed Central  PubMed  Google Scholar 

  3. Williams EI, Fitton F. Factors affecting early unplanned readmission of elderly patients to hospital. BMJ 1988; 297 (6651): 784–7.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  4. Tam VC, Knowles SR, Cornish PL, et al. Frequency, type and clinical importance of medication history errors at admission to hospital: a systematic review. CMAJ 2005; 173 (5): 510–5.

    Article  PubMed Central  PubMed  Google Scholar 

  5. Coleman EA, Smith JD, Raha D, et al. Posthospital medication discrepancies: prevalence and contributing factors. Arch Intern Med 2005; 165 (16): 1842–7.

    Article  PubMed  Google Scholar 

  6. Wong JD, Bajcar JM, Wong GG, et al. Medication reconciliation at hospital discharge: evaluating discrepancies. Ann Pharmacother 2008; 42 (10): 1373–9.

    PubMed  Google Scholar 

  7. Tjia J, Bonner A, Briesacher BA, et al. Medication discrepancies upon hospital to skilled nursing facility transitions. J Gen Intern Med 2009; 24 (5): 630–5.

    Article  PubMed Central  PubMed  Google Scholar 

  8. Lee JY, Leblanc K, Fernandes OA, et al. Medication reconciliation during internal hospital transfer and impact of computerized prescriber order entry. Ann Pharmacother 2010; 44 (12): 1887–95.

    Article  PubMed  Google Scholar 

  9. Stitt DM, Elliott DP, Thompson SN. Medication discrepancies identified at time of hospital discharge in a geriatric population. Am J Geriatr Pharmacother 2011; 9 (4): 234–40.

    Article  PubMed  Google Scholar 

  10. Unroe KT, Pfeiffenberger T, Riegelhaupt S, et al. Inpatient medication reconciliation at admission and discharge: a retrospective cohort study of age and other risk factors for medication discrepancies. Am J Geriatr Pharmacother 2010; 8 (2): 115–26.

    Article  PubMed Central  PubMed  Google Scholar 

  11. Gleason KM, Groszek JM, Sullivan C, et al. Reconciliation of discrepancies in medication histories and admission orders of newly hospitalized patients. Am J Health Syst Pharm 2004; 61 (16): 1689–95.

    PubMed  Google Scholar 

  12. Coleman EA, Berenson RA. Lost in transition: challenges and opportunities for improving the quality of transitional care. Ann Intern Med 2004; 141 (7): 533–6.

    Article  PubMed  Google Scholar 

  13. van Mil JW, Westerlund LO, Hersberger KE, et al. Drugrelated problem classification systems. Ann Pharmacother 2004; 38 (5): 859–67.

    Article  PubMed  Google Scholar 

  14. Schaefer M., Discussing basic principles for a coding system of drug-related problems: the case of PI-Doc. Pharm World Sci 2002; 24 (4): 120–7.

    Article  PubMed  Google Scholar 

  15. Smith JD, Coleman EA, Min SJ. A new tool for identifying discrepancies in postacute medications for community-dwelling older adults. Am J Geriatr Pharmacother 2004; 2 (2): 141–7.

    Article  PubMed  Google Scholar 

  16. Taylor CT, Stewart LM, Byrd DC, et al. Reliability of an instrument for evaluating antimicrobial appropriateness in hospitalized patients. Am J Health Syst Pharm 2001; 58 (3): 242–6.

    CAS  PubMed  Google Scholar 

  17. Stuijt CC, Franssen EJ, Egberts AC, et al. Reliability of the medication appropriateness index in Dutch residential home. Pharm World Sci 2009; 31 (3): 380–6.

    Article  PubMed  Google Scholar 

  18. Spinewine A, Dumont C, Mallet L, et al. Medication appropriateness index: reliability and recommendations for future use. J Am Geriatr Soc 2006; 54 (4): 720–2.

    Article  PubMed  Google Scholar 

  19. Tully MP, Cantrill JA. Inter-rater reliability of explicit indicators of prescribing appropriateness. Pharm World Sci 2005; 27 (4): 311–5.

    Article  PubMed  Google Scholar 

  20. Forrey RA, Pedersen CA, Schneider PJ. Interrater agreement with a standard scheme for classifying medication errors. Am J Health Syst Pharm 2007; 64 (2): 175–81.

    Article  PubMed  Google Scholar 

  21. Grant JS, Davis LL. Selection and use of content experts for instrument development. Res Nurs Health 1997; 20 (3): 269–74.

    Article  CAS  PubMed  Google Scholar 

  22. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health 2007; 30 (4): 459–67.

    Article  PubMed  Google Scholar 

  23. Lynn MR. Determination and quantification of content validity. Nurs Res 1986; 35 (6): 382–5.

    Article  CAS  PubMed  Google Scholar 

  24. Burke MJ, Dunlap WP. Estimating interrater agreement with the average deviation index: a user’s guide. Organizational Research Methods 2002; 5: 159–72.

    Article  Google Scholar 

  25. Pit SW, Byles JE, Cockburn J. Accuracy of telephone self-report of drug use in older people and agreement with pharmaceutical claims data. Drugs Aging 2008; 25 (1): 71–80.

    Article  PubMed  Google Scholar 

  26. Cicchetti DV, Sparrow SA. Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. Am J Ment Defic 1981; 86 (2): 127–37.

    CAS  PubMed  Google Scholar 

  27. Gwet KL. The definitive guide to measuring extent of agreement among multiple raters. 2nd ed. Gaithersburg (MD): Advanced Analytics, LLC, 2010.

    Google Scholar 

  28. Gwet KL. AgreeStat 2011.2 for MS Excel [Excel workbook; online]. Available from URL: http://www.agreestat.com/agreestat.html [Accessed 2010 Jul 2].

  29. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33 (1): 159–74.

    Article  CAS  PubMed  Google Scholar 

  30. Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol 1990; 43 (6): 551–8.

    Article  CAS  PubMed  Google Scholar 

  31. Classification for drug related problems. V5.01. Pharmaceutical Care Network Europe Foundation, 2006 [online]. Available from URL: http://www.pcne.org/sig/drp/documents/drp/PCNE%20classification%20V5.01.pdf [Accessed 2008 Aug 13].

  32. Pippins JR, Gandhi TK, Hamann C, et al. Classifying and predicting errors of inpatient medication reconciliation. J Gen Intern Med 2008; 23 (9): 1414–22.

    Article  PubMed Central  PubMed  Google Scholar 

  33. Stuffken R, Heerdink ER, de Koning FH, et al. Association between hospitalization and discontinuity of medication therapy used in the community setting in the Netherlands. Ann Pharmacother 2008; 42 (7): 933–9.

    Article  PubMed  Google Scholar 

  34. Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA 1995; 274 (1): 35–43.

    CAS  Google Scholar 

  35. Schmitt E, Antier D, Bernheim C, et al. Dictionnaire français de l’erreur médicamenteuse. 1st ed. Montry: Société Française de Pharmacie Clinique, 2006 [online]. Available from URL: http://adiph.asso.fr/sfpc/Dictionnaire_SFPC_EM.pdf [Accessed 2008 Jul 9].

  36. Rantucci MJ, Pigot-Renou D, Renou G, et al. Le dialogue pharmacien-patient. Rueil-Malmaison: Pro-officina, 2008.

    Google Scholar 

  37. Mallet L, Grenier L, Guimond J, et al. Manuels de soins pharmaceutiques en gériatrie. Saint Nicolas: Les Presses de l’Université de Laval, 2003.

  38. 5 Million Lives Campaign. Getting Started Kit: prevent adverse drug events (medication reconciliation) how-to guide. Cambridge (MA): Institute for Healthcare Improvement, 2008 [online]. Available from URL: http://www.ihi.org/IHI/Programs/Campaign/ADEsMedReconciliation.htm [Accessed 2008 Sep 8].

  39. Getting Started Kit: medication reconciliation: prevention of adverse drug events: how-to guide. The Institute for Safe Medication Practices Canada, 2007 [online]. Available from URL: http://www.saferhealthcarenow.ca/Default.Aspx? [Accessed 2008 Sep 8].

  40. Allenet B, Bedouch P, Rose FX, et al. Validation of an instrument for the documentation of clinical pharmacists’ interventions. Pharm World Sci 2006; 28: 181–8.

    Article  PubMed  Google Scholar 

  41. Kripalani S, Jackson AT, Schnipper JL, et al. Promoting effective transitions of care at hospital discharge: a review of key issues for hospitalists. J Hosp Med 2007; 2 (5): 314–23.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Thanks are expressed to all participants who contributed to the validation of the instrument. G. Stoddard, Co-Director, Study Design and Biostatistics Center, adjunct Assistant Professor, Department of Orthopedics, Division of Epidemiology, Department of Internal Medicine, University of Utah School of Medicine, is also greatly acknowledged for his useful explanations. We thank B. Sneyers and A. Bastos for their helpful comments.

C. Claeys was a Research Fellow of the Belgian Fonds de la Recherche Scientifique (F.R.S-FNRS), but this institution did not influence the design, nor did it participate in the analysis and reporting of the study.

No sources of funding were used to assist in the preparation of this study. The authors have no conflicts of interest that are directly relevant to the content of this article.

Part of this work was presented as oral communication and poster at the 38th Symposium of the European Society of Clinical Pharmacy, Geneva, 3–6 November 2009, by C. Claeys, P.M. Tulkens, J. Nève and A. Spinewine (number PEH-02).

Author information

Authors and Affiliations

  1. Laboratory of Pharmaceutical Chemistry, Faculty of Pharmacy, Université libre de Bruxelles, Campus Plaine 205-5, B-1050, Bruxelles, Belgique

    Coraline Claeys MPharm & Jean Nève

  2. Louvain Drug Research Institute, Université catholique de Louvain, Brussels, Belgium

    Paul M. Tulkens & Anne Spinewine

  3. CHU Mont-Godinne, Université catholique de Louvain, Yvoir, Belgium

    Anne Spinewine

Authors
  1. Coraline Claeys MPharm
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jean Nève
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul M. Tulkens
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anne Spinewine
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Coraline Claeys MPharm.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Cite this article

Claeys, C., Nève, J., Tulkens, P.M. et al. Content Validity and Inter-Rater Reliability of an Instrument to Characterize Unintentional Medication Discrepancies. Drugs Aging 29, 577–591 (2012). https://doi.org/10.1007/BF03262275

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF03262275

Keywords

Search

Navigation