Author: Jason A. Trubiano, MBBS, PhD; Centre for Antibiotic Allergy and Research, Department of Infectious Diseases, Austin Health, Heidelberg, Australia

Full Citation: Trubiano JA, Vogrin S, Chua KYL, Bourke J, Yun J, Douglas A, Stone CA, Yu R, Groenendijk L, Holmes NE, Phillips EJ. Development and Validation of a Penicillin Allergy Clinical Decision Rule. JAMA Intern Med. 2020 May 1;180(5):745–752

Abstract Link: https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2762878

Article Type: Diagnosis, Clinical Decision Rule derivation and validation

Ratings: Methods—4/5 Usefulness—3/5

Introduction

Background

Penicillin allergy is the most commonly reported drug allergy with an estimated prevalence of 5–10% in both pediatric and adult populations. However, about 90% of these reported allergies may be mislabeled. Penicillin allergy labels are associated with higher usage of broad-spectrum antibiotics, lower treatment effectiveness, higher rates of drug-related adverse events and higher treatment costs. Physicians are often inaccurate in their appraisal of the allergy history since it often involves vague or unknown symptoms and thus may not reliably identify patients at low risk of true penicillin allergy.

Objectives

To derive and validate a clinical decision rule to identify adult patients at low risk of having a true penicillin allergy who may not require formal allergy testing.

Methods

Design

Derivation and internal validation in a prospectively recruited cohort and retrospective external validation of a clinical decision rule in three different cohorts.

Subjects

Adults 16 years and older reporting a penicillin allergy.

Outcomes

Primary outcome was any positive result of a penicillin allergy test defined as skin prick test, intradermal test (immediate or delayed), patch test or oral challenge.

Main results

Internal derivation/validation: 622 patients from one center (59.0% female, median age 60 years).

In this cohort, a total of 732 penicillin allergies were reported by 679 patients. Patients who did not undergo an oral challenge following a negative skin test and those who had missing data were excluded. Consequently, 622 patients who underwent a skin test and/or had an oral challenge were included in the logistical analysis. Of those, 9.3% had a positive allergy test result (skin test or oral challenge testing) (95% CI 7.2–11.9%).

A multivariate logistic regression identified four factors associated with a positive result on penicillin allergy testing. Points were accorded to each of these predictive factors (mnemonic PEN-FAST):

  1. 1.

    Five years or less since reaction (2 points)

  2. 2.

    Anaphylaxis or angioedema (2 points)

  3. 3.

    Severe cutaneous reaction (SCARFootnote 1) (2 points)

  4. 4.

    Treatment required for allergy episodeFootnote 2 (undefined) (1 point)

The derivation accuracy was reported as AUC = 0.808; Hosmer–Lemeshow χ2 = 1.84; P = 0.61; the internal validation accuracy was AUC = 0.805 and mean optimism was 0.003.

The following four risk groups were developed:

  1. 1.

    Very low risk (0 points); 0.6% risk of allergy

  2. 2.

    Low risk (1–2 points); 5% risk of allergy

  3. 3.

    Moderate risk (3 points); 20% risk of allergy

  4. 4.

    High risk (4–5 points); 50% risk of allergy

A cutoff of less than three points classified 74% of the cohort as at low risk of allergy, of which 3.7% had a positive test result. This cutoff resulted in a sensitivity of 70.7% (95% CI 57.3–81.9%); specificity of 78.5% (95% CI 74.9–81.9%), PPV of 25.3% (95% CI 18.8–32.7%) and NPV of 96.3% (95% CI 94.1–97.8%).

External validation: 995 patients from 3 centers (70.1% female, median age 55 years).

The PEN-FAST decision rule performance was similar in these cohorts: AUC = 0.73 (95% CI 0.66–0.81) for Perth (334 patients), 0.78 (95% CI 0.68–0.88) for Sydney (80 patients) and 0.74 (95% CI 0.62–0.86) for Nashville (531 patients). The sensitivity and the NPV for Perth were 87.5% (IC 74.8–95.3) and 95.0% (89.4–98.1), for Sydney 70.4% (49.8–86.2) and 84.9% (72.4–93.3), for Nashville 73.7% (4.8–90.9) and 98.4% (96.3–99.5). Interestingly, the Sydney cohort, which had the highest prevalence of positive tests (33.8%), also had a NPV of only 84.9% (95% CI 72.4–93.3%) using a cutoff of at least three points to identify a positive test, though it had the smallest population.

Appraisal

Strengths

  • Targets a clinically relevant question

  • Potential impact on healthcare resource utilization

  • Prospectively collected data used for derivation and internal validation cohort

  • Simple to use. Easy to remember mnemonic; probably does not require much cognitive aid and thus is more likely to be adopted by clinicians.

Limitations

  • “Treatment required” criteria is undefined and may include a wide spectrum of interventions

  • No information supplied about baseline demographic characteristics of patients for the external validation cohorts

  • Retrospective external validation, not independent

  • Good NPV may be attributed to relatively low prevalence of true penicillin allergy (9.3% in the derivation cohort), though this prevalence is consistent with previously published data

  • No data on inter-user agreement in validations

  • Patients being referred to allergy testing by specialist might induce a selection bias

  • Does not address the pediatric population

  • Does not address other beta-lactams, other antibiotic classes or intravenous penicillins

Context

The predictors of true penicillin allergy identified in this derivation are consistent with previously published studies of mostly retrospective data [1]. Chiriac et al. derived and validated a decision rule on β-lactam using presence of anaphylaxis and time from event as predictors of allergy, but its performance was deemed insufficient (AUC, 0.67, NPV, 83%) [2]. Likewise, Siew et al. described a low-risk criteria for penicillin allergy with a NPV of 98.4% involving similar predictors but it was not externally validated [3]. A consulted local immunologist noted that most patients were diagnosed as allergic to penicillin based on skin test results alone (without an oral challenge which is considered the gold standard of diagnosis), increasing the risk of false-positive patients. This is an important fact since this score was created with the presumption that these patients were truly allergic. It was also noted that a possible history of DRESS or Stevens–Johnson syndrome (SJS) is an absolute contraindication to oral drug challenge, irrespective of this score result. Their inclusion in the “Severe cutaneous reaction” criteria might prompt clinicians to erroneously try an oral drug challenge. These patients should be carefully evaluated because even small amounts of the suspect drug can reactivate these life-threatening reactions.

Bottom line

Even though this study targets a very important question and has the potential to have an important impact on clinical practice, the performance of the decision rule in the external validation is somewhat underwhelming and generalization concerns remain. NPV should be interpreted with caution considering the low prevalence of true penicillin allergy. Independent external validation and impact studies are warranted before the PEN-FAST can be safely used.