The United States (US) overdose epidemic has been characterized by multiple waves over time, the first of which implicated pharmaceutical companies and healthcare systems as opioid prescriptions surged from the late 1990s until the early 2010s. As a result, policymakers,1 law enforcement task forces,2 and healthcare institutions3 have focused on reducing opioid prescribing in the hopes that reduced overdose mortality would result—an assumption we now understand to be oversimplified.4 As later waves of the overdose epidemic have been driven by heroin, illicit fentanyl, and co-use of multiple substances, clinicians and public health professionals have begun critically re-examining our understanding of the risks and benefits of prescription opioids,5, 6 as well as the relationship between prescribing patterns and overdose mortality.3, 7, 8 However, we have also seen the federal Drug Enforcement Administration (DEA)—a federal law enforcement agency whose mission is to reduce the availability of controlled substances (including those legally produced) on the illicit domestic and international markets in accordance with the Controlled Substances Act9—take on a heightened vigilance through the monitoring and surveilling of prescription drug monitoring programs (PDMPs).10, 11 An unintended consequence of this heightened vigilance has been an increased regulatory burden and, at times, a tenor of fear among clinicians.12 Despite reassurances brought by the 2022 Supreme Court ruling in Ruan v. US,13 many clinicians have abruptly reduced or discontinued prescription opioids,14 restricting access to primary care for patients15, 16 and potentially facilitating a transition to the illicit market where overdose and death are frighteningly common.17, 18 It is at this unique and dynamic intersection between the domains of public health and public safety that proprietary technologies marketed as clinical decision support (CDS) tools have entered clinical care without rigorous external validation, public data sharing, or regulatory approval.

Bamboo Health’s NarxCare is one such technology: a PDMP-based analytics platform marketed as a CDS tool for physicians, pharmacists, and policymakers.19 Its patented algorithm, embedded into clinic workflows, generates multiple “Narx Scores” purporting to represent a patient’s risk of non-medical use of prescription “narcotics” (opioids), sedatives, and stimulants, and a composite score quantifying the overall risk of overdose death.20 Clinicians view Narx Scores prominently displayed—such as next to a patient’s height, weight, and allergies21—in electronic medical record (EMR) systems across over 45 states, and in national pharmacies such as Sam’s Club and Walmart. Although it is unclear exactly how Narx Scores are used clinically, Bamboo Health’s promotional materials suggest NarxCare’s rapid implementation—amplified by near-universal state PDMP mandates—has positioned it to impact over one billion patient encounters annually across the US.21

As clinicians and researchers with experience in chronic pain, addiction medicine, and bioethics, we argue that NarxCare’s platform lacks sufficient evidence for its wide clinical implementation and are concerned its use may promote decisions that harm patients and exacerbate disparities in pain management. NarxCare is not approved by the US Food and Drug Administration (FDA), and Bamboo Health has not yet provided evidence that establishes the algorithm’s benefit on relevant clinical outcomes (e.g., overdose mortality, hospitalizations). Furthermore, public access to the algorithm’s data elements has not been granted to enable external validation, distinguishing it from transparent evidence-based clinical prediction models such as the Canadian CT head rule22, 23 and the TIMI risk score,24,25,26 among others, that were rigorously developed and validated in peer-reviewed literature. NarxCare, and other tools like it,27 have flourished because regulatory oversight of proprietary CDS platforms in the US has been in flux,28 and the ability of researchers to independently validate them is hampered by their proprietary nature. In the case of NarxCare, its rapid implementation has been supported by funds from state and federal justice departments,29, 30 without establishing its safety and efficacy. As clinicians in this setting, we are concerned that if our best judgment conflicts with fear of criminal liability,31 our ability to provide evidence-based, compassionate care to our patients may be compromised.

INSUFFICIENT EVIDENCE AND REGULATORY OVERSIGHT

Medical device manufacturers are required to undergo a process establishing the safety and efficacy of any new screening, diagnostic, or treatment devices to obtain FDA approval before the device can be marketed. However, there has been uncertainty as to whether this requirement extends to CDS tools like NarxCare, which the FDA considers “Software as a Medical Device products”—a category of products for which regulatory oversight is not always required. After concerns were raised about the potential harms of having an inconsistent approach,28 the FDA issued guidance in 2022 to clarify their stance on CDS software and to establish 4 criteria that CDS tools seeking to be excused from FDA regulatory oversight must fulfill to be considered non-device CDS.32 Although issued after NarxCare’s implementation, the FDA guidance seems to support that NarxCare should in fact be considered a medical device33 and undergo regulatory processes. To our knowledge, this has not yet occurred.

Irrespective of FDA approval, when we—as clinicians—see a clinical tool embedded in the EMR we use every day, we expect a certain evidentiary minimum to substantiate the tool’s place within the EMR. Indeed, Bamboo Health claims on its website that NarxCare improves outcomes for individuals prescribed opioid medications.19 However, this claim is based on a single study published after NarxCare’s integration into many state PDMPs34 that fails to meet the standard set by other validation studies of clinical prediction models.23, 25, 26 The study was limited in scope, using a convenience sample of 1 year of data from two states (Ohio and Indiana). Furthermore, its primary outcome was a self-reported measure, the World Health Organization Alcohol, Smoking, and Substance Involvement Screening Test (WHO ASSIST).34 While the WHO ASSIST is a reliable measure to identify substance use and related problems in primary care settings,35 it does not reflect clinical events such as opioid overdose as would be expected for a validation study. To date, no published study has examined the relationship between Narx Scores and relevant clinical outcomes.

In comparison to the WHO ASSIST, NarxCare was found to have a 17% false positive rate, 13% false negative rate, and Cohen’s kappa of 0.35 (considered poor-to-fair agreement).34 Tools with high false positive rates contain an embedded bias: they prioritize sensitivity, accepting the tradeoff of poor specificity (in many cases overestimating scores).36 In the context of the US overdose epidemic—and a relationship between opioid prescribing and overdose mortality that remains incompletely understood—there are real and substantial harms associated with overestimating a patient’s overdose risk. For example, a clinician might take a punitive action such as a forced opioid taper14 or patient abandonment16—both of which are associated with increased risk for overdose37,38,39—against a patient who had a high Narx Score but in reality was at low risk of overdose previously. Bamboo Health explicitly states clinicians should not make decisions using Narx Scores alone19; however, quality reporting has established that this is likely already occurring.40, 41 We fear NarxCare may paradoxically increase the risk for overdose by exerting pressure on clinicians to respond to scores that we lack the evidence to interpret or meaningfully address.

EXACERBATION OF DISPARITIES

While we do not know exactly which data elements are included in NarxCare, evidence suggests the algorithm differentially identifies individuals on the basis of work status, disability, and insurance, among other characteristics.42 Cochran et al.34 found patients with higher pain severity or interference, those who were widowed, on leave, retired, or disabled, were most likely to have artificially elevated Narx Scores (“false positives”). As the authors hypothesize, it is possible that patients with unmanaged pain related to work status or disability now also experience challenges accessing care due to having an artificially elevated Narx Score. Furthermore, individual proxy measures likely used in NarxCare’s algorithm target patients with complex medical histories. For example, one study found that 20% of patients identified as “doctor-and-pharmacy shopping” according to a commonly accepted definition using prescription-level data—a factor likely included in NarxCare’s algorithm—were in fact diagnosed with cancer, thus necessitating visits with multiple providers to adhere to complex treatment regimens.43 Patients without a primary care clinician, and/or those who see multiple different clinicians to manage pain associated with a complex medical condition (such as cancer), may be incorrectly flagged as high risk by NarxCare’s algorithm and create barriers to getting the care they need.

Perhaps most concerning is that proprietary CDS tools carry the potential to incorporate additional data sources that reflect systemic inequalities in society, yet are unrelated to clinical care. Any company with access to various types of data (ex., credit history, zip code, educational attainment, or criminal justice history, among others) has the capacity to include these data elements in their proprietary algorithms without announcing it publicly. In the case of predictive modeling formulaic algorithms such as NarxCare, the harms of doing so may be amplified. By using retrospective data elements as proxy measures, predictive modeling algorithms predict future patterns of behavior by modeling past patterns of behavior. They are not designed to identify or address the root causes of the patterns predicted, and in this sense, they can perpetuate, rather than address, the disparities in place when the algorithm was created.

This context is alarming as the medical community grapples with an increased awareness that clinical algorithms have the potential to discriminate—exemplified by the race adjustment embedded in the formula calculating glomerular filtration rate (GFR) that was used in clinical care and accepted by clinicians for decades without question, despite causing racial inequities in access to life-saving treatment.44, 45 Although Bamboo Health’s current promotional materials state only data from PDMP registries are included in NarxCare’s algorithm,19 state PDMP user manuals as recently as May 2020 stated that the platform could and will incorporate non-PDMP data sources in the future.20 It is already known that several states incorporate criminal justice histories into their PDMPs.10, 42 Furthermore, as recently as 2021 Bamboo Health belonged to an umbrella organization—Appriss—responsible for designing software leveraging outstanding warrants and incarceration status to predict patterns of criminal activity for law enforcement.44, 46 Regardless of whether criminal justice information—or other types of data unrelated to clinical care—are included in NarxCare, Bamboo Health has not sufficiently reassured clinicians that its algorithm is free from potentially discriminating sources of data. Furthermore, its opaque nature does not allow us to determine for ourselves whether this is the case.

ACTION STEPS ON NarxCare

There is not yet sufficient evidence to support nation-wide implementation of Bamboo Health’s NarxCare without more rigorous evidence and public data sharing. Shielded by a lack of federal regulatory oversight and proprietary formulas, CDS tools like NarxCare have flourished in recent years, exerting an unchecked and outsized influence on our patients’ care.

In Table 1, we suggest specific actions by federal regulatory agencies, healthcare institutions, individual clinicians, and researchers, as well as academic journals, to mitigate potential harms associated with unregulated tools like NarxCare. Actions at each level across the healthcare delivery system and the larger scientific community may help ensure a safer, more effective, and ethical engagement with proprietary technologies purporting to improve clinical care. Clinicians, in particular, are left with the question of: how much clinical decision-making are we willing to lend to opaque, proprietary, and unvalidated tools?

Table 1 Suggested Actions to Mitigate Potential Harms Associated with Unregulated Clinical Decision Support Tools Like NarxCare, by Actor
Full size table