Introduction

Mortality from opioid-related overdoses remains a significant public health issue [1], with a reversal in recent progress during the COVID-19 pandemic [2, 3]. Consequently, expanding naloxone availability remains a core harm reduction strategy to address the opioid epidemic in the US [4, 5]. State-level naloxone access laws (NALs) aim to stimulate greater naloxone availability to communities through various mechanisms, such as community pharmacies [6], community-based programs [7], emergency departments [8], and law enforcement initiatives [9]. Substantial evidence exists supporting efforts to reduce opioid-related overdose mortality through naloxone distribution [10].

Despite the priority of addressing the opioid crisis and empirical evidence supporting naloxone’s efficacy in reversing overdoses, there is substantial variation in state adoption of NALs [11], mixed results for the effectiveness of different NAL components [12,13,14,15,16], and growing partisanship regarding how best to address the crisis [17]. In the absence of clear and direct empirical evidence, formal consensus methods can help clarify what experts believe is (not) effective and implementable [18, 19]. Previous work found expert consensus on the average effectiveness and implementability of various state-level NALs [20], but there are numerous contextual considerations that may mediate or moderate the effects of policies as implemented in practice. Furthermore, given variability in potential barriers to effective and equitable implementation of NALs [21,22,23,24,25], it is important to clarify whether and why experts believe various policy options are (not) implementable. The objective of this qualitative study is summarizing experts’ rationale for ratings in an online-modified Delphi process on the extent to which NALs are effective, acceptable, feasible, affordable, and equitable.

Material and methods

The study was deemed exempt by the RAND Human Subjects Protection Committee (ID-2018–0506). We prospectively registered the study on the Open Science Framework (https://osf.io/f4hk8/). A separate manuscript reports quantitative findings from the online modified-Delphi process [20]. We developed this manuscript using the Standards for Reporting Qualitative Research [26].

Qualitative approach and researcher reflexivity

Our study was guided by the GRADE Evidence-to-Decision (EtD) Framework, which facilitates structured and transparent use of evidence to inform public health recommendations and decisions [27]. Our qualitative approach involved a post-positivist research paradigm employing a combination of coding grounded in the data and thematic analysis informed by constructs in the GRADE EtD Framework. The study authors have training in social science disciplines that emphasize evidence-informed decision-making and generally privilege quantitative methods for causal inference.

Sampling strategy

We recruited participants identifying with one of several stakeholder groups (advocates, healthcare providers, human or social service practitioners, policymakers, and researchers). We first developed a recruitment list from published research related to NALs, project advisory board suggestions, and relevant organizations’ member lists. We also used a “snowball sampling” approach allowing approached stakeholders to nominate further participants. We then emailed a recruitment survey to potential participants to indicate whether they would be willing to participate in the online modified-Delphi process, offering a $300 gift card or prepaid debit card for study completion. Individuals indicating interest provided demographic data and stated their preference to participate in the panel focused on either (a) effectiveness or (b) implementability of NALs. Participants electronically provided informed consent, which included information about sharing de-identified information and assurances of protecting confidentiality of responses and discussion comments. We aimed to recruit 40–80 participants (20–40 participants per panel), based on guidance for online expert panels that aim to engage large, diverse, and geographically-distributed groups of stakeholders in consensus exploration [28].

Data collection

We conducted two online modified-Delphi panels concurrently during summer 2020 using RAND’s ExpertLens system: one on NAL effectiveness and another on their implementability. We explicitly instructed participants that, while there are several critically important channels for naloxone distribution, we specifically were interested in understanding policies that target naloxone access and distribution through pharmacies (e.g., chain pharmacies, independent community pharmacies). In Round One, participants rated each policy on four outcomes (Effectiveness Panel) or implementability criteria (Implementation Panel). In Round Two, participants explored areas of agreement and disagreement by discussing Round One results in an anonymous, asynchronous online forum. In Round Three, participants re-rated Round One results following Round Two discussion [29]. Each round takes participants approximately one hour to complete and remains open for one to three weeks.

We constructed a list of 15 NALs through an iterative process of feedback from the project’s advisory board and cognitive testing on the ExpertLens platform (see Table 1). We developed criteria for assessing each policy using the GRADE EtD Framework [27] and APEASE framework from the Behaviour Change Wheel [30]. These frameworks underlie prominent approaches to evidence-informed decision-making via explicit criteria on the effectiveness, acceptability, feasibility, affordability and equitability of interventions under consideration (see Table 2).

Table 1 State-level naloxone access laws
Full size table
Table 2 Criteria for assessing state-level naloxone access laws
Full size table

Data analysis

To examine stakeholders’ views on NAL effectiveness and implementability, we systematically coded all comments from the two rating rounds and discussion round [31, 32]. The first author (SG) initially grouped all comments by NAL and criterion, ordered comment groupings by stakeholders’ numeric ratings, and thoroughly read and re-read the material [33, 34]. SG then conducted line-by-line coding, relating the raw data to codes that were grounded in the data itself and that emerged through constant comparison and refinement during coding. Lastly, SG systematically indexed the codes into preliminary themes, revised and integrated similar themes, relabeled the final themes, and identified quotations best exemplifying themes. The second author (RS) reviewed all codes, themes, and supporting quotations.

Results

Of 94 potential participants, 46 (48.9%) agreed to participate: 24 in the panel on NAL effectiveness (Panel A) and 22 in the panel on NAL implementability (Panel B). Most panelists were female (54%), White Non-Hispanic (78%), identified as researchers (61%), and resided in the Northeast (41%). We did not identify any discernable differences in final themes by participant demographics. Overall, participants provided 2,658 comments: 1,479 in Round One (698 in Panel A, 781 in Panel B), 319 in Round 2 (123 in Panel A, 196 in Panel B), and 860 in Round Three (478 in Panel A, 382 in Panel B). Across all comments, participants cited several studies as warrant for their claims [6, 12, 15, 35,36,37,38,39,40,41].

Overall themes

We organized overall themes by the domains of the GRADE EtD Framework: effectiveness, acceptability, feasibility, affordability, and equity (see Table 3).

Table 3 Overall themes for specific categories of the evidence-to-decision framework
Full size table

Effectiveness

Participants believe that numerous NALs can improve naloxone pharmacy distribution—and that several of these NALs can produce population-level reductions in opioid overdose mortality via greater probabilities of naloxone being present and administered during an overdose. However, participants consistently noted that meaningful reductions in mortality would be modest for any NAL that has pharmacy distribution as its mechanism of action, given pharmacies’ limitations as a setting for distributing naloxone. For example, many participants commented that barriers for pharmacy distribution include stigma towards people who use drugs, potential high out-of-pocket costs of naloxone, and community-based programs being more likely to reach those with higher risks of overdosing from opioids. Consequently, participants believed only NALs that make it easy and affordable for anyone to obtain naloxone without a prescription—and that align the stocking and dispensing of naloxone with routine pharmacy operations—have the potential for the substantial and sustainable increases in pharmacy naloxone distribution required for meaningful reductions in population-level overdose mortality.

Participants believed that NALs without links to prevention, screening, treatment, or recovery do not directly influence the prevalence of OUD or nonfatal opioid overdoses, as NALs primarily target opioid-related overdose mortality. That said, NALs that substantially increase naloxone distribution may indirectly have small and acute impacts on these outcomes: i.e., large reductions in fatal opioid overdoses result in more individuals with OUD surviving non-fatal overdoses. Four participants explicitly expressed concerns about this dynamic creating a moral hazard of naloxone distribution causing purposeful opioid misuse: i.e., concerns that people who use drugs will engage in riskier drug behaviors if they have access to naloxone. However, the consensus view was that NALs do not lead to “risk compensation”, but rather any increase in population-level OUD or non-fatal overdoses is a mechanical consequence of reducing fatal overdoses.

Acceptability

Participants rating NALs as highly acceptable to the general public often commented about actual implementation of specific NALs with little pushback, positive trends of public support for naloxone access, or lack of opposition due to lack of awareness of NAL existence. Nonetheless, participants also consistently noted that acceptability is still mired by stigma about people who use drugs—and naloxone as encouraging opioid use by extension. Participants offered strategies for mitigating this stigma, namely accompanying the passage of NALs with effective framing of naloxone as an evidence-based response to the opioid epidemic, and messaging that NALs would not increase opioid misuse.

Feasibility

Participants saw the feasibility of implementing NALs as dependent on existing resources and infrastructure, as well as levels of buy-in from stakeholders involved in implementation. Existing resources and infrastructure that modify feasibility included sustainable funding, health information technology, and physical space. Participants also noted that feasibility depends on whether an NAL adds new roles and responsibilities for the physicians, other prescribers, and pharmacists involved in naloxone distribution, as these professionals already feel overburdened by their current workloads: “requiring additional pharmacist time at the point of dispensing may be difficult” (B05). As with acceptability, participants noted stigma substantially influences stakeholder support for NALs and consequently NAL feasibility across a range of organizations and settings. Specific stakeholders included first responders, healthcare organizations, healthcare and social service providers, insurance companies, law enforcement, lawyers and legal experts, licensing boards, naloxone manufacturers, pharmacists and pharmacy chains, prescribers, professional associations, regulators, and state legislatures. Organizations and settings included including clinics, healthcare systems, hospitals, and pharmacies.

Affordability

Variability in ratings on the societal affordability of NALs was largely a factor of the cost of naloxone itself, who pays for naloxone, and effectiveness of an NAL in reducing overdoses. The cost of naloxone significantly influenced views on affordability because naloxone is the most significant direct and ongoing cost of NALs, and the cost of naloxone can vary due to numerous factors (e.g., impact of NAL on naloxone market pricing, type of naloxone product covered under an NAL). Consequently, as part of their rationale for affordability ratings, participants frequently commented on naloxone-related cost burdens for specific stakeholders (e.g., insurance coverage, out-of-pocket costs, pharmacy stocking costs, state purchasing, and subsidization costs). From a societal perspective, many participants rating NALs as highly affordable noted cost savings associated with decreased overdose mortality.

Equitability

Variability in equitability ratings largely related to inequities and disparities of society and the systems in which NALs are implemented, rather than the equitability of NALs per se. Namely, participants frequently commented on structural, systemic, and institutional oppression many populations face in the USA due to their race, ethnicity, socioeconomic status, and place of residence—and how the resultant unjust stratification of health opportunities and outcomes are likely to replicate in implementation of NALs that rely on pharmacies and the existing healthcare system. In addition, participants often identified discrimination and interpersonal biases held by healthcare providers and pharmacists as another potential source of inequitable implementation. Participants also noted pharmacies are a suboptimal mechanism for distributing naloxone to people most likely to experience or witness an overdose, as pharmacies are often less prevalent in specific areas (e.g., small, rural) and less accessible to specific populations (e.g., low-income). Consequently, even NALs that increase the number of places and ways that people can access naloxone still could widen disparities in naloxone access if implemented solely through pharmacies. Lastly, participants generally posited that lowering out-of-pocket naloxone cost increases an NAL’s equitability, particularly for low-income and under- or uninsured populations.

Themes for specific NALs

We organized themes for specific NALs by policy groupings: liability policies, education and training requirements, co-prescribing naloxone, layperson accessibility, expanded pharmacy access, and cost subsidization (see Table 4).

Table 4 Themes for Specific Categories of NAL
Full size table

Liability policies

Participants generally agreed that liability policies are ineffective because liability concerns are not a major barrier for naloxone prescribing and dispensing—even if liability protections may make some prescribers and pharmacists more comfortable prescribing and dispensing naloxone. Similarly, participants indicated in-principle support of liability protections for administration of naloxone by laypersons and first responders, although they did not view such policies as having a meaningful impact on naloxone distribution through pharmacies. The consensus that liability policies are acceptable to the public stemmed from perceptions of broad support for efforts to protect healthcare providers and laypersons acting in good faith to address the opioid epidemic. That said, several respondents noted that some members of the general public may be weary of the potential for liability protections to provide leeway for professional maleficence among prescribers and dispensers. Participants considered these policies as feasible and affordable primarily due to a lack of implementation challenges and ongoing costs once passed, especially because liability protection laws regarding naloxone are already well-established. Any concerns were related to potential pushback on the scope of liability protections during the legislative phase from law enforcement, licensing boards, professional societies, and trial lawyers. In contrast, participants generally considered liability policies only moderately equitable because these NALs at best do not counter and at worst risk replicating existing disparities of access to and biases in the healthcare system.

Education/training requirements

While considered acceptable to the general public, requirements of prescribers or pharmacists to offer training and education to recipients of naloxone were viewed as both ineffective and relatively less implementable. Participants postulated that these requirements create barriers to naloxone pharmacy distribution by increasing burdens on prescribers and dispensers, who are already pressed for time. The additional burden of this unfunded mandate would deter many physicians from prescribing and pharmacists from dispensing naloxone, offsetting any potential benefits of the education and training on knowledge about overdoses and competencies in using naloxone correctly.

Implementability concerns related to time constraints, reimbursement for training and education, the need to “train the trainers”, and the lack of infrastructure at pharmacies for confidential patient education. To mitigate these implementability concerns, numerous participants suggested offering education and training in flexible approaches and via streamlined technology (e.g., free, readily-available videos and handouts). For example, one participant noted that all communications and education tools from the federal Rural Health Opioid Program (Office of Rural Health Policy) are freely available for replication. Putting aside concerns about stigma of substance use and naloxone generally, perceived acceptability of these laws specifically stems from agreement among the general public to provide information about proper use for all medications. However, concerns about equitability arose by the interaction of the onerous aspects of this mandate with the structural oppression and interpersonal discrimination faced by many people and communities affected by these policies. Consequently, panel consensus reflected views that these policies likely disproportionately negatively impact marginalized communities, namely people of color, people with mental health disorders, people who are homeless, rural residents, and individuals with lower socioeconomic status or communities with less resources. That said, conditional on receiving training, these policies could improve equity among those who previously have been underserved in terms of medical education.

Co-prescribing naloxone

Naloxone co-prescribing requirements were generally viewed as effective in substantially increasing naloxone pharmacy distribution, but only modestly effective in reducing overdose mortality. While participants noted evidence that these policies expand access to naloxone through pharmacies [35,36,37], this substantially expanded access may not translate into large decreases in mortality because these policies focus on prescribed opioids rather than diverted prescriptions and illicit opioids. In addition, consensus ratings indicated only moderate implementability of these policies due to negative reactions from patients being labelled as persons needing naloxone, and providers being told what medications to prescribe and when to prescribe them. In addition, participants indicated concerns about ensuring that prescribers follow the mandate (e.g., through regulation, enforcement, and oversight). Another concern was the cost of this policy, given the number of opioids still prescribed in the USA and the perceived possibility for these policies to further incentivize pharmaceutical manufacturers to inflate naloxone prices. At the societal level, co-prescribing may be cost-effective through averted fatal opioid overdoses [42, 43], particularly if increased market competition from new naloxone products maintains affordability of naloxone itself [44]. Participants also raised concerns about the equitability of these policies, as they not only rely on access to pharmacies but also to access to prescribers—and for an issue (chronic pain) with documented racial and ethnic treatment disparities due to systemic racism and interpersonal biases [45].

Participants often showed support for the principle of using risk indicators beyond opioid overdose in order to cover overdose risk more broadly. However, participants noted numerous issues of this policy in-practice: patient perceptions of additional questions as invasive, provider burdens due to increased complexity and difficulty in obtaining this information (e.g., data privacy protections), uncertainty about the indicators to use (and subsequent waste in resources if the wrong indicators are used), and greater subjectivity (compared to the more objective criteria of using only prescribed opioid dosages) increasing the opportunity for interpersonal biases to yield further inequities.

Layperson accessibility

Participants generally viewed NALs facilitating greater accessibility of naloxone to anyone (regardless of opioid use status) as effective in removing barriers for laypersons and thereby increasing naloxone pharmacy distribution. Namely, over-the-counter (OTC) pharmacy supply was one of the NALs that participants believed had the best chance to destigmatize acquiring naloxone, because many third parties (especially family and caregivers) are likely to acquire naloxone via a pharmacy (rather than through a community-based naloxone distribution program). These ratings translated into perceptions of meaningful reductions in fatal overdoses from authorizing OTC naloxone pharmacy supply, but not from third party prescription due to the latter’s reliance on physician prescriptions and narrower focus on laypersons who may not actually witness overdoses. Conversely, participants viewed third party prescriptions as slightly more equitable than OTC pharmacy supply due to concerns about prohibitory retail costs of OTC naloxone for low-income persons. While both OTC supply and third-party prescribing had consensus ratings of “high” affordability, participants raised financial concerns for both (payer costs for third party prescribing, and patient out-of-pocket costs for OTC pharmacy supply), but viewed both as generally acceptable to the public and feasible changes to prescriber and dispenser practices.

Expanded pharmacy access

Participants also viewed expanded pharmacy access laws as effective in facilitating naloxone availability by removing patient barriers to acquiring naloxone (i.e., the need for patient-specific prescriptions and physician involvement) and enabling pharmacist autonomy. As evidence for their acceptability, participants frequently cited successful adoption of these NALs without much public pushback (likely due to their commensurability with obtaining other types of medication from pharmacists). Participants explained the “high feasibility” consensus was conditional on pharmacist willingness to dispense naloxone without physician involvement and lack of opposition from prescribers concerned about professional scope creep. “High affordability” was generally due to eliminating costs associated with prescriber office visits. Statewide standing/protocol orders and pharmacist prescriptive authority had the added benefit of lesser administrative costs compared to collaborative practice agreements (i.e., one statewide policy versus multiple agreements). Removing the need to access prescribers increased participant views of equitability, though participants noted the remaining possibilities of pharmacist bias and limited access to pharmacies.

Cost subsidization

Participants viewed cost-subsidization NALs as effective in significantly facilitating naloxone pharmacy distribution because they address the significant barrier of out-of-pocket costs to individuals. Participants particularly viewed statewide free naloxone as the most equitable NAL in effectively facilitating naloxone pharmacy distribution and reducing overdose mortality, given its elimination of out-of-pocket costs and potential for destigmatizing naloxone. However, they also considered it the least affordable policy for states—and least acceptable as a result. While insurance coverage was not seen as equitable and effective in reducing overdose mortality due to many high-risk patients not having insurance, participants viewed insurance coverage as the most implementable cost-subsidization policy due to public support of shifting burdens to insurance companies covering the medication. Participants similarly viewed state subsidies for naloxone purchase through insurance as less effective due to its reliance on high-risk patients having insurance, although they also saw it as less implementable than insurance coverage as it only provides assistance with co-pays (rather than full coverage) and costs fall on the state (rather than insurance companies).

Discussion

Experts believed that the effectiveness of NALs in reducing opioid-related mortality requires sustained, significant impacts on population-level naloxone availability. This necessitates addressing implementation barriers that apply broadly across NALs, including affordability of naloxone itself, reliance on an inequitable healthcare system, and stigma—which can mitigate the effectiveness of all types of policies considered [46, 47]. Experts also generally believed that increased naloxone availability does not have appreciable negative impacts on the prevalence of opioid misuse, OUD, and non-fatal opioid overdoses. This contrasts with recent work suggesting increasing rates of both nonfatal opioid-related overdoses and opioid-related crime following standing order or third-party prescribing laws [48], but aligns with several studies showing no evidence that take-home naloxone provision promotes increased opioid use or overdose [49]. Additionally, while experts expect any short-term mechanical increases in OUD or non-fatal overdoses to be small, many do not think they are negligible or insignificant (i.e., policymakers should consider and plan for them).

Regarding specific NALs, expanded pharmacy access laws appeared to be the most valuable set of policies—with statewide standing or protocol orders considered particularly “high-value”—as these policies remove the most barriers to naloxone distribution and access. This does not imply these policies are absent implementation barriers, such as failure of pharmacies to stock naloxone and high out-of-pocket costs to purchasers [50]; thus, these policies may operate best when coupled with cost subsidization policies, although these are often expensive for states to implement. In contrast, laws requiring education or training to naloxone recipients were seen as "low-value" policies, both ineffective in improving health-related outcomes and burdensome to implement; this may be increasingly true in contexts where user-friendly naloxone formulations or naloxone training in non-medical settings are readily available [51]. Liability protections were supported but not seen as an effective mechanism for substantial naloxone pharmacy distribution, while co-prescribing laws involve a trade-off between widening eligibility criteria and provoking stigma around questions beyond opioid dosage. Lastly, prescriptions themselves are a barrier to layperson accessibility, as participants believed OTC naloxone provided a better option than third party prescriptions, particularly if accompanied by state subsidies or requirements that insurers cover OTC naloxone costs [52].

The following limitations should be considered when interpreting the study’s results. Our method required participants to have stable Internet access, proficiency with online survey systems, and several hours of availability over the three rounds. While our sample size is commensurate with recommendations for online modified-Delphi processes [28], we did not use random sampling procedures for our stakeholder populations. In addition, our recruited sample is entirely US-based, largely non-Hispanic white, and predominantly consisted of researchers. Lastly, each stakeholder could only participate in one panel (effectiveness or implementability) to reduce burden and attrition across rounds, which may have yielded different comments than stakeholders responding to both effectiveness and implementability criteria.

Conclusions

Experts believe NALs that significantly increase naloxone access are associated with less overdose mortality without risking substantial unintended public health outcomes. To maximize impact, “high-value policies” explicitly counter existing inequities in the healthcare system, address stigmatization of opioid use and naloxone, maintain reasonable prices for purchasing naloxone, and target settings beyond community pharmacies to distribute naloxone.