FormalPara Key Summary Points

Why carry out this study?

Misuse and abuse of prescription opioids have increased.

Assessing the risk of opioid abuse and misuse is crucial for prevention.

No study is available on the validity of the Spanish versions of the Opioid Risk Tool (ORT) and the Screener and Opioid Assessment for Patients with Pain—Revised (SOAPP-R).

What was learned from the study?

The ORT showed close to acceptable diagnostic capacity and poor predictive capacity.

The SOAPP-R showed excellent diagnostic capacity, acceptable predictive capacity regarding misuse, and poor predictive capacity regarding abuse.

Introduction

Opioids are frequently prescribed for many chronic pain conditions. In the USA and some European Union countries, there has been a dramatic increase in the misuse and abuse of prescription opioids [1]. There is well-documented evidence on the adverse consequences of opioid abuse [2,3,4,5,6], including increased mortality due to unintentional overdosing and cardiorespiratory problems [2, 7, 8]. A recent report showed that the opioid crisis is increasing within Hispanic/Latino communities in the USA and that the language barrier hinders their access to adequate care [9]. In these communities, treatment alternatives are often scarce [10], and the monitoring of opioid misuse and abuse is typically not conducted [11]. One reason for the latter situation is that the appropriate instruments have not been adapted to Spanish-speaking populations.

There is general agreement on the necessity to assess the risk of opioid misuse and abuse in patients with noncancer chronic pain before initiating treatment [3]. Assessment before prescription can help tailor treatments to the patients' needs and characteristics and minimize the risk of opioid misuse and abuse [12]. Several measures have been created to assess the risk of developing aberrant behavior in the use of prescribed opioids for noncancer chronic pain conditions. There are Spanish translations of the Opioid Risk Tool (ORT) [13] and the Screener and Opioid Assessment for Patients with Pain–Revised (SOAPP-R) [14, 15]; however, there is no empirical evidence on their capacity to detect and predict opioid misuse or abuse.

Both instruments rely on the general assumption that the more aberrant the behavior of the individuals, the more likely the individuals are misusing or abusing opioids or will do so in the future [13, 14]. Substance misuse was defined as using a drug in a way that differs from the prescription, and substance abuse was defined as use that is detrimental to the user or others or is illegal [13, 14].

The ORT included the following risk factors: a personal and family history of substance abuse; age between 16 and 45 years old; history of preadolescent sexual abuse; and certain psychological disorders [13]. The results on the capacity of the ORT to predict aberrant drug-related behavior are mixed, ranging from acceptable to no discrimination [16,17,18,19,20,21,22]. Several authors have suggested that some studies had follow-up periods shorter than 1 year, which could explain these inconsistent results [18], given that the duration of the follow-up period should be at least 1 year. This requirement was fulfilled in the initial validation study of the instrument [13]. Another factor underlying the aforementioned contradictory results could be social desirability bias fostered by the explicit nature of the items of the ORT. Thus, some patients can easily manipulate their answers to appear to be at lower risk than is actually the case [21, 22]. Indeed, a study demonstrated that the way in which the ORT was administered made a significant difference to the results because aberrant drug-taking was better predicted by the clinician-completed ORT than by the patient-completed ORT [21]. The authors suggested that the discrepancies were mainly due to comprehension issues [21].

To try to remedy this shortcoming, the SOAPP included subtle items that are not obviously related to aberrant drug behavior (e.g., feeling bored, impatient, angry). A panel of pain and addiction experts identified eight conceptual clusters of risk factors for potential problems with opioids in people considered for opioid therapy: antisocial behavior/history, substance abuse history, medication-related behavior, doctor–patient relationship factors, psychiatric history, emotional attachment to pain medications, personal care, lifestyle issues, and psychosocial problems. The SOAPP comprised items representing each of the eight identified concepts [23]. The SOAPP-R was the outcome of later refinements of this initial conceptual framework and subsequent empirical studies to select the items that were the best predictors of medication misuse [14]. Results on the diagnostic and predictive capacity of the SOAPP-R are inconsistent [16, 18,19,20, 22, 24].

Methods

Study Aim

The aim of the study was to provide preliminary evidence of the diagnostic and predictive capacity of the Spanish translations of the ORT and the SOAPP-R in a sample of people with chronic pain, given that there is no empirical evidence on their capacity to detect and predict opioid misuse or abuse. To overcome the shortcomings of the aforementioned research, in this study, clinicians orally administered all the instruments to control for social desirability bias and avoid comprehension issues. We also included a follow-up period of more than 1 year (18 months).

Study Design

We used the Current Opioid Misuse Measure (COMM) [25] as a criterion measure to test the capacity of the ORT and the SOAPP-R to identify patients who were misusing opioids at the time of the assessment. Eighteen months later, we used the COMM [25] and the Drug Abuse Screening Test (DAST-10) [26, 27] to test their predictive capacity in a subsample of patients.

Study Setting

Participants were recruited through two local associations of people with fibromyalgia and two pain units.

Inclusion Criteria

The inclusion criteria were as follows: at the time of the study, participants were experiencing pain and had been experiencing pain for at least the last 3 months; they were over 18 years old; they were not being treated for a malignancy, terminal illness, or psychiatric disorder; they had been under opioid treatment for more than 90 days [28]; and they were able to understand Spanish, the instructions, and the questionnaires.

Participants

We tested the capacity of the ORT and the SOAPP-R to classify patients according to opioid misuse using a convenience sample of 147 individuals with noncancer chronic pain; 18 months later, 42 of them completed the second assessment. Tables 1, 2, and 3 show the descriptive statistics of the demographic and clinical variables. The daily dose of opioids was calculated and converted to oral morphine milligram equivalents (MME) using recommended conversion factors [29]. The median MME per day was “moderate” (51–89 MME/d) [29] (Table 2) in the initial sample and “low” (Table 3) in the subsample. Opioids and benzodiazepines were simultaneously consumed by 37.41% of participants in the initial sample and 26.19% in the subsample.

Table 1 Description of the participants
Full size table
Table 2 Means, standard deviations, and correlations between variables, n = 147
Full size table
Table 3 Means, standard deviations, and correlations between variables, n = 42
Full size table

Data Collection Tools

Pain Index

Participants were asked to rate their least, average, and worst pain during the past 2 weeks and their current pain on an 11-point Likert scale. The mean of these ratings was calculated to obtain a composite pain intensity score [30].

Opioid Risk Tool (ORT)

The ORT [13] is a 10-item instrument used to predict the risk of engaging in aberrant drug-related behavior in patients with chronic pain receiving prescribed opioid therapy. Respondents are questioned on each risk factor and their answers are weighted from 1 to 5 depending on the item. Previous studies on the capacity of the ORT to predict aberrant drug-related behaviors reported area under the curve (AUC) values that ranged from 0.358 to 0.735, sensitivity values that ranged from 0.20 to 0.75, and specificities that ranged from 0.54 to 0.88 [16,17,18,19,20,21,22]. One study [17] found that, after excluding the item related to a history of preadolescent sexual abuse, the unweighted version of the ORT was superior to the original ORT in detecting patients with and without opioid use disorder. Thus, we computed four scores: two weighted scores (one with and one without the item related to a history of preadolescent sexual abuse) and two unweighted scores (one with and one without this item). We used the Spanish translation of the questionnaire (Webster & Webster, https://www.lynnwebstermd.com/opioid-risk-tool/).

Screener and Opioid Assessment for Patients with Pain—Revised (SOAPP-R)

The SOAPP-R is a 24-item questionnaire used to identify a patient’s risk of abnormal drug-related behavior [14]. It is scored on a scale from 0 to 4. Previous studies have obtained a great range of values regarding the sensitivity and specificity of the SOAPP-R, ranging from 0.91 to 0.54 and from 0.39 to 0.71, respectively [16, 18,19,20, 22, 24]. We used the Spanish translation of the questionnaire published by its authors [15].

Current Opioid Misuse Measure (COMM)

This instrument is used to monitor chronic pain patients receiving opioid therapy who may be manifesting behavior suggestive of substance abuse [25, 31]. The COMM comprises 17 items rated on a scale from zero to four. A total score of nine or more indicates positive opioid misuse. The Spanish adaptation showed high internal consistency (α = 0.80), test–retest reliability (ICC 0.97; 95% CI 0.94–0.99), and adequate internal, criterion, and convergent validity [32].

Drug Abuse Screening Test (DAST-10)

The DAST-10 is designed to identify problems related to drug abuse during the past year [26]. Using DSM-IV TR as a criterion measure and a cutoff point of ≥ 3, the Spanish version has been shown to correctly classify 95.36% of participants [27].

Data Collection Procedure

Demographic and clinical data were obtained via semi-structured interviews with a psychologist who also administered the ORT, SOAPP-R, and COMM. Data were collected between October 2018 and January 2020. In December 2020, participants in the initial sample who had been assessed 18 months before were contacted and assessed again. At this time point, they were interviewed regarding medication intake and pain intensity, and the COMM and DAST-10 were administered.

Ethical Issues

All the procedures were conducted in accordance with the Helsinki Declaration of 1964 and its later amendments. The project of which this study is part received ethical clearance from the Institutional Ethics Review Board (reference: CEUMA 66-2019-H). Participants provided a signed informed consent and confidentiality was maintained at every stage of the study.

Statistical Analyses

Data were analyzed using SPSS 22 (Statistical Package for the Social Sciences; Chicago, USA). We calculated means, standard deviations, and Pearson correlations. We also performed t tests to determine if there were significant associations between the sex of the participants and the mean total scores on the ORT, SOAPP-R, COMM, and DAST-10. The guidelines proposed by Cohen [33] were used to assess the size of correlations. Receiver operating characteristic (ROC) curve analysis was used to calculate the AUC (c-statistic) [34, 35]. Values of c equal to 0.50 indicate no discrimination, values between 0.70 and 0.80 are considered acceptable, values greater than 0.80 but less than 0.90 indicate excellent discrimination, and values greater than 0.90 indicate outstanding discrimination [34, 35]. ROC analysis also provides estimations of sensitivity and specificity. Sensitivity is the proportion of true positives (i.e., people abusing or misusing opioids) that are correctly identified, and specificity is the proportion of true negatives (i.e., people who are not abusing or misusing opioids) that are correctly identified. We used MedCalc v.9.5.2.0 software to determine the optimal cutoff points and the sample size.

Sample Size Calculation

MedCalc v.9.5.2.0 indicated that, for an AUC of 0.80, a sample size of 30 participants would indicate that the analysis had high power (0.80) to reject the null hypothesis (value of c = 0.50, meaning no discrimination) at the 0.05 significance level. For an AUC of 0.70, a sample size of 72 participants would indicate that the analysis had high power (0.80) to reject the null hypothesis (value of c = 0.50, meaning no discrimination) at the 0.05 significance level.

Results

Descriptive Analyses

No associations were found between the sex of the participants and the mean total scores on the ORT, SOAPP-R, COMM, and DAST-10. Tables 2 and 3 show the means and standard deviations of the continuous variables and their correlations. As expected, strong correlations were found between the four total scores of the ORT. In the initial sample, weak to moderate positive correlations were found between scores on the ORT and the SOAPP-R. In the follow-up sample, moderate to strong positive correlations were found between scores on the ORT and the SOAPP-R.

In the initial sample, a strong positive correlation was found between scores on the COMM and the SOAPP-R, whereas a weak to moderate positive correlation was found between scores on the COMM and the ORT.

In the follow-up sample, weak correlations were found between scores on the ORT, the SOAPP-R and the COMM. A weak correlation was found between scores on the DAST-10 and the ORT, and a moderate positive correlation was found between scores on the DAST-10 and the SOAPP-R. A strong correlation was found between scores on the DAST-10 and the COMM. In both the initial and follow-up samples, a negative correlation was found between the age of the participants and all measures of risk, misuse, and abuse.

Capacity of the ORT and the SOAPP-R to Identify Patients Misusing Opioids Using the COMM as the Criterion Measure

According to the COMM cutoff score, 119 participants (80.95%) in the initial sample were misusing opioids. The ROC analysis showed that the SOAPP-R had an excellent capacity to identify participants who were misusing opioids at the time of assessment (Table 4). Regarding the ORT, although the AUC values were statistically significant, they can only be considered “almost acceptable” [34, 35]. Tables 5 and 6 show the sensitivity and specificity values for scores on the ORT and the SOAPP-R, respectively, and Fig. 1 shows the associated ROC curve. In the case of the ORT, we present the ROC curve and the coordinates for the unweighted scoring excluding the item related to sexual abuse because it is the one with the highest AUC value. Table 5 shows that for a score equal to or greater than 0.50, sensitivity was high (0.874) and specificity was low (0.357), whereas for a score equal to or greater than 1.50, sensitivity was considerably lower (0.454) and specificity was higher (0.786). Given that the ORT is a screening tool, we chose a cutoff point of 1 to reduce the possibility of failing to identify high-risk patients. For a score of 1, the proportion of people misusing opioids who were correctly identified was 87.39% and the proportion of people not misusing opioids who were correctly identified was 35.71%. The positive predictive value was 85.25%, the negative predictive value was 40%, and the positive likelihood ratio was 1.36.

Table 4 ROC analysis
Full size table
Table 5 Sensitivity and specificity values of the total scores on the ORT (unweighted scores excluding the item related to sexual abuse) for detecting opioid misuse (criterion: COMM), n = 147
Full size table
Table 6 Sensitivity and specificity values of the total scores of the SOAPP-R for detecting opioid misuse (criterion COMM), n = 147
Full size table
Fig. 1
figure 1

Receiver operating characteristic curves comparing ORT (unweighted scores excluding the item related to sexual abuse) and SOAPP-R to detect opioid misuse (criterion measure: COMM). n = 147. ORT Opioid Risk Tool, SOAPP-R Screener and Opioid Assessment for Patients with Pain—Revised, COMM Current Opioid Misuse Measure

Full size image

Regarding the SOAPP-R, Table 6 shows that values between 21 and 24 showed high sensitivity values and moderate specificity values. Thus, a cutoff point of 21 or 22 would be appropriate, as shown by the sensitivity values, specificity values, positive predictive values, negative predictive values, positive likelihood ratios, and negative likelihood ratios for these scores (Table 7).

Table 7 Assessment of the SOAPP-R cutoff points
Full size table

Capacity of the ORT and the SOAPP-R to Predict Opioid Misuse (COMM) and Abuse (DAST-10)

According to the COMM cutoff score, 28 participants (59.57%) in the follow-up sample were misusing opioids, and according to the DAST-10 cutoff score of 3, 32 participants (76.19%) in the follow-up sample were abusing opioids.

None of the AUC values were significant (Table 4, Figs. 2, 3). Regarding the ORT, the AUC values indicated poor predictive capacity. In the case of the SOAPP-R, the AUC value was “almost acceptable” in relation to the COMM cutoff score and poor regarding the DAST-10 cutoff score. Tables 8 and 9 show the sensitivity and specificity values of the total scores of the ORT and the SOAPP-R for predicting opioid misuse (COMM) and abuse (DAST-10). In the case of the ORT, we used the weighted score excluding the item related to sexual abuse because it was the score with the highest AUC value.

Fig. 2
figure 2

Receiver operating characteristic curves comparing ORT (weighted score excluding item related to sexual abuse) and SOAPP-R to predict opioid misuse (criterion measure: COMM). n = 42. ORT Opioid Risk Tool, SOAPP-R Screener and Opioid Assessment for Patients with Pain—Revised, COMM Current Opioid Misuse Measure

Full size image
Fig. 3
figure 3

Receiver operating characteristic curves comparing ORT (weighted score excluding item related to sexual abuse) and SOAPP-R to predict opioid abuse (criterion measure: DAST-10). n = 42. ORT Opioid Risk Tool, SOAPP-R Screener and Opioid Assessment for Patients with Pain—Revised, DAST-10 Drug Abuse Screening Test

Full size image
Table 8 Sensitivity and specificity values of the total scores on the ORT (weighted scores excluding item related to sexual abuse) and the SOAPP-R for predicting opioid misuse (criterion COMM), n = 47
Full size table
Table 9 Sensitivity and specificity values of the total scores on the ORT (weighted scores excluding item related to sexual abuse) and the SOAPP-R for predicting opioid abuse (criterion DAST-10), n = 47
Full size table

In the case of the ORT, a cutoff point of 1 should be used with the COMM and DAST-10 because there is a marked decrease in sensitivity at higher values (Tables 8, 9). Regarding the SOAPP-R, a cutoff point of 21 should be used with the COMM (Table 8). Using this cutoff point, sensitivity was 85.71%, specificity was 21.43%, the positive predictive value was 68.57%, the negative predictive value was 42.86%, the positive likelihood ratio was 1.09, and the negative likelihood ratio was 0.67. In relation to the DAST-10, the cutoff of the SOAPP-R was not computed because the AUC value was very small (0.423) (Table 4).

Discussion

Firstly, it is noteworthy that the results of the COMM showed that large percentages of the participants in the initial and follow-up samples (80.98% and 59.57%, respectively) were misusing opioids. The results of the DAST showed that 76.19% of the participants in the follow-up sample were abusing opioids. These results highlight the extent of the phenomenon and agree with those of previous research showing that there has been a dramatic increase in the misuse and abuse of prescription opioids [1].

This study showed that 37.41% of participants in the initial sample and 26.19% of those in the follow-up sample received simultaneous prescriptions for opioids and benzodiazepines. These results agree with those of previous research showing that the simultaneous prescription of opioids and benzodiazepines is increasing, especially among patients receiving opioid treatment for more than 90 days [36]. This was the case for the participants in this study. This finding is particularly worrisome because previous research has shown that the risk of accidental death by overdose and cardiorespiratory problems increases when opioids and benzodiazepines are prescribed together [1, 7, 37].

As measured with the SOAPP-R, the significant positive association found between pain intensity and the risk of opioid misuse was low to moderate (0.24, initial sample; 0.39, follow-up sample). This finding agrees with those of previous research [5, 12] showing that people who report higher pain intensity may be at a higher risk of developing aberrant behavior in the use of prescribed opioids in an attempt to obtain analgesic effects.

In this study, a negative association was found between older age and the risk of misuse and abuse and abuse/misuse behavior: this relationship was of a higher magnitude in the initial sample. In fact, one of the items of the ORT includes being aged between 16 and 45 years as a risk factor. Although this is a frequent finding [38,39,40,41,42], a systematic review and meta-analysis concluded that further research should address this issue more deeply, given that most of the previous studies were short-term ones and excluded persons with a history of substance abuse, which is a recognized risk factor for opioid abuse [43].

The correlational analyses showed moderate and moderate-to-high positive correlations between the ORT and the SOAPP-R in both samples, suggesting that although these tools are related, there is no overlap between them. Previous studies have not reported on correlations between the scores of both instruments.

The aim of the present study was to provide preliminary evidence of the diagnostic and predictive capacity of the Spanish translation of the ORT and the SOAPP-R in a sample of people with chronic pain. Values of the area under the curve and sensitivity, specificity, and predictive values showed that the discriminant capacity of the ORT was not acceptable in the diagnostic or predictive study in relation to misuse and abuse. These findings agreed with those of previous studies showing that the diagnostic and predictive capacity of the ORT was not adequate [18,19,20,21,22]. It is also remarkable that, according to the analyses, a score of just 1 on the ORT (i.e., the presence of a single risk factor) was indicative of the patient being at risk of developing aberrant behavior when prescribed opioids.

Values of the area under the curve and sensitivity, specificity, and predictive values showed that, regarding misuse, the Spanish version of the SOAPP-R had high diagnostic efficiency and adequately classified 83% of the participants. Note that to determine the cutoff point, we prioritized sensitivity over specificity to reduce false negatives because of their risk to the patients' health and quality of life. These results agree with those of previous studies in which the SOAPP-R showed a sensitivity of 0.81 and a specificity of 0.68 for detecting aberrant medication-related behavior [14] and excellent discrimination between high- and low-risk patients [44, 45]. Other studies have also shown that a high SOAPP-R score is associated with using multiple providers for controlled substance prescriptions [24] and with an increased likelihood of drug abuse [3].

Conversely, the capacity of the SOAPP-R to predict opioid misuse and abuse was limited because it only correctly classified 67% and 42% of the participants, respectively, in the follow-up study. The predictive capacity of the SOAPP-R may be limited by the inclusion of items reflecting problematic behavior that is not necessarily associated with opioid misuse or abuse, but which are associated with the condition of experiencing chronic pain. For example, the items related to mood swings, feeling bored, tension at home, or a difficult relationship with doctors are common issues in people with chronic pain.

The generalizability of the results of this study may be limited due to the sample sizes and the overrepresentation of women. Among the limitations of the present study, opioid misuse and abuse were measured using self-report instruments. Future research on the validity of the ORT and the SOAPP-R should use other methods. Another limitation is that the participants' responses to the two questionnaires may have been affected by social desirability bias [2, 21]. Although social desirability bias decreases when these two instruments are heteroadministered [21], as in the present study, future research should measure and control for its possible influence. The results may have also been influenced by the interaction of social desirability and the age of the participants, which have been shown to have a positive association [46,47,48,49]. In this study, the average age of the participants was around 60 years, which could be associated with higher social desirability. Future research is needed to investigate whether social desirability is a mediator of the relationship between age, self-reported risk factors, and opioid abuse or misuse.

Despite the preliminary nature of this study and the methodological limitations that may have biased the results, we suggest that clinicians should exercise caution when using the Spanish versions of the ORT and the SOAPP-R to help make decisions on opioid prescription. We need good-quality evidence on risk factors to develop accurate instruments for detecting people at risk of prescription opioid abuse and misuse [50]. Several recent models have postulated a reciprocal interaction between the psychological factors that contribute to the development of substance abuse and the psychological factors that contribute to adaptation to chronic pain [51, 52]. Future research could include these risk factors in instruments such as the ORT and SOAPP-R in order to improve their capacity to detect this type of misuse and abuse in such patients. The detection and prevention of opioid misuse and abuse is and will always be an essential part of good health care [53,54,55]. On the basis of social equality, this type of intervention must be made available to Hispanic/Latino communities wherever they form underserved minority populations. The adaptation of assessment instruments into Spanish would represent a step forward in this direction [56].

Conclusion

Further research is needed on the diagnostic and predictive capacity of the Spanish versions of Opioid Risk Tool and the Screener and Opioid Assessment for Patients with Pain—Revised. When using these instruments to make decisions on opioid administration, clinicians should rely on additional information on the psychological factors that contribute to adaptation to chronic pain.