FormalPara Key Points

Nearly all prescription claims for long-acting injectable antipsychotic medications were linked to a corresponding injection in the medical record.

Dose and dosing interval for second-generation long-acting injectable antipsychotic medications can be accurately estimated from claims data.

1 Introduction

Non-adherence to antipsychotic medication contributes to symptom recurrence and relapse, hospitalization, unemployment, homelessness and criminal victimization and perpetration [1,2,3,4] Long-acting injectable antipsychotics (LAIAs) extend the interval between doses, ensuring sustained exposure to a therapeutic dose and entailing regular contact with health care providers for medication administration [5, 6]. LAIAs were shown to reduce hospitalization, health resource use and mortality compared with oral antipsychotics in real-world settings [7,8,9]. Compared with randomized controlled trials, observational studies may be better situated to measure real-world safety and effectiveness of LAIAs, where their primary advantage over oral antipsychotics is to improve adherence [10].

Estimates of drug dose and dosing interval derived from quantity dispensed and days supplied variables in prescription claims data are widely used [11]. However, concerns have been raised about the validity of dose and dosing interval estimated from prescription claims for LAIAs [12, 13]. In particular, comparison of different methods to correct errors in days supplied values for LAIAs in administrative data led to significant differences in adherence and persistence estimates [12], and days supplied values in prescription claims data were shown to be inconsistent with the labelled dosing interval from product monographs [14]. First-generation long-acting injectable antipsychotics (FG-LAIAs), such as flupentixol, fluphenazine, haloperidol and zuclopenthixol are supplied in ampules or multidose vials where only a portion of the quantity dispensed is administered. In the case of ampules, any remaining medication must be discarded as sterility cannot be maintained. In the case of multidose vials, the remainder may be saved for future injections [13]. While second-generation antipsychotics, such as risperidone, paliperidone, aripiprazole and olanzapine have long been preferred over first-generation antipsychotics, the effectiveness of FG-LAIAs and the high cost of second-generation long-acting injectable antipsychotics (SG-LAIAs) have contributed to continued use of FG-LAIAs [9].

The aims of the present study were to assess the validity of quantity dispensed, dispensed date and days supplied fields in prescription claims data to estimate dose, administration date and dosing interval for LAIAs; and to assess validity of persistence estimates obtained from administrative data.

2 Methods

2.1 Validation Data Set

The Drug Programs Information Network (DPIN) is a comprehensive, population-based record of prescription medications dispensed in Manitoba, Canada, excluding in-hospital pharmaceuticals [15]. DPIN contains de-identified person-level prescription and demographic data and is part of the Manitoba Population Data Repository housed at the Manitoba Centre for Health Policy (MCHP). DPIN data can be linked to other administrative health databases through a scrambled personal health identification number (PHIN). All Manitoba residents are eligible for a provincial Pharmacare program, which covers 100% of drug costs above an income-based deductible [16]. Variables for validation included dispensation date, quantity dispensed and days supplied. To allow for direct comparison of the different drugs studied, we also assessed daily dose measured in defined daily dose (DDD) [17, 18]. Dispensations with a days supplied value of 1 (n = 11) were assumed to be pharmacy-level data entry errors and corrected to the median days supplied value [11, 19].

2.2 Reference Standard Data and Linkage

Medical records from patients attending an outpatient psychiatry clinic in the largest tertiary hospital in Winnipeg, Canada comprised the reference standard. Winnipeg is the capital city of the province of Manitoba, and in the study period, nearly 60% of Manitoba’s population of 1.34 million lived in Winnipeg [20]. Each patient was under the care of a psychiatrist, registered psychiatric nurse and case worker; missed injection appointments were documented in the medical record by the clinic nurse and a follow-up appointment scheduled by the case worker. Adult patients who were dispensed an LAIA from an on-site outpatient pharmacy between April 1, 2015 and March 31, 2016 were eligible for medical record review (n = 264). This time frame was selected because we observed roughly equal numbers of users of first- and second-generation LAIAs in administrative data during this time frame [21]. Eligible records were randomly selected for data extraction until the minimum sample size was achieved. Doses were stratified by type of LAIA used (first-generation LAIA (FG-LAI), risperidone-LAI (R-LAI), paliperidone-LAI (P-LAI)].

We extracted drug name, date of administration, dose and dosing interval for each LAIA dose scheduled during the study period from the medical record. Data were extracted by clinicians specialized in psychiatry (RR) and pharmacy (DJ), and 10% of sampled records were selected for independent two-person data extraction. Subjects who did not attend a scheduled injection were recorded as receiving a 0-mg dose on the scheduled date. Identifying variables for each subject (full name, date of birth, PHIN, sex, study ID number) were recorded separately and submitted to the data provider to create a crosswalk file containing scrambled PHIN and study ID number. The crosswalk file was subsequently transferred to the secure MCHP Repository for analysis. The data extraction form can be viewed in electronic supplementary material (ESM). Ethics and data access approvals were obtained from the Health Research Ethics Board of the University of Manitoba, Health Information Privacy Committee, Winnipeg Regional Health Authority Research Access and Approval Committee, Health Sciences Centre Research Impact Committee and the Manitoba Centre for Health Policy.

2.3 Sample Size Estimation

Minimum sample size was determined using principles for cross-sectional surveys [22], assuming the days supplied value equaled the prescribing dosing interval for at least 15% of FG-LAI [13], 87% of R-LAI [14] and 60% of P-LAI [14]. A minimum of 197 doses of FG-LAI, 174 doses of R-LAI and 369 doses of P-LAI were required. A total of 74 medical records (1233 confirmed doses) were included in the final analysis. Minimum sample size was exceeded for FG-LAI (n = 651) and R-LAI (n = 390) but not for P-LAI (n = 191). However, since both R-LAI and P-LAI are supplied in single-use dosage forms and have fixed manufacturer-recommended dosing intervals, we do not expect findings in the P-LAI stratum to differ significantly from R-LAI [23, 24].

2.4 Data Analysis

2.4.1 Sample Representativeness

DPIN data were linked to hospital discharge abstracts, medical services claims and insurance registry data to obtain demographic characteristics, diagnoses and concomitant medications of patients in the validation sample, and in all Manitoba patients who were dispensed an LAIA during the study period. Income category was determined using the socioeconomic factor index, a summary score that assigns an income category based on the average household income, percent of single-parent households, unemployment rate and high-school education rate within an individual’s dissemination area [25]. Descriptive statistics and standardized differences were used to compare characteristics of patients in the validation sample with all LAIA users in the Manitoba population.

2.4.2 Dispensation Date

Doses were classified as medical record positive (confirmed administered) or negative (scheduled but not administered), and DPIN positive or negative (dispensed or not dispensed within 3 days before injection). Sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV) and kappa were calculated based on the number of dispensations determined to be true positive, false positive, false negative or true positive, using medical record classification as the reference standard (Fig. 1). For dispensations with a confirmed injection within the days supplied field in DPIN, the mean and median time between dispensation and administration was determined.

Fig. 1
figure 1

Mean time (days) between DPIN dispensation date and medical record administration date. CI confidence interval, DPIN Drug Programs Information Network, FG-LAI first-generation long-acting injectable antipsychotics, IQR interquartile range, P-LAI paliperidone long-acting injectable, R-LAI risperidone long-acting injectable

Full size image

2.4.3 Dose and Dosing Interval

Among dispensations with a confirmed injection within the days supplied field, the proportion of dispensations where quantity dispensed and daily dose from DPIN exactly matched administered dose and daily dose, respectively, from the medical record was determined for each drug and stratum. Similarly, the proportion of days supplied values exactly matching the prescribed dosing interval was determined. Mean and median quantity dispensed, daily dose and days supplied from DPIN data were compared with mean and median dose, daily dose and dosing interval, respectively, from the medical record, rounded to the nearest milligram. When the days supplied value was greater than the expected dosing interval, for example prescriptions dispensed in 84 or 90 days supplied, the median days supplied for that drug was used as a proxy.

2.4.4 Persistence

Algorithms were developed and assessed to evaluate persistence estimates from DPIN data using Kaplan–Meier survival analysis. We also used the proportion of patients covered (PPC) method, allowing subjects to re-start treatment after a gap [26]. All LAIA doses dispensed or administered to eligible subjects within the 2015/16 fiscal year were included. Subjects were classified as exposed from the date of the first dispensation (DPIN data) or administration (medical record data) of an LAIA between April 1, 2015 and March 31, 2016. To account for the extended half-life of LAIAs at steady state, a grace window of 90 days, 180 days, 1.5*days supplied (or dosing interval), or 2*days supplied (or dosing interval) was added to the days supplied or dosing interval for each dispensation or dose administered.

2.4.5 Sensitivity Analyses

Several sensitivity analyses were conducted. We varied grace windows between dispensation and administration, evaluating windows of 0, 1, 7, 30 and 90 days. We evaluated a second algorithm that restricted to users with two or more dispensations in a 90-day period between April 1, 2015 and March 31, 2016; subjects were classified as exposed from the date of the second dispensation. We repeated analyses of FG-LAIs after excluding repeat doses administered from a multidose vial. We also repeated all analyses after allowing dispensations to occur a maximum of 1, 3 or 7 days after administration. Finally, we repeated analyses of time between dispensation and administration, dose, daily dose and dosing interval using true positive dispensations ≤0, 1, 3, 7, 30 and 90 days before administration.

3 Results

3.1 Description of LAIA users

We identified 1145 patients with a dispensation for an LAIA during the study period in DPIN records, 74 of whom were included in the validation sample. Compared with the population of LAIA users in the Manitoba population, patients in the validation sample had similar distributions of age, sex and FG- versus SG-LAI use (Table 1). Patients in the validation sample were higher income, had fewer psychiatric comorbidities and used fewer concomitant medications.

Table 1 Comparison of baseline characteristics of long-acting injectable antipsychotic users in the validation sample and Manitoba population
Full size table

3.2 Medical Record Review

We recorded a total of 1232 LAIA doses from medical records, 1133 of which were confirmed administered and 99 of which were scheduled but not administered. Of these, 651 were for FG-LAIs (610 administered, 41 not administered), 390 were for R-LAI (369 administered, 21 not administered) and 191 were for P-LAI (154 administered, 37 not administered).

3.3 Validity Assessment

Internal validity was assessed by random sample of 11 charts (179 doses) selected for independent data extraction by a second investigator, with >96% agreement for all variables (date administered: 96.6% agreement, dosing interval: 97.2% agreement, dose: 97.8% agreement). Reasons for disagreement included dose recorded by only one investigator (n = 4), disagreement in date administered (n = 2) and disagreement between dosing interval (n = 1).

3.4 Dispensation Date

Sensitivity, specificity, PPV, NPV and kappa results with a 3-day grace window are displayed in Table 2. Overall sensitivity was 32.8%, specificity was 68.7%, PPV was 92.3% and NPV was 5.8%. Compared with FG-LAI, R-LAI and P-LAI had higher PPV and sensitivity, but lower specificity. Results from sensitivity analyses are found in ESM Table 1. Specificity was maximized with a 0-day grace window between dispensation and administration, reaching 100% for R-LAI and P-LAI and 92.7% for FG-LAI. Sensitivity was maximized with a 90-day grace window, reaching 99.7% and 98.1% for R-LAI and P-LAI, respectively, and 80% for FG-LAI.

Table 2 Accuracy of long-acting injectable antipsychotic dispensations in 2015/2016, stratified by antipsychotic type, allowing a 3-day grace window between dispensation and administration
Full size table

A total of 652 dispensations were confirmed administered within the days supplied interval (FG-LAIs, n = 158; R-LAI, n = 349; P-LAI, n = 145). The median time between dispensation and administration was 9 days for FG-LAI and 1 day for R-LAI and P-LAI (Fig. 1).

3.5 Dose and Dosing Interval

Quantity dispensed exactly matched administered dose for 99.7% and 100.0% of R-LAI and P-LAI doses, respectively, but only 8.9% of FG-LAI doses. DPIN overestimated dose administered for all FG-LAI drugs, with mean (95% CI) differences ranging from −1550.7 mg (−1694.0 to −1407.4) for zuclopenthixol to −166.9 mg (−89.2 to 244.7) for haloperidol. Daily doses exactly matched for 21.3% of FG-LAI, 97.4% of R-LAI and 89.0% of P-LAI. The mean (95% CI) absolute daily dose differences were −1.0 DDD (−1.1 to −0.9) for FG-LAI, 0.0 DDD (−0.02 to 0.0) for R-LAI and 0.6 DDD (−0.4 to 1.6) for P-LAI (Fig. 2A). The difference in DDD was <1 for all FG-LAIs except fluphenazine (mean difference −2.6 DDD).

Fig. 2
figure 2

(a) Mean absolute difference in defined daily dose (DDD) measured by DPIN versus medical record. (b) Mean absolute difference in dosing interval (days) measured by DPIN versus medical record. CI confidence interval, DPIN Drug Programs Information Network, FG-LAI first-generation LAIAs, LAIA long-acting injectable antipsychotic, SG-LAI second-generation long-acting injectable

Full size image

Days supplied exactly matched prescribed dosing interval for 28.3% of FG-LAI, 97.1% of R-LAI and 76.6% of P-LAI doses. The mean (95% CI) absolute interval differences were −5.3 days (−5.9 to −4.7) for FG-LAI, 0.3 days (0.1 to 0.6) for R-LAI and 0.8 days (−0.2 to 1.7) for P-LAI (Fig. 2B). Among the FG-LAIs, the greatest difference was observed for fluphenazine (−11.8 days), followed by haloperidol (−4.9 days), flupentixol (−4.6 days) and zuclopenthixol (1.6 days).

3.6 Persistence

Kaplan-Meier curves were similar for all algorithms, and log-rank and Wilcoxon tests showed no statistically significant differences (Fig. 3, ESM Fig. 2, ESM Table 1). PPC results also showed that persistence estimates from DPIN dispensations were similar to those from the medical record (Fig. 4, ESM Fig. 3). The algorithms accurately estimated trends in persistence across all strata investigated (ESM Figs. 4–6). However, DPIN-derived estimates of PPC overestimated PPC for FG-LAI and P-LAI and underestimated PPC for R-LAI (ESM Figs. 7–9).

Fig. 3
figure 3

Survival analysis results comparing persistence estimates from medical record versus DPIN claims data. a Maximum 90-day gap. b Maximum 180-day gap. c Maximum gap 1.5 × dosing interval. d Maximum gap 2 × dosing interval. DPIN Drug Programs Information Network

Full size image
Fig. 4
figure 4

Proportion of patients covered by estimates from medical record versus DPIN data. a Maximum 90-day gap. b Maximum 180-day gap. c Maximum gap 1.5 × dosing interval. d Maximum gap 2 × dosing interval. DPIN Drug Programs Information Network

Full size image

4 Discussion

This study establishes the validity of DPIN dispensations as a proxy for long-acting injectable antipsychotic administration, with a PPV of 80% with a 0-day grace window, and >90% with a 7-day window. When stratified by LAIA type, PPV for R-LAI and P-LAI was 100% at 0 days. Among true positive doses, quantity dispensed and days supplied fields slightly underestimated dose and interval for FG-LAIs but provided valid estimates for SG-LAIs. However, this study also found that a large proportion of administered doses could not be linked with a dispensation, and the absence of dispensation in DPIN does not rule out LAIA administration. Possible explanations for this finding include the use of samples or dispensing from an inpatient pharmacy. DPIN-derived estimates of persistence over 1 year were similar to medical record-derived estimates using Kaplan-Meier survival analysis.

A previous validation study comparing Medicaid claims data with medical records showed the days supplied variable in claims data significantly underestimated days supplied compared with the medical record, and only 13.8% of LAIA dispensations had days supplied values that exactly matched the medical record [13]. This study was published in 1999, before the development of SG-LAIs. While the FG-LAI stratum from the present study showed smaller differences in days supplied than reported by Shireman et al. (5.3 days vs 62.6 days), both studies showed underestimation of days supplied in claims data. In the current study, days supplied in DPIN data was much more reliable among the SG-LAI dispensations, which are supplied in single-use vials or pre-filled syringes. Subsequently, prescription claims data may be able to classify SG-LAI exposure with greater certainty than for oral medications. The prolonged exposure to therapeutic blood levels after an LAIA injection eliminates the need to assess or make assumptions about adherence between injections. Even if an injection is delayed, therapeutic drug levels will subside gradually over time; patients who have achieved steady state may sustain therapeutic levels for weeks or even months after their last dose. As a result, the choice of grace window between dispensations had minimal impact on persistence estimates.

DPIN data showed PPV consistently >80% compared against the medical record reference standard, regardless of the specific agent used or selection of grace window between dispensing and administration. We have identified algorithms with a high specificity, which can be applied in future studies of LAIA safety and effectiveness. However, the algorithms that maximized specificity had many false negative doses, and a cohort formed from this data source will not capture all LAIA users. False negatives were particularly prevalent in the FG-LAI stratum. Excluding doses administered from multidose vials reduced the proportion of doses classified as DPIN negative. This led to only a modest reduction in false negatives and slight improvement in true negatives in the shorter grace window algorithms (≤ 30 days). This resulted in reduced specificity but improved sensitivity and NPV, and a positive or neutral effect on PPV; overall the impact was small.

All algorithms performed poorly based on kappa values. This may be explained by the small number of negative doses observed in medical record data, particularly for FG-LAI and R-LAI, resulting in some time windows having no doses classified as false negative or true negative. The kappa statistic may have limited value when the prevalence of one classification category approaches zero, regardless of model specificity and sensitivity [27, 28]. Nevertheless, we have reported kappa values as the low prevalence of negatively classified doses was unexpected, and we planned to calculate kappa values at the design of this study. Post-hoc algorithm assessment using additional performance measures can be found in ESM Table 1.

Generalizability of our findings outside of the validation sample population may be limited. While demographic characteristics of the validation sample were similar to those of the general population of LAIA users, there were some clinical differences. Fewer individuals in the validation sample had comorbid mood or anxiety disorders; concomitant prescriptions for oral antipsychotics, mood stabilizers, antidepressants and sedative-hypnotics were also less prevalent. This may indicate the validation sample patients had more stable disease, or may reflect differences in diagnosis and treatment patterns for patients enrolled in a specialized treatment program versus usual care. Still, the clinic from which subjects were randomly selected for the validation sample comprised almost 25% of all LAIA users in the province of Manitoba in the study period, and LAIA users are closely monitored by care providers regardless of care setting. These findings may be relevant in other populations with comparable demographic and clinical profiles in similar care settings.

5 Conclusions

Prescription claims data are a valuable tool for clinicians conducting medication histories, and an important data source for drug safety and effectiveness studies. We have shown prescription claims data are a valid source of data on positive LAIA exposures, particularly for SG-LAI. Quantity dispensed and days supplied from dispensation data provide approximate estimates of dose and dosing interval for LAIA medications. Persistence estimates using claims data were not significantly different from the medical record.