FormalPara Key Points

The Psoriasis Longitudinal Assessment and Registry (PSOLAR) is a large observational study designed in 2007 to meet postmarketing safety commitments for infliximab and ustekinumab for the treatment of psoriasis.

Key limitations in analysis methodology identified in a comprehensive review of a potential safety risk were the inclusion of patients who used biologics before registry entry, unanticipated imbalances in patient characteristics across cohorts, and limited availability of all relevant clinical data.

Fundamental modifications to predefined analyses confirm no increased risk of major adverse cardiovascular events for ustekinumab, and emphasize the need for periodic assessment of study design and analytical methods to maintain validity in long-term observational studies.

1 Introduction

Long-term treatment with systemic and biologic agents is needed due to the chronic and debilitating nature of psoriasis. Safety profiles for psoriasis therapies derived from clinical trials designed to support regulatory approval are limited by strict inclusion/exclusion criteria and a relatively short duration of treatment and follow-up. To fully understand the benefit–risk profiles of both novel and established therapies, safety surveillance reflecting actual clinical use through observational studies is valuable [1,2,3,4]. Patient registries are especially useful for detecting uncommon safety events that become apparent only in larger populations and for longer periods of time than that evaluated in randomized clinical trials.

The Psoriasis Longitudinal Assessment and Registry (PSOLAR) is a multicenter, prospective, disease-based registry for assessing the long-term safety experience of patients with psoriasis. PSOLAR was designed to detect potential safety signals as part of postmarketing commitments to health authorities for two biologic therapies for psoriasis: infliximab (REMICADE®, Janssen Biotech, Inc., Horsham, PA, USA) and ustekinumab (STELARA®, Janssen Biotech, Inc., Horsham, PA, USA) [5, 6]. The registry was initially launched in June 2007 to evaluate infliximab-treated patients with psoriasis; the first patient receiving ustekinumab for psoriasis was enrolled in March 2009. As of the annual data cut-off in 2019, a total of 12,090 patients had been enrolled in 16 countries and followed for a median of 7.54 years (73,133.6 patient-years [PY] of follow-up).

PSOLAR was designed as a signal-detection registry and, using prespecified methodology based on guidance from health authorities, data are analyzed by the sponsor (Janssen Scientific Affairs, LLC, Horsham, PA, USA) on an annual basis. Results are provided to regulatory agencies as Interval Safety Registry Reports and presented to the PSOLAR Steering Committee composed of dermatologists with expertise in psoriasis. These annual reports focus on cumulative rates of reported adverse events, with specific attention given to adverse events of special interest (such as all-cause mortality, serious infections, malignancy, and major adverse cardiovascular events [MACE]). As the main purpose of the registry is to evaluate safety data, the study protocol stipulates that any safety finding identified using the prespecified statistical methodology should be further evaluated based on appropriate post hoc analyses to determine if the finding represents a true safety risk.

A recent annual review of registry data indicated a meaningful difference in adjusted hazard ratios (HRs) for MACE with ustekinumab treatment versus the non-biologics cohort, indicating a potential risk. Further investigation revealed key methodological limitations that had occurred over the long-term course of the registry. It became apparent that initial design features of the registry may not support adaptation to the evolving treatment landscape or account for changing patient characteristics due to switching therapies. To mitigate these longitudinal weaknesses, a series of post hoc analyses were performed. In this study, we discuss the design of PSOLAR in light of MACE risk by detailing both prespecified and post hoc analytical methods and outcomes. These lessons learned from PSOLAR highlight the importance of re-evaluating methodology over time and may provide insight for other real-world registries.

2 Methods

2.1 Objective and Design of the Study

The design of the PSOLAR registry (ClinicalTrials.gov: NCT00508547) has been reported in detail [5, 6]. Briefly, patients were adults (≥ 18 years of age) who were eligible to receive, or were receiving, any systemic therapy for psoriasis. Each patient was to be followed for 8 years from the date of enrollment, with data collected approximately every 6 months after the initial visit. Patients were assigned to one of four treatment cohorts, including those treated with ustekinumab, other sponsor biologics (i.e. infliximab [Remicade®]), non-sponsor biologics (i.e. other tumor necrosis factor-α inhibitors, interleukin (IL)-17 inhibitors, IL-23 inhibitors), or non-biologics, depending on treatment at registry entry or changes in treatment made during the course of participation in the registry.

2.2 Prespecified Statistical Methodology

The predefined analysis plan for MACE included the calculation of cumulative unadjusted incidence rates per 100 PY across four treatment cohorts and modeled analyses utilizing Cox proportional HR methodology. The biologic cohorts considered in the prespecified analyses for annual safety reports are defined based on the manufacturer of the biologic (i.e. ‘Other Sponsor’ or ‘Non-Sponsor’) compared with ustekinumab and non-biologic agents. Starting in 2019, an expanded search strategy was used to identify MACE in the registry [7,8,9].

Treatment differences in the time to MACE outcomes were estimated using a Cox proportional hazards regression model adjusting for baseline covariates and time-varying effects of biologic and immunomodulator therapies. Adjusted HRs, 95% confidence intervals (CIs), and corresponding p-values based on a Wald Chi-square test were calculated for the ustekinumab cohort compared with the non-biologics cohort. To adjust for confounding related to imbalances between treatments within the Cox model, covariates deemed relevant to the development of MACE or the selection of treatment were included in the multivariable analyses. Demographic and disease characteristics and treatment exposures included in the model were age at baseline, sex, ethnicity, body mass index at baseline, calendar year, duration of disease, Physician’s Global Assessment score at baseline, psoriatic arthritis diagnosis at baseline, history of immunomodulator use, history of biologic use, history of diabetes, history of cardiovascular disease (e.g. angina, atherosclerotic disease, congestive heart failure, coronary artery disease, myocardial infarction, transient ischemic attack/cerebrovascular accident/stroke, hypertension, hyperlipidemia), smoking status, and time-dependent use of ustekinumab, other biologics, and immunomodulators.

Primary time-to-event analyses were performed for all patients ever exposed to ustekinumab, including those who started treatment before, at the time of, or after registry entry (regardless of the timing of the most recent ustekinumab exposure relative to the event). To evaluate potential risk related to more proximal exposure, additional analyses were performed based on counting events only within the 91-day window after ustekinumab exposure.

The cumulative unadjusted incidence rates of MACE in a recent annual data cut (Table 1) were similar to those determined in previous years in all treatment cohorts. However, in the ever-exposed analysis for MACE, the HR showed a meaningful difference for the ustekinumab cohort versus the non-biologics cohort (HR 1.533, 95% CI 1.103–2.131) (Table 2). The secondary analysis of patients experiencing a MACE within 91 days of the previous exposure also yielded an elevated HR (but to a lesser extent and not statistically significant) for the comparison between the ustekinumab and non-biologics cohorts (HR 1.193, 95% CI 0.917–1.553).

Table 1 Cumulative unadjusted incidence rates of MACE across treatment cohorts; ever-exposed and 91-day-exposed populations
Full size table
Table 2 Cox proportional hazard regression models for MACE
Full size table

2.3 Post Hoc Analyses

Given the signal-detection design of the registry, a cross-functional team of clinical experts representing cardiology, epidemiology, and biostatistics conducted a comprehensive assessment of the methodologies for analysis of MACE risk; the review identified several methodological limitations. Treatment paradigms for psoriasis have undergone significant shifts with the introduction of new therapies; at the initial design stage of this long-term registry (i.e. up to 8 years of patient follow-up), such changes could not have been anticipated. In some cases, patient characteristics diverged between treatment cohorts over time as treatments switched (and consequently cohorts changed), otherwise known as population drift. The following were considered pertinent to influencing the outcomes of MACE and potentially other adverse events of special interest in the prespecified analyses: (1) inclusion of prevalent users of biologics (those with ongoing use of biologics with initiation of treatment pre-dating registry entry) in addition to incident users (those who started treatment at enrollment or during registry participation); (2) unanticipated imbalances in patient characteristics between treatment cohorts at baseline resulting from the non-randomized, observational nature of the registry; (3) limited availability of relevant clinical data after registry entry; and (4) divergence of characteristics associated with outcomes among the comparator groups over time on registry.

  1. (1)

    Prevalent versus incident users

The registry includes both prevalent and incident users of biologics. While inclusion of prevalent biologic users (i.e. those who received treatment at any time, even before enrollment) in the registry allowed for efficient and timely recruitment and was necessary to achieve a sample size required by regulatory agencies, it introduced bias into the prespecified analyses as there were no true disease severity data at baseline to assess pretreatment (i.e. prebiologic) status and patient characteristics. Because two-thirds (66%) of patients in the ustekinumab cohort are prevalent users, our analysis could not adequately measure disease severity between treatment groups at baseline. As biologic treatment for psoriasis typically follows a stepwise paradigm with a biologic introduced after ineffective non-biologic therapy, we hypothesized that ustekinumab was used to treat more severe disease that could be associated with increased risk for MACE [10, 11].

Analyses focusing on incident users, i.e. those who started biologic treatment on-registry, provide more pertinent information than those using both prevalent and incident users. The former ensures that baseline data collected at the time of treatment initiation are accurate and that all time on treatment is captured in the analysis [12, 13]. Balance and consistency between cohorts is more likely to be achieved when enrollment occurs at inception of treatment (incident use). Furthermore, analyses from observational studies that included only incident users reportedly produce results that are more consistent with findings from randomized trials compared with those that included both prevalent and incident users [14].

  1. (2)

    Imbalances between treatment cohorts

Due to the non-randomized nature of the registry, baseline differences in several demographic and disease characteristics are expected between the treatment cohorts (Table 3). As some baseline features that differed are also risk factors for MACE (e.g. age, obesity), it was necessary to conduct adjusted HR modeling analyses to estimate risk. To assess the robustness of these analyses and further discern residual imbalances between the ustekinumab cohort and the prespecified non-biologics comparator cohort that could bias the resulting HR estimate, additional methods of covariate adjustment based on epidemiologic principles were considered.

Table 3 Baseline characteristics across treatment cohorts; prevalent users ever-exposed to treatment
Full size table

Propensity scoring is an accepted method to control for measured confounding in non-interventional studies. The propensity score summarizes all information from explanatory variables, such as disease severity and comorbidities, and estimates the probability of a patient receiving the intervention of interest (i.e. ustekinumab) [15]. Individual patients may have the same or similar propensity score, yet only some will have received the intervention of interest. If baseline risk factors and disease characteristics are comparable, the estimated propensity score distributions would appear to be similar for all patients irrespective of treatment (as would be expected in a 1:1 randomized clinical trial). The propensity score is often estimated for each patient from a logistic regression model in which treatment assignment is the outcome variable.

Different propensity score implementation methods may be used to model the estimation of effect, including inverse probability of treatment weighting (IPTW). The IPTW approach uses the propensity score to weight each treatment group to have a distribution of covariates similar to that of the overall study population, which balances these characteristics in each treatment group. While propensity scores are generally useful for controlling confounding, they can sometimes introduce bias (e.g. if the propensity score model includes incorrect variables). It is also important to consider ‘trimming’ both ends of the propensity score distribution to eliminate patients who are treated contrary to prediction, as this may reduce the impact of bias due to unmeasured confounding caused by frailty [15]. Critical inspection of the distribution of weights is essential, particularly for assessing the impact of excessively large weights that could have an unduly high influence on results [16, 17].

In PSOLAR, propensity scores were estimated to generate IPTW balancing in the ustekinumab and non-biologics cohorts using the same baseline risk factors that were prespecified for the primary Cox regression model. While neither approach showed statistical significance, the IPTW approach gave an HR closer to the null value (1.176, 95% CI 0.929–1.490), compared with that calculated without IPTW (propensity-score adjusted). However, the lack of overlap between score distributions on the propensity score histogram indicated that there was residual covariate imbalance between patients receiving ustekinumab and those receiving non-biologic treatments (Fig. 1). This indicated that an active comparator for ustekinumab may be more appropriate to address this limitation when evaluating certain adverse events of interest, such as MACE.

  1. (3)

    Limited availability of relevant clinical data after registry entry

Fig. 1
figure 1

Histogram of propensity scores for the ustekinumab and non-biologics cohorts; prevalent and incident users in the ever-exposed population (prespecified analyses)

Full size image

Understanding how treatment cohorts change over time and adjusting for these differences are essential for ensuring unbiased and valid results. While PSOLAR provides a robust dataset regarding clinical characteristics of patients with psoriasis collected at baseline and over time, some health-related parameters that may be relevant to cardiovascular outcomes are not captured after enrollment. For example, the occurrence of new or worsened comorbidities is not collected beyond baseline. In addition, use of concomitant non-psoriasis medications (e.g. statins or aspirin) is not collected at any time during the registry. The collection of these parameters over time was not implemented to reduce operational burden associated with data capture at each study site. Therefore, the lack of relevant data after baseline is a study design issue that cannot be mitigated by modifying analytical methods.

  1. (4)

    Divergent comparator groups over time on the registry

Another consideration related to the evolution of the registry over time is the difference in the duration of registry participation or treatment period between cohorts. For example, the time to discontinuation or treatment switch was longer for the ustekinumab cohort compared with the non-biologics cohort (median 8 vs. 4 years) (Fig. 2). While the Cox model regression analysis accounts for differing time on registry, it does not account for potentially important differences in how the clinical condition of patients may evolve over time, especially at the time of registry discontinuation or treatment switch. In PSOLAR, patients who began treatment with a non-biologic may have stepped up to more effective treatment with a biologic agent, whereas patients treated with a biologic were not likely to switch to a non-biologic. This is in keeping with ‘healthy adherer’ bias, wherein patients who remain in a study and adhere to an existing therapy are likely to be healthier than those who discontinue or switch therapies [13]. Therefore, the comparison between the ustekinumab and non-biologics cohorts may be further biased by the retention of healthier patients in the non-biologics cohort or those who have limited access, which may reduce the utility of the non-biologics cohort for the primary comparison and support the use of an active comparator in the analysis. Furthermore, differences between treatment cohorts at baseline are likely to have diverged further over time, particularly in light of the long-term nature of the registry.

Fig. 2
figure 2

Kaplan–Meier analysis of time to discontinuation of treatment switch in the ustekinumab and non-biologics cohorts; ever-exposed population

Full size image

3 Results

To address the identified methodological flaws, the prespecified methodology was modified in several ways in post hoc analyses (Table 4). First, to align observation of the treatment period, only incident users were included in the multivariable Cox model regression analysis for MACE. In addition, the model was weighted with propensity scores to address residual imbalances in baseline characteristics between treatment cohorts beyond the prespecified HR model. Finally, due to the disparities identified both at baseline and over the course of the registry, a comparator other than non-biologics was deemed more clinically appropriate for the evaluation of potential MACE risk associated with ustekinumab.

Table 4 Methodological limitations in prespecified analyses and analytical solutions based on a post hoc approach
Full size table

Review of the most important considerations for choosing a potential comparator indicated that adalimumab (Humira®, AbbVie Inc., Chicago, IL, USA) was the most appropriate candidate for comparison with ustekinumab. The adalimumab cohort included the most patients (and the most PY of follow-up) in PSOLAR after the ustekinumab cohort; therefore, using the adalimumab cohort provided sufficient statistical power for subsequent analyses. Furthermore, clinical characteristics at baseline were generally comparable between patients receiving ustekinumab and those receiving adalimumab in the registry (Table 5). Furthermore, ustekinumab and adalimumab were approved by regulatory authorities within a similar timeframe in the US (25 September 2009 and 22 January 2008, respectively), and the clinical efficacy profiles from their psoriasis registrational trials are similar [18, 19]. Finally, adalimumab has no known risk of MACE [19].

Table 5 Baseline characteristics by treatment; incident users ever exposed to treatment
Full size table

In the post hoc analysis, the revised histogram following propensity scoring for incident users demonstrated smaller differences between the ustekinumab and adalimumab cohorts compared with those between the ustekinumab and non-biologics cohorts, indicating that the ustekinumab and adalimumab cohorts are more comparable (Fig. 3). Using this more robust analysis for the ever-exposed population, the updated HR was 0.820 (95% CI 0.532–1.265), indicating no meaningful difference in the risk of MACE for ustekinumab- vs. adalimumab-treated patients with psoriasis. When the analysis was based on incidence of MACE within 91 days of the previous exposure, similar results were observed (HR 0.592, 95% CI 0.366–0.956) (Table 2).

Fig. 3
figure 3

Histogram of propensity scores for the ustekinumab and adalimumab cohorts based on revised analyses; incident users in the ever-exposed population (post hoc analyses)

Full size image

4 Discussion

An evaluation of the analytical methodologies of the long-term ongoing PSOLAR program was triggered by a potential safety signal for MACE in ustekinumab-treated patients compared with non-biologics (HR 1.533, 95% CI 1.103–2.131). Following the application of more methodologically appropriate analyses, no meaningful difference in the risk of MACE with ustekinumab was observed (HR 0.820, 95% CI 0.532–1.265). Our comprehensive review highlighted important limitations that should be considered when designing studies and interpreting results from other long-term patient registries. Notable findings included the challenge of evaluating patients starting treatment prior to enrollment (prevalent users), the potential impact of baseline imbalances between treatment cohorts, the limited availability of some relevant data after enrollment, and the divergence in the health status of the comparator groups as the registry matured. Additionally, the longitudinal nature of PSOLAR (>10 years) could not account for the evolving availability of new treatment options that contributed to the opportunity for switching therapy. Post hoc analyses were performed to mitigate the issues around prevalent user data, imbalances between cohorts, evolving standards of care over time, survival bias, and unequal discontinuation rates. Although these limitations are presented in the context of the MACE analyses, the lessons learned may apply to all safety assessments in PSOLAR and shed light on analytical approaches for future disease-based registries and observational studies.

Our results confirm the power of PSOLAR to evaluate safety events and document an effective evaluation of a potential signal. Although the prespecified analyses showed a potential risk of MACE for ustekinumab compared with non-biologics during registry follow-up in the ever-exposed population, critical review and appropriate modification of the original analytical procedures did not confirm this as a true risk. This example highlights that careful consideration of the dynamic nature of registries is necessary to generate clinically meaningful data.

Some of the limitations described here are inherent in the design of all observational studies, such as the lack of randomization resulting in substantial imbalances between treatment cohorts at baseline and the potential impact on long-term outcomes. Many are commonly encountered in observational studies (e.g. patient participation and treatment selection bias, inability to adjust for potentially relevant covariables, and the complex effects of switching treatments during enrollment) and have been documented in reports of PSOLAR and other psoriasis registries [20,21,22,23,24,25,26,27,28,29,30]. Careful planning of study design, particularly data collection and statistical methodology, during protocol development, can abate many of these issues.

5 Conclusion

Our findings confirm that, although they are critically important in the overall evaluation of pharmacologic products, observational studies require ongoing evaluation and possible adaptation to ensure that study design and analytical methods continue to be appropriate in order to yield valid results through the lifecycle of the study.