Background

An analysis of observational data can complement a randomized trial by addressing questions that could not be answered by the trial. For example, observational analyses may sometimes be used to extend inferences made in the trial to a longer follow-up, or to populations not included or underrepresented in the trial. Attempts at extending inferences from a trial can be supported by benchmarking, that is, a systematic comparison of an observational analysis and a trial that aim to ask the same question [1,2,3]. To increase confidence in the benchmarking process, it is preferable to complete the observational analysis before the trial findings are known. Investigators can then be confident that data have not been manipulated until they produce estimates that agree with the trial. Also, if possible, the observational analysis would be undertaken in the same population from which trial participants were recruited [4,5,6,7].

As an example, the Randomized Evaluation of Decreased Usage of Betablockers After Myocardial Infarction (REDUCE-AMI) randomized trial [8], mainly embedded in the Swedish Web-system for Enhancement and Development of Evidence-based care in Heart disease Evaluated According to Recommended Therapies (SWEDEHEART) registry in Sweden [9], started recruitment in September 2017 and is expected to report initial results in 2024. Because the SWEDEHEART registry also contains data on individuals with myocardial infarction before September 2017, there is an opportunity to design an observational analysis that emulates a trial similar to REDUCE-AMI using data from the same underlying population.

Benchmarking such an observational analysis against the, currently unknown, results of REDUCE-AMI will be deemed “successful” if the same clinical decision would be made on the basis of either source of evidence (i.e., using the trial or the observational analysis results). If this prospective benchmarking is successful, it supports the credibility of analyses using the same observational data to rapidly provide answers to questions that could not be answered by the initial trial. If benchmarking is not successful, we will conduct a “postmortem” analysis to identify the reasons for the discrepancy, which will inform future observational analyses.

Here, we design a target trial as similar as possible to the REDUCE-AMI randomized trial, and then emulate the target trial using observational data from the SWEDEHEART and linked registers.

The index trial: REDUCE-AMI

REDUCE-AMI (ClinicalTrials.gov ID: NCT03278509) is an ongoing registry-based, prospective, randomized, open-label, parallel trial that started recruitment on 11th September 2017 in Sweden (38 centers), Estonia (1 center), and New Zealand (6 centers). An overview of the trial’s protocol is in Table 1. Briefly, individuals are deemed eligible within seven days of type 1 myocardial infarction with preserved left ventricular systolic ejection fraction with a coronary angiography that documented obstructive coronary artery disease. Major exclusion criteria are any indication (other than for secondary prevention) or contraindication for beta blocker treatment according to the treating physician, and any condition that may affect their ability to comply with the study protocol.

Table 1 Outline of protocols of the REDUCE-AMI trial, a similar target trial, and of an emulation of the target trial using the SWEDEHEART Registry
Full size table

Consenting individuals are randomly assigned to either beta blockers or no beta blockers. Individuals randomized to beta blockers are administered the assigned treatment (metoprolol or bisoprolol) during the hospital stay and receive a prescription for continued use after discharge. The treating physician is encouraged to aim for a daily dose of at least 100 mg for metoprolol or 5 mg for bisoprolol, and participants are encouraged to continue the beta blockers indefinitely following discharge, unless contraindications develop. Individuals randomized to no beta blockers are discouraged from using beta blockers as long as there is no new indication. All individuals recruited into the trial receive written information about the importance of continuing the assigned treatment and an ID-card size with the same information in case of medical contact. For blood pressure control, guidelines recommend treatments other than beta blockers as first-line treatment. If an individual is already using beta blockers when enrolled into the study and assigned to no treatment, a tapering of the beta blocker is carried out during the following two to four weeks. It is recommended that tapering ends with at least 4 days of lowest possible dose, corresponding to 12.5 mg metoprolol or 1.25 mg bisoprolol.

The primary outcome is the composite of death from any cause or new myocardial infarction. Information on death is obtained from the Swedish Total Population Register; information on new myocardial infarction during the initial hospital stay and readmission due to a non-fatal myocardial infarction is collected from the SWEDEHEART registry. To estimate the intention-to-treat effect, all individuals will be included in an intention-to-treat analysis, which will be based on events of all follow-up time of each individual from randomization to end of follow-up. Absolute risks of all endpoints will be estimated using Kaplan–Meier curves in each group. Hazard ratios and their 95% confidence interval will be estimated via Cox proportional hazards regression. The trial stopped recruitment on 3rd May 2023, at which point 5014 individuals had been enrolled [8].

The observational analysis

Causal inference from observational data can be seen as an attempt to emulate a pragmatic randomized trial—the target trial—that would answer the question of interest. The approach for emulating a target trial has 2 steps: 1) specify the protocol of the target trial, and 2) emulate the target trial using observational data and appropriate methods [10,11,12,13]. The target trial framework helps articulate a precise causal question, assists with discussions about trade-offs regarding study design, and facilitates comparisons with randomized trials that aim to ask the same question as the observational study. Explicit target trial emulation means the only unavoidable difference between the target trial and its emulation is how treatment is assigned to eligible individuals at the start of follow-up: at random in the target trial; under clinical practice in the emulation. A successful emulation that relies on conditional exchangeability at baseline, therefore, requires detailed data on baseline confounders.

The target trial

To compare REDUCE-AMI to an observational analysis that attempts to answer similar clinical questions, we first specified the protocol of a target trial similar to the protocol of REDUCE-AMI, with deviations only when the observational data did not correspond to the information collected in the trial (see also Table 1) [10]. Here, we briefly discuss the main differences.

Recruitment for the target trial would only be in the Swedish centers participating in REDUCE-AMI (not Estonia or New Zealand) and would run from 1st September 2010 until 10th September 2017 (the day before the start of recruitment for REDUCE-AMI); this difference only reflects data availability. The eligibility criteria are the same as for the index trial except that eligibility extends through 30 days after angiography, individuals are required to have received statin and anti-thrombotic treatment (because most individuals who initiate beta blockers also receive these two treatments), and no prior use of beta blockers is allowed (because we cannot reliably emulate a protocol-mandated tapering of beta blockers in the no beta blocker group). Table 1 describes the operationalization of contraindications and indications to beta blockers, as well as of conditions that limit an individual’s ability to adhere to the assigned treatment. The treatment strategies and outcomes are the same as REDUCE-AMI, and follow-up ends at the earliest of 31st December 2017, five years after baseline, or at the outcome of interest. The causal contrasts in the target trial are the intention-to-treat and per-protocol effects.

The intention-to-treat analysis is the same as for the index trial under the assumption that assignment to beta blockers in the index trial is analogous to the (more pragmatic) target trial where assignment would be a beta blocker prescription. Absolute risks can be estimated nonparametrically via Kaplan–Meier or parametrically, using a smooth function of time, by fitting a pooled logistic regression model with an indicator for assigned strategy, time of follow-up (restricted cubic spline with knots at 6, 12, 24, and 48 months), and a product term between the assignment indicator and time of follow-up. The 5-year risk in each group is then compared via risk differences and ratios. An estimate analogous to the hazard ratios from a Cox regression model can be obtained using the pooled logistic regression model without the assignment—time product term [14]. Inverse probability (IP) weighting can be used to adjust for prognostic factors that are imbalanced at baseline, that is, each individual receives an IP weight whose denominator is, informally, the probability of being assigned to the individual’s assigned group conditional on the prognostic factors. These probabilities can be estimated via a logistic regression model [15]. To estimate the total effect on myocardial infarction, individuals who die are treated as not experiencing the outcome after death [16].

The per protocol analysis in the target trial is similar to the intention-to-treat analysis except participants are censored if and when they deviate from their assigned treatment strategy. To adjust for the potential selection bias induced by this censoring, we need to adjust for prognostic factors that are associated with non-adherence. Given the available information in SWEDEHEART and linked registers, we select the following post-baseline (time-varying) prognostic factors: diagnosis of renal disease; and dispensation of angiotensin 2 receptor blockers, ACE inhibitors, calcium channel blockers, diuretics, nitrates, or diabetes treatments. These variables are used to estimate time-varying IP weights for all individuals [17].

Because adherence to the assigned treatment may vary substantially between the randomized trial and the routine clinical setting, the intention to treat effects estimated in each setting may differ even if both estimates are correct. Though the index trial protocol does not consider estimating the per-protocol effect as a primary aim of the study, we will aim at doing so when its data become available.

The observational emulation of the target trial

We used observational data from the SWEDEHEART registry and linked registers to emulate the target trial. SWEDEHEART includes all patients hospitalized for acute coronary syndrome or undergoing coronary or valvular intervention for any indication in all relevant hospitals across Sweden [9]. The registry was created by merging 4 existing cardiovascular health-care quality registries in 2009: the Register of Information and Knowledge About Swedish Heart Intensive Care Admissions (RIKSHIA), the Swedish Coronary Angiography and Angioplasty Registry (SCAAR), the Swedish Heart Surgery Registry, and the National Registry of Secondary Prevention (SEPHIA). SWEDEHEART is also regularly linked to the Swedish Total Population Register; the Swedish National Patient Register, which records all primary and secondary diagnoses and procedures from inpatient hospitalizations and outpatient specialist care visits across Sweden; the Swedish Cause of Death Register, which records all deaths and causes of death; and the Prescribed Drug Register, which collects information on all prescribed and dispensed medications [18,19,20].

Each component of the protocol of the target trial was emulated as closely as possible using the SWEDEHEART registry data (see Table 1 for details). Eligible individuals were “assigned” to the beta blocker group if they received a prescription of beta blockers within 30 days of angiography, and to the no beta blocker group if they did not receive such prescription. The period of eligibility in the emulation was longer compared with the index trial as many individuals are prescribed beta blockers in the community soon after hospital discharge in routine practice. Individuals with an event in the 30 days following angiography were excluded, which is unlikely to introduce bias because an acute effect of beta blockers is not expected and because individuals at very high risk of an imminent outcome will not be included in the index trial.

In the beta blocker group, the daily treatment dose was assumed to be 100 mg for metoprolol and 5 mg for bisoprolol as the daily treatment dose in Prescribed Drug Register is only correct for primary indication of treatment, and secondary prevention after myocardial infarction is not the primary indication for beta blockers. The intended length of each prescribed dispensation was then calculated from the number of pills and pill dose divided by daily dose. The REDUCE-AMI protocol does not specify a definition of adherence, so we proposed the definition of adherence to be a gap of less than 180 days between the end date of one dispensation and the following dispensation, unless this was preceded by a contraindication, in the beta blocker group, and continuing to not dispense beta blockers, unless this was preceded by a new indication, in the no beta blocker group.

We assumed that assignment (first prescription) was random within levels of the baseline covariates: hospital, year of index, age, sex, smoking status, hypertension, diabetes, previous myocardial infarction, previous stroke, previous percutaneous coronary intervention, history of coronary heart failure, previous cardiac surgery, renal disease, other serious diseases, angiotensin 2 receptor blockers, ACE inhibitors, calcium channel blockers, diuretics, nitrates, diabetes treatments, infarction type, cardiopulmonary resuscitation before hospital, thrombolysis before hospital, cardiogenic shock, electrocardiogram (ECG) rhythm, ECG QRS annotation, ECG ST- & T-wave changes, percutaneous coronary intervention, angiography finding, stenosis class, proportion stenosis, IV beta blockers, IV diuretics, IV inotropic drugs, IV nitrates, heart rate, systolic blood pressure, diastolic blood pressure, LDL cholesterol, HDL cholesterol, creatinine, body mass index (see Supplementary Table 1). The outcomes were the same as the target trial and also identified in the Swedish Total Population Register and SWEDEHEART like REDUCE-AMI, but data were not available on migration.

The intention-to-treat analysis (i.e., effect of first prescription) was the same as for the target trial with all baseline covariates adjusted for. The per-protocol analysis was the same as the target trial. For primary analyses, missing data for continuous covariates were imputed using the median of all non-missing instances, and missing categories were included for categorical variables. Non-parametric bootstrapping with 500 samples was used to calculate 95% confidence intervals.

Sensitivity analyses

We assessed the robustness of our effect estimates to several analytic decisions. For the intention-to-treat analysis, we aimed to understand: (1) the impact of different modelling assumptions by including baseline covariates in the pooled logistic regression model instead of using IP weights; (2) the sensitivity of the results to emulation of the eligibility criteria “no indication for beta blockers other than as secondary prevention” by (2a) modifying the eligibility criteria to not exclude individuals with a beta blocker before baseline (and additionally adjusting for prior beta blockers recorded in either SWEDEHEART or in the prescribed drug register in the prior 3 years), and (2b) reformulating the operationalization of prior use of beta blockers to be only those with registered beta blocker use on hospital admission in SWEDHEART rather than in both SWEDHEART and the prescribed drug register (and additionally adjusting for prior beta blockers in the prescribed drug register in the prior 3 years); (3) the sensitivity of the results to the missing data assumptions by (3a) carrying out a complete case analysis and (3b) categorizing continuous variables with a “missing” category (categories specified in Supplementary Table 1); and (4) the impact of the decision to extend baseline through 30 days following angiography on the risk of misalignment of eligibility, treatment assignment, and start of follow up by using a cloning and censoring approach (full details in “Appendix”) [21, 22]. For the per-protocol analysis, we reduced the time allowed between prescription dispensations for the definition of continuous treatment in the beta blocker group to 90 days rather than 180 days to understand if having larger gaps between dispensations affected results.

Results

Figure 1 shows a flowchart of selection into the target trial emulation, and Table 2 shows the baseline characteristics for the 10,926 eligible individuals. Compared with those assigned to no beta blockers (1,198 individuals), those assigned to beta blockers (9,728 individuals) were, on average, younger, and more likely to smoke and to present with NSTEMI with a higher proportion of coronary stenosis, and less likely to have prior cardiac diseases. They also had higher heart rate and blood pressure.

Fig. 1
figure 1

Flowchart for selection of eligible individuals into an emulation of a target trial of beta blockers versus no beta blockers in individuals with myocardial infarction with preserved ejection fraction in Sweden, 2011–2017

Full size image
Table 2 Baseline characteristics of eligible individuals for an emulation of a target trial of beta blockers versus no beta blockers in Sweden, 2011–2017 (continued in Supplementary Table 5)
Full size table

Intention-to-treat effect: the effect of prescription versus no prescription of beta blockers at baseline

Median follow up was 41 months in the beta blocker group and 36 months in the no beta blocker group. The estimated 5-year risks of the composite outcome were 10.2% (9.4%, 11.1%) under beta blockers and 11.9% (8.5%, 15.3%) under no beta blockers, which results in a risk difference of − 1.7% (− 5.5%, 1.9%), a risk ratio of 0.86 (0.64, 1.23) (Table 3 and Fig. 2), and an average hazard ratio by 5 years of 0.78 (0.59, 1.12) (Table 4). We also show the hazard ratio with follow up time terminated at 3 and 4 years in Table 4.

Table 3 Estimated risks of death and myocardial infarction by 5 years in an emulation of a target trial of beta blockers versus no beta blockers in Sweden, 2011–2017
Full size table
Fig. 2
figure 2

Estimated risk of death and myocardial infarction under beta blockers versus no beta blockers. Intention-to-treat analysis of an emulated target trial in individuals with myocardial infarction with preserved ejection fraction in Sweden, 2011–2017 (shaded intervals represent limits of the pointwise 95% CIs)

Full size image
Table 4 Estimated hazard ratios of the composite outcome for beta blockers versus no beta blockers by 3, 4, and 5 years in an emulation of a target trial in Sweden, 2011–2017
Full size table

For each component of the composite outcome, the estimated 5-year risks of death were 5.6% (4.9%, 6.3%) under beta blockers and 5.9% (3.7%, 8.4%) under no beta blocker, which results in a risk difference of − 0.3% (− 2.9%, 1.9%), a risk ratio of 0.96 (0.67, 1.45), and an average hazard ratio by 5 years of 0.85 (0.59, 1.38); the estimated 5-year risks of myocardial infarction were 5.3% (4.7%, 5.8%) under beta blockers and 6.9% (4.4%, 9.7%) under no beta blockers, which results in a risk difference of − 1.6% (− 4.5%, 1.0%), a risk ratio of 0.76 (0.53, 1.22), and an average hazard ratio by 5 years of 0.75 (0.53, 1.21).

Results in sensitivity analyses were broadly similar to those from the main analysis for the composite outcome (Appendix Table 2).

Per-protocol effect: the effect under full adherence to the treatment strategy in the protocol

Adherence was 74% in the beta blocker and 91% in the no beta blockers group. After censoring at non-adherence to the assigned treatment strategy but before IP weighting, median follow up was 29 months in the beta blocker group and 31 months in the no beta blocker group. The estimated 5-year risks of the composite outcome were 10.2% (9.3%, 11.2%) under beta blockers and 11.0% (7.6%, 14.4%) under no beta blockers, which results in a risk difference of − 0.8% (− 4.5%, 2.8%), a risk ratio of 0.92 (0.69, 1.37) (Table 3 and Supplementary Fig. 1), and an average hazard ratio by 5 years of 0.84 (0.64, 1.24) (Table 4). We, again, show the hazard ratios with follow up terminated at 3 and 4 years in Table 4.

For each component of the composite outcome, the estimated 5-year risks of death were 5.6% (4.9%, 6.5%) under beta blockers and 5.7% (3.5%, 8.5%) under no beta blockers, which results in a risk difference of − 0.1% (− 2.8%, 2.2%), a risk ratio of 0.99 (0.67, 1.60), and an average hazard ratio by 5 years of 0.85 (0.58, 1.36); the estimated 5-year risks of myocardial infarction were 5.2% (4.6%, 5.9%) under beta blockers and 5.9% (3.3%, 8.5%) under no beta blockers, which results in a risk difference of − 0.7% (− 3.3%, 2.1%), a risk ratio of 0.88 (0.60, 1.59), and an average hazard ratio by 5 years 0.86 (0.60, 1.53).

Results in the sensitivity analyses were broadly similar to the main analysis for the composite outcome (Supplementary Table 3).

Discussion

We used observational data from SWEDEHEART and linked registers to emulate a target trial with a protocol similar to that of REDUCE-AMI, an ongoing trial which is also embedded in SWEDEHEART. Had everyone adhered to the treatment strategy as specified in the target trial protocol, we estimated a reduction in the 5-year risk of death or myocardial infarction of 0.8 percentage points for beta blockers compared with no beta blockers. However, effects ranging from a reduction of 4.5 percentage points to an increase of 2.8 percentage points are compatible with our data under conventional statistical criteria. Once results of REDUCE-AMI are published, we will compare results of our observational analysis against those from the trial to understand if our emulation was successful.

Randomized trials and observational studies differ, of course, in the treatment assignment procedure, which is randomized in the trials but not in the observational data. Also, randomized and observational analyses that ask a similar question often differ with respect to eligibility criteria, causal contrast, and measurement of treatment or outcomes. We will now consider these differences and their potential impact when comparing results between REDUCE-AMI and our observational emulation.

Despite coming from the same registry, the study populations of the target trial emulation and the index trial will be slightly different for three reasons. First, the target trial emulation cannot exclude individuals who would not enrol in a randomized trial. Because generally, healthier individuals of higher socioeconomic status agree to enrol in trials, we expect index trial participants will have less severe disease and a lower comorbidity burden than those in the target trial emulation [23]. Second, the target trial emulation cannot fully replicate eligibility criteria that are vaguely articulated for the index trial. Specifically, the index trial will exclude individuals if they have an “indication for beta blockers other than as secondary prevention according to the treating physician” (but prior beta blocker use is not an explicit exclusion) [8]. To operationalize this ambiguous eligibility criterion, the target trial excludes anyone with prior use of any beta blocker (recorded in SWEDHEART or the prescribed drug register in the prior 3 year) or with characteristics that would mean they were indicated for a reason other than secondary prevention (high heart rate, high blood pressure, heart failure, atrial flutter). When we included those with prior beta blocker use in a sensitivity analysis, the 5-year risk was higher in both groups and the effect estimates was attenuated, compared with the main analysis. Third, the target trial emulation was restricted to individuals who received statins and anti-thrombotic treatment at baseline, which occurred in 94% of otherwise eligible individuals. This eligibility criterion excludes individuals who are ineligible for statins or anti-thrombotics (e.g.,  individuals that do not tolerate statins or have very high bleeding risk) and who perhaps did not receive these treatments because of poor post-myocardial infarction prognosis. Subgroup analyses may be needed to understand if a different composition of the study populations explains different effect estimates between the index trial and the target trial emulation.

REDUCE-AMI will estimate an intention-to-treat effect, which is the effect of assignment to one of the treatment strategies under the level of adherence observed in the trial over follow-up. The magnitude of the intention-to-treat effect depends on the type of assignment and adherence. Therefore, estimates of the intention-to-treat effect in the index trial and the observational emulation of the target trial may differ because of differences in assignment (a randomized prescription in the trial versus a routine prescription in the observational data) or adherence (still unknown in the trial) [24]. A more comparable contrast is the per protocol effect [25], which we plan to estimate once trial data become available (as part of a fully harmonized analysis, including the handling of competing events).

A key advantage of our study is that outcome definitions should be the same as those used in REDUCE-AMI as both studies use routinely collected data from Swedish registers (SWEDEHEART and the Total Population Register) to identify individuals with an outcome. Variation would be possible if, the coding of myocardial infarction had changed between the time of the trial and observational analysis, but this is not the situation in our example.

The main effect measure in REDUCE-AMI will be the hazard ratio from a Cox model, that is a weighted average of the time-varying hazard ratios during the follow-up. Because this average hazard depends on the length of follow-up and the distribution of censoring [26], hazard ratio estimates in the index trial and the emulated target trial may differ. To ameliorate this problem, we report average hazard ratios when follow up is terminated at three, four, and five years in Table 4. More comparable effect measures are the risk difference and risk ratio at a fixed time of follow-up (e.g., five years), which we plan to estimate in both studies once trial data become available.

The observational data used to emulate the target trial had detailed clinical information on the index myocardial infarction required for eligibility. However, there were still data limitations that restricted the emulation, such as no access to primary care records. A consequence of this is that we may have missed individuals with other beta blocker indications or with a beta blocker contraindication. Additionally, without continuously updated information on lab values and diagnoses that did not require a hospital visit for the duration of follow up, we miss potential prognostic factors associated with nonadherence to the treatment protocol. This could be a source of residual bias in our per protocol analyses.

In summary, we have designed a target trial with a protocol similar to REDUCE-AMI, then emulated the target trial using observational data from the same population in which REDUCE-AMI participants are recruited. Randomized trials cannot answer all important clinical questions, and observational data can be used to complement these trials and fill evidence gaps; benchmarking an observational analysis against a randomized trial then using the observational data to ask additional questions is one way to achieve this. Prospective benchmarking shifts the investigator focus away from an endeavour to use observational data to estimate similar results as a completed randomized trial, to a systematic attempt to align the design and analysis of the trial and the observational analysis.