FormalPara Key Points

Guidelines, pharmacoeconomic assessments, and policies for reimbursement can make comparisons in pharmacoepidemiological studies difficult to conduct due to selection bias and confounding.

Use of a contextual cohort could be considered as an option to characterise the adoption of new medications into clinical practice and describe the prevalence of clinical characteristics and risk factors.

1 Introduction

Pharmacoepidemiological studies examine the use of medications and any effects from their use (including adverse drug reactions) in populations of interest [1]. While safety information from randomised controlled clinical trials (RCTs) is useful in providing a measure of effect for adverse events (comparing the new treatment with another treatment) [2], the primary endpoint of these trials may not be to compare safety but rather to show efficacy. Trials can be small in size and insufficiently powered to inform on rare adverse events. In addition, they often have exclusion criteria that will not always apply in real-life use, making them unrepresentative of the medication’s use in the general population [3]. RCTs are most commonly conducted in the premarketing phase of a medicinal product, while other study designs are used in the postmarketing phase to enable real-life use to be monitored. Pharmacoepidemiological comparison studies are used to determine if a new treatment has a reduced, equal, or greater risk of adverse events than other available treatments in real-world populations, even in cases where the comparisons are problematic or inappropriate.

However, consideration of guidelines, pharmacoeconomic assessments, and policies for reimbursement have made pharmacoepidemiological studies far more difficult to conduct in recent years. Adherence to guidelines for prescribing issued by organisations that conduct health technology assessments, such as the National Institute for Health and Clinical Excellence (NICE) in the UK [4] and, internationally, local formulary restrictions (where prescription of medications is restricted to those approved by a committee), can significantly impact prescribing decisions that control allocation to treatment, with little variation based on physician judgement and choice. In the absence of prescribing guidelines, differences between patients prescribed a medication and those who are not may mean it is inappropriate to compare the two cohorts. For instance, health insurance policies can restrict treatment options that prescribers can provide to an individual patient. Patients may not be prescribed new medications unless it has been established that they are intolerant to or have not responded to another existing medication, or they have specific characteristics or conditions which are a contraindication for use of another treatment. These patients may have more severe forms of the indicated illness compared with patients who sufficiently respond to standard treatment. Alternatively, new treatments may be provided to the healthiest patients who are considered at lowest risk of adverse events. Cost of the medication is often a major consideration as it may not be recommended for prescription unless there is evidence that a patient cannot use a cheaper medication or that the more expensive medication has superior efficacy. While reducing costs and adhering to budgets is important, it can have a major impact on pharmacoepidemiological research. Where certain subject characteristics influence the probability of being exposed to a treatment, this can introduce systematic bias caused by selection bias and confounding.

In such situations, the characteristics between patients who are prescribed the medication under study may be substantially different to patients prescribed an alternative treatment (or no treatment), and comparisons made through conventional pharmacoepidemiological designs may therefore be inappropriate. The contextual cohort methodology can be used in these instances to overcome limitations and challenges encountered if conventional study designs were applied. These limitations may be encountered in studies conducted across different populations, therefore there is wide applicability of the contextual cohort approach.

2 Existing Pharmacoepidemiological Study Designs

In pharmacoepidemiology, observational designs are most commonly applied. These include classic designs such as the cohort and case-control designs, and case-only designs such as the case-crossover and self-controlled case series. These designs allow comparison of treatment effects between different groups of patients, which is a key aspect of pharmacoepidemiology; however, these designs are susceptible to bias and confounding. Minimisation of selection bias can be challenging where guidelines dictate treatment choices. Confounding also needs to be considered and if all confounding factors are not properly adjusted for in the analysis, this could result in an incorrect estimate of the measure of effect [5]. In case-only designs, the patient is used as their own control. This eliminates confounding from non-time-variant factors (i.e. factors that do not change over time), although time-varying confounding, such as changes in concomitant medication, still needs to be considered [6, 7]. Case-only designs are based on a number of assumptions, which restrict the use of these studies in pharmacoepidemiology to only certain events and exposures; use in inappropriate settings could result in bias [6, 7]. Restriction is a method of controlling confounding at the design stage. The method involves restricting the study population on one or more baseline covariates, to eliminate the confounding effect of that covariate [8]. Study participants are limited to one category of the confounding factor. For example, if restricting on smoking status, one might include only non-smokers in the study to eliminate the confounding effect of smoking. However, the method suffers some limitations, such as the inability of examining the effect of the restricted variable and limiting the sample size and power. Where the interest is in examining outcomes in the real-world, restriction may not be a suitable approach.

Pragmatic trials, such as the series of Randomised Evaluations of Accepted Choices in Treatments (REACT) trials [9, 10], can be used to bridge the evidence gap between RCTs and pharmacoepidemiological studies. Their randomised nature removes the issues of selection bias and confounding, but inherent limitations remain. Physicians could be unwilling to prescribe branded medication to patients against formulary guidelines, potentially affecting the use of a randomised study design. There may also be difficulty in recruiting patients into the study, therefore obtaining sample sizes with sufficient power may not be possible. Other issues relate to the choice of drug and initial indication of a new drug, in addition to outcome definitions and the requirement for safety reporting.

The choice of a study comparator is another issue related to pharmacoepidemiological studies. Most commonly, the choice of a study comparator when examining a new treatment would be a current, well-established medication that is usually the most widely used treatment for the disease in question. However, the use of such a comparator can be a challenge in observational research if a new treatment is unlikely to be administered to the same population of patients receiving the current treatment. The two treatment groups could be different in many aspects, such as age of the patient, disease severity, concomitant medications, comorbidities, and other risk factors. In this case, a cohort based on the counterfactual ideal, which is equivalent to the cohort of interest except for the exposure, may be difficult to create. Another option for choosing a comparator is to use another new treatment for the same indication, where one is available. While patient characteristics are more likely to be similar in real-life use, the main problem with this option is that safety in relation to the usual standard of care cannot be examined. It may be that the choice of comparator has to remain reliant on the choice of study design and whether selection bias can be minimised effectively. An important point to consider in the study design is that differences in patients’ characteristics between those who receive the new treatment and patients who receive the comparator are augmented (thus introducing systematic differences) when the prescribing of the new treatment is mostly, or nearly always, in compliance with guidelines that are adhered to, compared with doctors prescribing only on the basis of their knowledge and clinical judgement.

2.1 Managing Limitations in Pharmacoepidemiological Studies

Where confounding is likely to be an issue in pharmacoepidemiological comparison studies, a decision needs to be made on how to deal with this at either the design stage or the analysis stage of a study. At the design stage, randomisation is the favoured technique to control for confounding where appropriate, since potential confounders are randomly distributed between groups [11]. It also deals with selection bias, unlike other methods that deal with confounding. However, in pharmacoepidemiology, randomisation is not often practical when use of a medication in real-life populations needs to be explored. Therefore, statistical methods are often employed to handle observed confounders at the analysis stage; such methods include, but are not limited to, stratification, regression analysis, and propensity scores. However, such methods possess limitations of their own. For example, regression requires information on all potential confounding factors to have been collected and cannot account for unmeasured confounders. Models can become unstable when attempting to account for too many confounding factors, particularly when the sample size is small. In general, 10 observations per risk factor are needed in a model to ensure stability [11]. As such, small studies with multiple potential confounders may not be analysable using regression techniques. Conversely, the use of propensity scores involves ‘throwing out’ patients who cannot be matched, and this can cause considerable problems where patients have completely different risk profiles and therefore unequal probability of receiving treatment. If a large number are excluded, then the representativeness of patients in the analysis may be questioned and may no longer reflect patients in routine clinical practice. It may be that construction of propensity scores identifies that the two treatment groups are not comparable at all, or that the analysable cohort is a very small proportion of the recruited cohort [12]. In comparison to regression models, a larger number of potential confounders can usually be accounted for with propensity scores, and a small study with a low number of events is still analysable [13]. However, evidence is limited that propensity score methods provide substantially different estimates compared with multivariable regression analyses to control confounding [14]. Use of a contextual cohort provides an alternative method, avoiding some of the limitations seen with these more commonly used methods to deal with bias and confounding in pharmacoepidemiology.

3 The Contextual Cohort

Comparisons based on the counterfactual ideal in pharmacoepidemiology are sometimes not feasible when examining real-life medicinal use. A different approach is to gather information on a contextual cohort in addition to the group receiving the treatment of interest, to determine the prescribing patterns of two products licensed for a similar indication at baseline. Comparison of the reasons for prescribing in the treatment group against a comparator group using standard (existing) therapy can put utilisation observations into context. The distinguishing feature between the contextual cohort approach and a conventional comparator group is that no comparison of measure of effect is made between the two cohorts (e.g. no risk ratio is calculated).

3.1 Use of Contextual Cohort Methods

Contextual cohort methods may be useful when comparative pharmacoepidemiological study designs are inappropriate, especially when there are systematic differences in certain characteristics or risk factors between the two groups to be compared. The use of a contextual cohort in such situations avoids the limitations that would be encountered when using a conventional design, such as biases, confounding, and obtaining a sample size appropriately powered to make comparison, because no formal comparisons are made between the two cohorts.

The method allows comparison of patient characteristics, distribution of risk factors for the outcome of interest, and non-clinical reasons for prescribing in each group. This is particularly useful for newly marketed medications, where the adoption of the product into clinical practice can be characterised. For instance, it might be possible to determine the reasons to choose prescribing the newer product over a more established medication. Where patients are switched from established medications to new medications, the reasons for switching can also be explored and the risk profile of switchers can be examined further.

3.2 Methodology

The goal of contextual cohort methodology is not to make formal comparisons of risk of a primary outcome between the drug of interest and the product used in the contextual cohort. It is the differences between the groups in terms of patient characteristics that are of interest, rather than differences in outcome.

Therefore, patient characteristics for both the primary group of interest and the contextual cohort are examined separately. These may be tabulated and descriptive statistics used to describe the characteristics of each cohort. It is acceptable to calculate absolute risks, rates, or odds of an outcome in each group, but to calculate their ratio would be inappropriate.

This approach will highlight biases and/or confounders that might have affected results had the study been conducted using a conventional pharmacoepidemiological design, where formal comparison of the effect measures in each group would have been calculated (through risk ratio, rate ratio, odds ratio).

3.3 Analyses and Interpretation

The aim of using a contextual cohort design is to characterise the adoption of a new medication into clinical practice and explore the factors that drive prescribing of the new medication instead of existing options. Comparison of the patient cohorts prescribed each medication identifies possible differences in factors, such as setting, prevalence of (non-clinical) reasons for prescribing, physician prescribing preference factors, and those clinical characteristics that are known risk factors for the primary outcomes of interest. The key purpose of the use of a contextual cohort is to explore the variation in the distribution and determinants of prognostic and clinical risk factors at baseline.

Use of the contextual cohort negates the need to control for confounding, since the purpose is not to compare estimates of effect. However, other modelling techniques can be considered to explain variation in prescribing. For example, multi-level modelling can provide insight into sources of variability. An alternative approach to examine prescribing variability between two cohorts is the use of propensity scores for descriptive purposes only. Propensity scores can be calculated for each cohort to examine the distribution for each treatment, rather than for matching purposes, as is done in conventional pharmacoepidemiological studies. This allows exploration of the differences between the two cohorts, focusing on whether patients have any overlap in the predicted probability of receiving treatment between the treatment of interest and the contextual cohort. If little to no overlap is detected, then this would support the assertion that use of a comparator based on the counterfactual ideal is inappropriate.

3.4 Limitations of Contextual Cohort Methodology

The choice of contextual cohort can be difficult, particularly if more than one existing treatment is used for the same indication. In the majority of cases, it might be considered most appropriate to use the treatment that has been most commonly used or has been marketed for the longest duration as the standard of care. Alternatively, it might be that the medication of interest is expected to target a similar user population to that of another treatment already used in clinical practice, and therefore analysis of the reasons for prescribing one versus the other would be of interest. The choice of contextual cohort will therefore depend on the medication of interest and the question(s) that the study seeks to answer.

While the contextual cohort method is useful to highlight different patient characteristics between the two groups and to generate baseline knowledge, it is not possible to determine whether one medication has excess risk of an outcome versus the other because the patient characteristics are often too dissimilar to make direct comparisons. Analyses taking into account both patient groups are therefore limited. Where calculating risk within each cohort though, comparisons can be made with corresponding clinical trial data or observational data. This allows the risks to be related to the known literature, without attempting a formal comparison between study drugs. The collection of a broad range of data on risk factors also allows quantification of risk factors for the outcome of interest. With regard to sample size calculations, studies that use a contextual cohort can be powered based on known risk from clinical trials. The principle is to estimate the risk so that it lies within an acceptable range (margin of error) of the estimate from clinical trials, assuming this represents the true value. Ideally this margin of error (also called precision) should be as narrow as possible so that the frequency is estimated as precisely as possible. The sample size is chosen so that it will yield a confidence interval of a predefined width for the risk.

4 Examples of Contextual Cohort Use

The contextual cohort design has been utilised previously in a Specialist Cohort Event Monitoring (SCEM) study [15], evaluating the safety and utilisation in patients prescribed rivaroxaban for the prevention of stroke in patients with atrial fibrillation (AF), the treatment of deep vein thrombosis (DVT) and pulmonary embolism (PE), and the prevention of recurrent DVT and PE in a secondary care setting in England and Wales, using the SCEM technique [16,17,18]. Rivaroxaban is a direct oral anticoagulant (DOAC), a class of drug recommended as an alternative to conventional anticoagulant treatment with vitamin K antagonists (VKAs) in international guidelines [19, 20]. The Rivaroxaban Observational Safety Evaluation (ROSE) study included a contextual cohort of patients prescribed warfarin, in order to compare reasons for choice of anticoagulation type and to explore differences in both the clinical setting of initiation and baseline risk of bleeding and stroke [16,17,18]. Bleed outcomes could be estimated, but due to the differences between the rivaroxaban and warfarin cohorts, direct comparisons between them were not appropriate. An evidence-based national technology appraisal (TA) for the use of rivaroxaban in the prevention of stroke and embolism in people with AF was first issued by NICE in May 2012, while a TA for the use of rivaroxaban in treating PE and preventing recurrent venous thromboembolism was published by NICE in June 2013 [21]. In the UK, the National Health Service (NHS) is legally bound to fund and resource medicines and treatments recommended by NICE TAs. In addition, local NHS trusts maintain formularies that also govern prescribing practices. These guidelines influence prescribing of rivaroxaban and warfarin in the UK, which is why differences between the two cohorts arise. Within the ROSE study, it was found that prescribing variability was dominated by differences between NHS trusts and between prescribers (within trusts) [22]. Some patient factors were important in treatment choice but accounting for patient differences did not fully explain the variance both between prescribers (within trusts) and between trusts [22].

Meanwhile, contextual factors have been examined in previously published literature, especially within social sciences. These studies refer to the concept of the contextual phenomenon, where health status varies based on different environmental influences [23, 24]. One such example is that of a review of RCTs investigating the performance of healthcare workers in sub-Saharan Africa [25]. The purpose of including contextual factors in the review was to understand how differences in these contextual factors contribute to differences in the local effect of interventions to improve performance [25]. In this study, contextual factors were grouped into the categories ‘management’, ‘staffing’, and ‘local environment’ [25]. The authors concluded that the importance of local contextual influence in interventional trials should be reflected in the planning, design, and reporting of the study results [25]. A second example used contextual information, such as factors that can affect the impact of an intervention, to understand whether programmes are providing interventions to those in need and reasons for or against this [26]. In both of these studies, contextual data were found to be important in improving interventions in real-world settings [25, 26].

A review of qualitative research addressing antibiotic prescribing in long-term care facilities highlighted that prescribing is heavily influenced by the context of care [27]. Contextual factors affecting antibiotic prescribing in this setting included a lack of resources (including doctors and diagnostic equipment), the relationship between the healthcare provider and patient, and whether care in the long-term facility was a priority over hospitalisation [27]. A similar review describing antibiotic prescribing in hospitals also assessed contextual aspects of prescribing; it was determined that such factors are partly responsible for the differences in antibiotic use [28].

5 Conclusion

Overall, the contextual cohort is an appropriate option within pharmacoepidemiological study designs to consider when a conventional comparator may not be available or appropriate, or where the primary study purpose is to explore the differences between treatment groups rather than compare effect estimates. Where prescribing guidelines are present, conducting comparisons in pharmacoepidemiology produces many challenges that are not often correctly overcome by using conventional pharmacoepidemiological study designs. Use of a contextual cohort could be considered as an option to characterise the adoption of new medications into clinical practice and describe the prevalence of clinical characteristics and risk factors in the two cohorts, rather than compare event rates and produce an estimate of effect.