Introduction

The arrival of anti-calcitonin gene-related peptide (CGRP) pathway monoclonal antibodies for the treatment of migraine has significantly impacted the clinical field of headache medicine in recent years. Several monoclonal antibodies have been tested in large randomized clinical trials with positive results in terms of efficacy, tolerability and safety in both episodic (EM) and chronic migraine (CM) [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23]. Since 2018, four monoclonal antibodies have been approved by regulatory agencies and are commercially available: eptinezumab, erenumab, fremanezumab and galcanezumab.

The settings of a clinical trial and the profile of its participants may however not fully reflect everyday practice. Real-world data (RWD) from routine clinical care allow to assess effectiveness (as opposed to efficacy) of new treatments, to compare new treatments to the standard of care (useful in guideline development), and provide postmarketing safety information. According to the FDA, RWD are “the data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources” [24]. Analysis of RWD can generate Real-World Evidence (RWE), “the clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD” [24].

Prospective and non-randomized observational studies reflect everyday practice best but are of lower quality compared to randomized controlled trials in the hierarchy of evidence-based medicine [25]. The former studies however may help in understanding real-world experiences from clinicians treating patients with new therapeutics. RWE studies can also be advantageous in collecting data from patient groups which may be excluded from randomized controlled trials, such as patients in higher age groups, patients with high numbers of prior preventive treatment failures or patients with certain comorbid conditions. Quality control of those studies should be equally rigorous. Methodological considerations made during study design and the reporting of methodology within a scientific article are profoundly important aspects of evidence-based medical research. When the methodological quality of the study is high, the RWE generated can be regarded as complementary to data from randomized controlled trials [26]. RWD of the highest quality may be used for decision making processes by regulatory medicine agencies [24, 27, 28].

The International Headache Society (IHS) has created guidelines for the development and conductance of clinical trials, but recommendations or guidelines for the collection of RWD from prospective observational studies are currently not available. Standardization of definitions, baseline characteristics and outcome measures is needed to understand treatment effects and to compare different studies [29,30,31].

In this systematic review, we study the methodology of prospective, observational, clinic-based studies investigating effectiveness, tolerability and safety of anti-CGRP pathway monoclonal antibodies for the treatment of migraine. The primary objectives of this systematic review are 1) to summarise the used definitions within these studies, 2) to investigate the reported outcomes used and 3) to perform a risk of bias assessment for each of the different studies.

Methods

This systematic review was prospectively registered with PROSPERO (CRD42022369366) and adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [32,33,34,35]. A systematic search within the databases of MEDLINE (PubMed interface) and Embase was developed and performed by authors NV and KPi (Table 1). Query results were filtered between May 1st 2018 (i.e. following the approval of erenumab by the FDA, the first market authorization for an anti-CGRP pathway monoclonal antibody globally) and September 30th 2022 (pre-determined end date). The search queries can be found below. Articles were screened by reading the titles, abstracts and keywords and if needed the full text.

Table 1 Systematic search for the databases of MEDLINE (PubMed interface) and Embase
Full size table

Study selection, inclusion and exclusion criteria

Following the retrieval of the query results for all databases, two authors (NV and KPi) independently screened the individual abstracts for eligibility. The full texts of records deemed eligible were retrieved after which two authors (NV and KPi) independently read and evaluated the manuscripts for inclusion. When confronted with discordance, a decision based on consensus after retrieval of the full text was done by both reviewing authors (NV and KPi).

We included research articles providing prospective, clinic-based, observational data on adult human subjects treated with anti-CGRP pathway monoclonal antibodies and with primary endpoints or outcomes of effectiveness, tolerability and safety of these drugs in clinical practice. Studies could only be initiated by clinical researchers, not by pharmaceutical companies. The research article had to declare the prospective design of the study. Studies reporting on participants with EM and/or CM, with or without medication-overuse headache (MOH), were eligible. Exclusion criteria were: articles with an experimental primary focus not involving effectiveness, tolerability or safety of the drug; retrospective analyses; pharmaco-economic database studies; systematic reviews and meta-analyses. There were no geographical restrictions but papers needed to be written in English.

Research articles fitting the criteria formulated above were independently assessed in the systematic review on their own methodology, predefined outcomes and definitions and reported outcomes. Therefore, multiple manuscripts which were part of a larger research effort by the same study group remained analysed separately for the following reasons: 1) fully equivalent methodological approaches in terms of outcomes or definitions across manuscripts could not be assumed a priori to the analysis, 2) the focus of this review was on the reporting standards and the methodological considerations of the manuscripts rather than the actual collected study results.

Data extraction

A structured digital form was established for data collection. Data points on the following characteristics were collected: general study characteristics, treatment regimens, headache-related definitions utilized within the study, baseline characteristics of participants, headache characteristics (medical history, symptomatology, medication usage), headache diary usage, primary and secondary endpoints and objectives, usage of validated questionnaires, collection of tolerability and safety aspects (adverse events, serious adverse events, discontinuation rates and aspects and pregnancies during registration). Outcomes analysed were based on the IHS guidelines for controlled trials of preventive treatment of CM in adults [31]. Possible outcomes for a data point were “present”, “not present” or “no information”. Unless the authors specifically stated that a data point was not available or not reported in the study manuscript, all information that was missing was documented as “no information”.

Study quality assessment

Study quality assessment (i.e. good, fair or poor) of the included studies was conducted using the “National Heart, Lung and Blood Institute (NHLBI) Study Quality Assessment Tool for Before-After (Pre-Post) Studies with No Control Group” [36, 37]. Two authors (NV and KPi) assessed the quality of each manuscript independently by applying this tool. When confronted with disconcordance, a decision based on consensus was made by both authors (NV and KPi).

Results

Included articles

Both queries resulted in a total of 3788 articles (PubMed = 1109 results, EMBASE = 2599 results). After removal of duplications and screening of records, 228 records were assessed in more depth for eligibility. Finally, 46 studies fitted the criteria for the systematic review and were analysed [38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83]. The flow diagram can be found in Fig. 1. Included studies can be found in Table 2.

Fig. 1
figure 1

PRISMA 2020 flow diagram for systematic reviews

Full size image
Table 2 Included studies (N = 46)
Full size table

General study characteristics

All 46 studies included participants of male or female sex [38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83]. Twenty-seven studies included participants with either EM or CM [38, 40,41,42,43,44,45,46, 48, 54,55,56, 60, 61, 65, 69,70,71, 73,74,75,76, 78, 80,81,82,83]. Nineteen studies only looked at participants with CM [39, 47, 49,50,51,52,53, 57,58,59, 62,63,64, 66,67,68, 72, 77, 79]. Forty studies (87%) reported the use of the ICHD-3 criteria for the diagnosis [38, 40, 43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62, 64,65,66,67,68,69,70,71,72, 74,75,76,77, 79,80,81,82,83,84].

Studies had a median number of 111 participants (interquartile range 61 to 164 participants). Nineteen studies (41.3%) formulated exclusion criteria for participation in the study [46, 49, 50, 52,53,54,55,56, 60, 61, 63,64,65,66,67, 69, 72, 73, 83]. Twenty-five studies (54.3%) used a minimum age [40, 41, 43, 45, 46, 49, 50, 53, 54, 56, 58,59,60, 62, 64, 66, 67, 70, 72,73,74, 79,80,81, 83]. Eleven 11 studies (23.9%) used a maximum age in the inclusion and exclusion criteria [41, 43, 45, 46, 53, 54, 66, 67, 72, 74, 83]. The start date was not reported in 7 studies (15.2%) [39, 51, 57, 69, 73, 77, 78]. The end date was not reported in 6 studies (13%) [39, 51, 57, 69, 73, 77].

Participants with a concurrent diagnosis of MOH were included in 32 studies (69.6%) [38, 41, 43, 44, 46, 47, 49, 51,52,53,54, 56, 57, 59,60,61,62,63, 65,66,67,68, 71, 72, 74, 76, 77, 79, 81,82,83]. In those 32 studies, 13 declared the use of the ICHD-3 criteria for the diagnosis (40.6%) [40, 43, 44, 47, 51, 52, 57, 61, 65,66,67,68, 81].

Baseline period duration was 4 weeks for 11 studies (23.9%) [45, 50, 54, 55, 63, 69,70,71], 1 month in 9 studies (19.6%) [40, 47, 59, 62, 76, 77, 79,80,81], 3 months in 8 studies (17.4%) [51,52,53, 67, 72,73,74, 83] and 6 months in 1 study (2.2%) [64]. Seventeen studies (37.0%) did not mention the baseline period duration [38, 39, 42, 46, 48, 49, 56,57,58, 60, 61, 65, 66, 68, 75, 78, 82].

In 28 studies (60.9%), a minimum of 1 failed previous preventive drug was required to enter the study [38,39,40,41, 45, 47, 50, 54,55,56, 58,59,60, 62,63,64,65,66, 70,71,72, 74, 76,77,78,79, 82, 83]. Of those 28 studies, a minimum of 2 past previous preventive therapies was required in 8 studies (28.6%) [50, 54, 60, 65, 71, 79, 83], 3 past previous preventive therapies in 17 studies (60.7%) [38,39,40,41, 47, 56, 58, 59, 62,63,64, 66, 74, 76,77,78, 82] and 4 past previous preventive therapies in 3 studies (10.7%) [55, 70, 72].

A formal sample-size calculation was performed in 10 studies (21.7%) [40, 53, 57, 70, 78,79,80,81,82,83]. In 9 studies (19.6%) this was not done [38, 39, 41, 43, 47, 58, 59, 63, 76]. There was no information on sample size calculations in 27 reports (58.7%) [42, 44, 46, 48,49,50,51,52, 54,55,56, 60,61,62, 64,65,66,67,68,69,70, 72,73,74,75, 77].

Treatment regimens

The following drugs and subcutaneous dosing schemes were used: erenumab 70 mg monthly (n = 23, 50%) [42, 45, 47,48,49, 51, 54, 56, 58,59,60, 62, 66,67,68, 70,71,72,73, 75, 77, 82, 83], galcanezumab 240 mg loading dose followed by 120 mg monthly (n = 21, 45.7%) [38,39,40, 47, 56, 58,59,60,61, 67, 69, 70, 74,75,76, 78,79,80,81,82], erenumab 140 mg monthly (n = 20, 43.5%) [38, 45, 47,48,49, 54, 56, 58,59,60, 62, 67, 69,70,71,72, 75,76,77, 82], erenumab 140 mg every 4 weeks (n = 11, 23.9%) [39, 43, 46, 50, 52, 53, 55,56,57, 63, 65], erenumab 70 mg every 4 weeks (n = 11, 23.9) [39, 41, 43, 46, 52, 53, 55,56,57, 64, 65], fremanezumab 225 mg monthly (n = 8, 17.4%) [44, 52, 56, 58, 60, 69, 70, 75] and fremanezumab 675 mg every three months (n = 3, 6.5%) [44, 56, 60]. There was a fixed starting dose for every participant in 31 studies (67%) [38, 41,42,43,44, 47, 50, 51, 53, 55, 58, 61,62,63,64,65,66,67,68,69, 71,72,73,74, 76, 78,79,80,81, 83], starting dose not fixed in 7 studies (15%) [46, 48, 49, 52, 54, 59, 60] and no information on this in 8 studies (17%) [39, 45, 56, 57, 70, 75, 77, 82]. Whether a dose increase was allowed was declared in 15 studies (32.6%) [41, 43, 46, 48, 53, 55, 58, 62, 65,66,67,68,69, 71, 72], with only 1 study utilizing a fixed dosing scheme [63]; 30 studies did not report if a dose increase was allowed [38,39,40, 42, 44, 45, 47, 49,50,51,52, 54, 56, 57, 59,60,61, 64, 70, 73,74,75,76,77,78,79,80,81,82,83]. Two studies reported that a dose decrease was allowed (4.3%) [41, 50], 1 study did not allow a dose decrease (2.2%) [62] and there was no information on this in the remaining 43 studies (93.5%) [38,39,40, 42,43,44,45,46,47,48,49, 51,52,53,54,55,56,57,58,59,60,61, 63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83]. The funding source for the drug treatment was mentioned in only 12 studies: public health care system (n = 6, 13%) [40, 50, 58, 59, 63, 83], pharmaceutical company (n = 4, 8.7%) [49, 53, 62, 76], patients themselves (n = 1, 2.2%) [77] and hospitals (n = 1, 2.2%) [79]; 34 studies (73.9%) did not mention the source of funding for the drugs [38, 39, 41,42,43,44,45,46,47,48, 51, 52, 54,55,56,57, 60, 61, 64,65,66,67,68,69,70,71,72,73,74,75, 78, 80,81,82].

Regarding concomitant migraine treatments, 21 studies allowed (45.7%) [41, 43, 44, 46, 47, 53, 58,59,60,61,62, 64, 66,67,68, 71, 72, 76, 77, 83] and 3 studies disallowed oral preventive medications (6.5%) [55, 69, 75]; no information on this was found in 22 studies (47.8%) [38,39,40, 42, 45, 48,49,50,51,52, 54, 56, 57, 63, 65, 70, 73, 74, 78,79,80, 82]. OnabotulinumtoxinA as concomitant therapy was allowed in 12 studies (26.1%) [41, 44, 46, 47, 58,59,60, 64, 72, 76, 77, 83] and disallowed in 3 studies (6.5%) [55, 61, 75]; no information on this was found in 30 studies (65.2%) [38,39,40, 42, 43, 45, 48,49,50,51,52,53,54, 56, 57, 62, 63, 65,66,67,68,69,70,71, 73, 74, 78,79,80,81,82]. In none of the studies information was found on concomitant use of transitional treatments or interventions (e.g. nerve blocks), neuromodulation or physical therapy.

There was no specific information on the management of MOH in 38 studies (82.6%) [38,39,40,41,42,43,44,45,46, 48, 50, 54,55,56,57,58,59,60,61, 64,65,66, 68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83], no intervention for MOH mentioned in 5 studies (15.6%) [47, 51,52,53, 63], education only in 1 study (3.1%) [49], education with inpatient withdrawal in 1 study (3.1%) [67] and education with outpatient withdrawal in 1 study (3.1%) [62].

Definitions

Ten studies (21.7%) defined a migraine day for the study (Table 3) [39, 44, 45, 54, 55, 58, 62, 69, 70, 76], while only 5 studies defined a headache day for the study (10.9%) [39, 45, 55, 62, 76]. One study defined moderate-to-severe headache day [63]. No study defined a migraine attack or headache attack.

Table 3 Study definitions of a migraine day (N = 10)
Full size table

Headache diaries

Thirty-four studies (73.9%) mentioned the use of headache diaries [38,39,40,41, 43,44,45, 47, 49, 50, 52,53,54,55, 58,59,60,61,62,63,64, 66, 67, 69,70,71,72, 74, 76, 77, 79,80,81,82,83]. We found no information on headache diaries in 12 studies (26.1%) [42, 46, 48, 51, 56, 57, 65, 68, 73, 75, 77, 78]. A baseline headache diary was required in 31 studies (67.4%) [39,40,41, 43,44,45, 47, 49, 50, 53,54,55, 58, 59, 61,62,63,64, 66, 67, 69,70,71,72, 74, 76, 79,80,81,82,83]. Of the 34 studies describing the use of headache diaries, 6 reported using electronic headache diaries (17.6%) [38, 39, 45, 47, 55, 76], 7 reported paper diaries (20.6%) [41, 44, 49, 58, 72, 74] and 21 studies did not specify the modalities of the headache diaries (61.8%) [40, 43, 52,53,54, 59,60,61,62,63,64, 66, 67, 69,70,71, 79,80,81,82,83]. Twenty-seven of the 34 studies mentioned recording the use of acute medications in the diaries (79.4%) [39,40,41, 43,44,45, 47, 49, 50, 53,54,55, 58, 59, 61,62,63, 66, 67, 69, 70, 74, 76, 80,81,82,83].

Baseline characteristics of participants

Age and sex were reported in all studies. Weight or BMI were reported in 12 studies (26.1%) [40,41,42,43,44, 57, 67, 79,80,81,82]; height only in 2 studies (4.3%) [73, 81]. Blood pressure was recorded at baseline in 2 studies only (4.3%) [62, 63]. Cardiovascular comorbidities were reported in 9 studies (19.6%) [41, 43, 48, 57, 67, 79,80,81, 83], gastro-intestinal comorbidities in 8 studies (17.4%) [40, 41, 43, 57, 67, 79,80,81] and psychiatric comorbidities in 18 studies [39,40,41, 43, 44, 53, 56, 57, 60, 61, 66,67,68, 76, 79,80,81, 83].

As for headache characteristics, 17 studies documented the age of headache onset (37.0%) [38, 40, 44, 45, 47,48,49, 53, 56, 61, 64, 67, 72, 74, 79, 80, 83]. Thirty studies reported the duration of CM (65.2%) [38,39,40,41,42,43,44,45,46,47, 49, 53, 54, 56, 58, 59, 61, 62, 65,66,67,68, 72,73,74, 76, 79,80,81, 83]. The presence of aura was reported in 15 studies (32.6%) [38, 39, 45,46,47, 49, 53, 58, 59, 62, 64, 69, 70, 76, 83], site of headache in 9 studies (19.6%) [39, 41, 43, 44, 47, 76, 79,80,81] and severity of pain in 19 studies (41.3%) [40,41,42,43,44, 47, 48, 54, 58, 59, 64,65,66, 74, 76, 79,80,81, 83]. Associated symptoms of headache were reported in 12 studies (26.1%) [40,41,42,43,44, 47, 48, 66, 76, 79,80,81], presence of premonitory symptoms in 3 studies (6.5%) [40, 41, 81] and presence of cranial autonomic symptoms in 7 studies (15.2%) [40, 41, 43, 44, 79,80,81].

Baseline information on response to onabotulinumtoxinA treatment for CM was reported in 14 studies (30.4%) [39, 41, 43, 44, 46, 47, 49, 57, 61,62,63, 66, 67, 80], and treatment response to triptans was documented in 10 studies (21.7%) [40,41,42,43,44, 47, 66, 79,80,81].

Outcomes, endpoints and objectives

The most common primary endpoint/objective of the studies was change in monthly migraine days (n = 16, 34.8%) [38, 41,42,43,44, 46, 54, 55, 58, 62, 69,70,71, 78, 80, 82], followed by responder rate (n = 15, 32.6%) [38, 39, 45, 49, 50, 53, 54, 59, 61, 64,65,66, 72, 79, 81] and change in monthly headache days (n = 5, 10.9%) [38, 44, 60, 74, 80]. Other primary endpoints or objectives defined by the researchers were model building (n = 3, 6.5%) [47, 56, 83], change in acute medication intake (n = 3, 6.5%) [38, 54, 71], change in validated questionnaire or scale score (n = 3, 6.5%) [59, 75, 77], change in pain intensity (n = 2, 4.3%) [54, 74], conversion from CM to EM (n = 1, 2.2%) [40] and conversion from MOH to non-MOH (n = 1, 2.2%) [67]. Eight studies did not specifically define the primary endpoint/objective in the paper (17.4%) [48, 51, 52, 57, 63, 68, 73, 76]. Five studies declared multiple primary endpoints/objectives (10.9%) [38, 54, 59, 71, 74].

Migraine days were used by 37 studies (80.4%) as any endpoint [38,39,40,41,42,43,44,45,46,47,48,49,50, 52,53,54,55,56,57,58,59, 62,63,64,65, 68,69,70, 72, 75,76,77,78, 80,81,82,83]; 31 studies (67.4%) used headache days as any study endpoint [38, 39, 41, 43,44,45,46,47,48,49,50, 52,53,54,55,56,57, 60,61,62,63,64, 66, 67, 69,70,71,72, 76, 80]. A detailed overview can be found in Table 4.

Table 4 Headache-related endpoints
Full size table

Thirty-nine studies (84.8%) presented responder rates to the drugs: 19 presented results on migraine days only (41.3%) [42, 46, 49, 50, 53,54,55, 58, 59, 62, 64, 65, 68, 70, 75, 78, 81,82,83], 9 on headache days only (19.6%) [52, 56, 60, 63, 66, 67, 71, 72, 74], 9 on both headache days and migraine days (19.6%) [38, 39, 41, 43,44,45, 47, 76, 80] and 2 on moderate to severe headache days (4.3%) [61, 79]. Fifteen studies used 1 percentage outcome only (14 studies with ≥ 50% responder rate [38, 39, 45,46,47, 49, 52, 53, 60, 65,66,67, 71, 82]; 1 study with ≥ 30% responder rate [75]), 4 studies with 2 percentage outcomes (≥ 50/75% responder rate in 3 studies [68, 74, 83]; ≥ 30/50% responder rate in 1 study [70]), 16 studies with 3 percentage outcomes (≥ 50/75/100% responder rate in 12 studies [41,42,43,44, 58, 59, 64, 76, 78,79,80,81]; ≥ 30/50/75 in 4 studies [50, 56, 63, 72]) and 4 studies with 4 percentage outcomes (≥ 30/50/75/100% responder rate in 5 studies [54, 55, 61, 62]).

No studies reported on onset of effect, cumulative hours per day of moderate to severe headache. One study (2.2%) evaluated the adherence to the treatment as one of the study’s endpoints [55]. No studies described pharmaco-economic endpoints.

Thirty-eight studies (82.6%) did not report using any anxiety/depression scales found in the list provided in the IHS guidelines. Two studies used 1 scale (4.3%) [53, 56], 4 used 2 scales (8.7%) [38, 47, 61, 76] and 2 studies used 3 scales (4.3%) [72, 74]. A detailed overview can be found in Table 5.

Table 5 Depression and anxiety scales
Full size table

Nine studies (19.6%) did not report using a scale on patient’s reported outcome measures or healthcare outcomes/quality of life provided in the guideline by the IHS [39, 40, 50, 54, 60, 63, 64, 70, 71]. Seventeen studies used 1 scale (37%) [41,42,43, 45, 49, 51, 52, 55, 56, 62, 65, 66, 68, 69, 73, 77, 82], 13 used 2 scales (28.3%) [44, 46, 47, 57,58,59, 61, 67, 78,79,80,81, 83], 2 studies used 3 scales (4.3%) [48, 75], 4 studies used 4 scales (8.7%) [38, 53, 74, 76] and one study used 5 scales (2.2%) [72]. A detailed overview can be found in Table 6.

Table 6 Patient’s reported outcome measures and healthcare outcomes/quality of life
Full size table

Biomarker collection

No study collected saliva or cerebrospinal fluid for analysis. One study collected blood serum samples to determine polymorphic variants of calcitonin receptor-like receptor and receptor activity modifying protein 1 genes [83].

Adverse events

Thirty-six studies reported adverse events (78.3%) [41, 43,44,45,46,47,48,49,50,51,52,53, 55, 58,59,60,61,62,63,64,65,66,67,68, 71,72,73,74, 76,77,78,79,80,81, 83]; 19 studies specifically reported on serious adverse events (41.3%) [41, 43, 45, 46, 50, 52, 62, 64,65,66,67,68, 71, 72, 74, 76,77,78, 83]. Twenty studies (43.5%) reported on reasons for discontinuation [41, 44, 46, 47, 49, 50, 55, 58, 61,62,63,64,65, 71, 72, 74, 77, 78, 80, 83]. Only 3 studies reported whether there were any pregnancies during treatment (6.5%) [49, 62, 76]. No study reported on ECG recording, neuroimaging or systematic blood pressure monitoring.

Quality assessment

The results of the overall quality rating by applying the NHLBI Study Quality Assessment Tool for Before-After Studies are as follows (Table 7). Thirty-four studies were graded as “good” quality [38,39,40,41, 43,44,45,46,47, 50, 52,53,54,55,56, 58,59,60,61, 63,64,65,66,67, 70, 71, 74,75,76, 79,80,81,82,83]. Twelve studies were graded as “fair” [41, 48, 49, 51, 57, 62, 68, 69, 72, 73, 77, 78]. The details of the quality assessment are available as an online supplement.

Table 7 Quality rating by applying the NHLBI study quality assessment tool for before-after studies
Full size table

Discussion

To our knowledge this is the first systematic review in the field of migraine to analyse the methodology of RWE studies on preventive treatments. We have specifically focused on prospective, observational, clinic-based studies with anti-CGRP pathway monoclonal antibodies for the preventive treatment of both EM and CM.

The majority of studies were deemed to be of “good” or “fair” quality based on the quality assessment tool. These studies help the scientific community to create proper insights and inferences on the efficacy, tolerability and safety parameters of anti-CGRP pathway monoclonal antibodies use in real-world clinical settings. We did however find rather large heterogeneity on multiple methodological aspects such as endpoint determination, key definitions and longitudinal data recordings (e.g. the use of headache diaries). Interesting observations for future development of RWE studies in migraine will be discussed below.

Definitions are important aspects of clinical trials to help the interpretability, reproducibility of results and comparison between studies. Unfortunately our conclusion from this systematic analysis is that the large majority of RWE studies analysed do not provide the audience with definitions on migraine days and headache days. Only 10 studies (21.7%) defined a migraine day. After analysing the wordings most of the definitions rely on the ICHD-3 criteria for a migraine attack, since ICHD-3 does not contain formal criteria for a migraine day. Interestingly enough, definitions for migraine day and moderate/severe headache day are available in the clinical trial guidelines of the IHS but these were rarely used in the RWE studies [30, 31]. The lack of a formal definition of a migraine and/or a headache day in ICHD is all the more important as the change in monthly migraine days, the change in monthly headache days and the responder rate (which itself is related to migraine and/or headache days) are typical primary endpoints of this type of real-world studies. Our recommendation for a new iteration of ICHD would be to include a formal definition of migraine day (and perhaps headache day) based on consensus within the headache expert community; alternatively the IHS may develop a guideline for the conductance of RWE studies.

Headache diaries are indispensable for clinical research of headache treatments [30, 31]. Almost three-quarters of studies in our systematic review used them, but still a non-negligible number of studies are not mentioning their use. What is interesting is that 61.8% of studies using headache diaries did not mention the modalities of use and only 6 studies used electronic diaries. Digitization of society provides opportunities as digital headache diaries limit the amount of recall bias and provide more structured data on headache/migraine days and acute medication intake. Therefore, our results show there is room for improvement of RWE studies in terms of the quality of the recording of migraine/headache days.

The enrolment process in prospective observational studies is different from randomized-controlled trials. Most studies relied on consecutive enrolment of participants from headache clinics. A minority of studies (10 studies, 21.7%) performed a sample size calculation. However, sample size calculations in RWE studies are recommended to produce reliable results and to improve generalizability of the study, and data from phase 2 or phase 3 studies are available to that end.

Defining the primary endpoint is crucial for medical research, but this was missing in almost 1 in 5 studies. The majority of primary endpoints chosen are distributed across studies looking at reductions in migraine or headache days versus responder rates in terms of migraine and/or headache days (e.g. 30 or 50% reduction). Currently the headache community supports both types of endpoints [30, 31].

Surprisingly, very little attention was given to cardiovascular parameters (including blood pressure monitoring and ECG) in the studies. CGRP is a highly potent vasodilator and may act as a vasodilatory safeguard during cerebral and cardiac ischemia. Post-marketing retrospective analysis and a recent prospective follow-up study revealed signals of elevated blood pressure after exposure to certain anti-CGRP pathway monoclonal antibodies, a phenomenon that was not observed in the pivotal randomized clinical trials [85, 86]. As the aim of RWD is not only to assess effectiveness but also to provide safety information, we expected higher number of studies investigating cardiovascular safety. One reason may be that almost all of the included RWE studies in this systematic review were performed in Europe where the Summary of Product Characteristics (SmPC) of anti-CGRP pathway monoclonal antibodies does not require monitoring cardiovascular parameters, including blood pressure. While risk of hypertension is a matter of debate, it may be recommended for future RWE studies to include at least blood pressure monitoring.

Thirty-eight studies (82.6%) did not report using an anxiety/depression scale as part of the patients’ assessment, but on the contrary only nine studies (19.6%) did not report using a scale on patient’s reported outcome measures or healthcare outcomes/quality of life, as suggested in the guideline by the IHS. Our impression is that researchers aim to quantify the burden of disease from the migraine disorder but refrain to specifically look into the dimension of anxiety and depression. However anxiety and depression are highly prevalent comorbid disorders of migraine and bring additional burden to the patient. The inclusion of anxiety and depression scales helps the fine-grained analysis of the patient cohort but also gives way to additional insights into the treatment effect of the investigational products on these comorbid conditions. We should note that e.g. treatment with onabotulinumtoxinA leads to a significant reduction of both CM severity and comorbid major depressive disorder [87]. It is recommended to use validated scales for anxiety and depression in future RWE studies.

Strengths of this systematic review were the systematic approach by multiple investigators to use predefined protocol, research questions, entry forms and analysis methodology to tackle all tasks. Our analysis was limited to prospective, non-randomized observational studies so no inferences on other forms of RWE studies (e.g. retrospective studies or case series) can be made.

Two additional remarks should be made. First, the IHS, led by an international collective of clinical and scientific headache experts, has published guidelines for clinic-based headache registries. The document stipulates the importance and value of good quality clinic-based data for a wide variety of purposes which may serve many actors in healthcare in decision making steps. It stresses the importance of a formal research protocol to collect data in the best way possible. The experts acknowledge that RWD from well-designed headache registries can provide wide-ranging and novel insights into the characteristics, burden, and treatment of headache disorders and ultimately lead to improvements in the management of patients with headache [88]. We greatly appreciate the new guidelines and we hope our analysis encourages further efforts to improve rigorous designs of real-world studies in the field of headache disorders. Secondly, prospective, observational, clinic-based studies can be excellent settings for the exploration and testing of new outcome variables or study hypotheses. Therefore, by presenting our results, we do not insist on a single standard for all prospective, observational, clinic-based studies with formalized parameters but welcome new approaches, as long as they have been classified a priori as exploratory variables/outcomes and they have received ethical approval before the start of the study and data collection.

Conclusion

This is the first systematic review on methodology of RWE studies in migraine, in particular regarding the preventive treatment with monoclonal antibodies against the CGRP pathway. We have identified multiple areas of potential improvement for future RWE studies, including the need for universal definitions of migraine/headaches, the use of (electronic) diaries, the calculation of sample sizes, and the use of anxiety and depression scales. In particular regarding CGRP pathway monoclonal antibodies systematic monitoring of blood pressure is recommended. We hope our analysis will be of benefit for future research, and ultimately patients with migraine.