On apples, oranges, and ARUBA
Arteriovenous malformations (AVMs) are congenital vascular lesions that are associated with long-term excess mortality and morbidity essentially related to haemorrhagic stroke . The prevalence of brain AVMS (BAVMs) is believed to be between 15 and 18 per 100,000 adults , and the incidence is estimated at 1 per 100,000 per year . Roughly half of the patients with BAVMs present with intracerebral haemorrhage (ICH), resulting in a first-ever hemorrhage rate of 0.55 per 100,000 person-years . The annual risk of haemorrhage is estimated at 1–4 % , but it may be as low as 0.9 % in patients with unruptured, superficially located brain AVMs with superficial drainage and may be as high as 34 % in patients with ruptured, deeply seated brain AVMs with deep venous drainage . The known risk factors for bleeding include AVM size, deep venous drainage, deep location, and associated aneurysm [4, 9, 20, 21]. BAVMs are commonly classified according to the five-tier Spetzler-Martin scale, which is a composite score of nidus size, eloquence of adjacent brain and presence of deep venous drainage , although, recently, a simplified three-tier classification has been proposed .
The increasing use of non-invasive diagnostic imaging for various indications has increased the incidence of unruptured BAVMs and inevitably raises the question of their curative treatment . The goals of BAVM treatment are to eliminate the risk of haemorrhage with the preservation of functional status , be it via microsurgery, endovascular embolisation or stereotactic radiosurgery (SRS). In order to obtain the former goal, the BAVM nidus must be obliterated, as subtotal treatment does not confer protection against future haemorrhage . However, the argument for a conservative approach for patients with unruptured BAVM is based on three factors, namely that the annual risk of ICH for patients with unruptured BAVM may have overestimated older studies , that the morbidity associated with BAVM bleeding may be less than previously thought , and thirdly, that the risks of treatment of unruptured BAVM may be greater than previously described .
The ARUBA trial
The A Randomized Trial of Unruptured Brain Arteriovenous Malformations (ARUBA) study seeks to determine whether or not the risks of treatment outweigh the risks of conservative management at 5 years for patients with unruptured BAVMs . The study was a prospective, multi-centre, parallel design, randomized, controlled trial. One hundred and four clinical sites in the U.S. & Europe were proposed as participating centres. The patients enrolled would be followed up for a minimum of 5 years and a maximum of 7.5 years (mean 6.25 years) from randomization (http://research.ncl.ac.uk/nctu/documents/ARUBA/Protocol.pdf).
The site selection was based on the following three criteria: centre experience with management of at least 10 brain AVMs per year, presence of a multidisciplinary AVM treatment team, and documented academic interest in clinical brain AVM research .
The original design approved by the NIH/NINDS was for 800 patients to be randomised. The statistical power analysis was based on an assumed 5-year event rate of 12 % in the medical management group and 22 % in the interventional therapy group. There was an intention-to-treat analysis and two interim analyses were planned. Cumulative stroke-free survival was estimated by the Kaplan-Meier method and Cox proportional-hazards regression models were used to estimate hazard ratios .
The primary hypothesis was that medical management improves the long-term outcome of patients with unruptured BAVMs compared to interventional therapy. The plan was to determine whether a conservative strategy would prove A) superior to, or B) non-inferior to a prophylactic “best possible eradication” of brain AVM (irrespective of whether the procedure was endovascular embolisation, SRS, or microsurgical resection alone or in any combination) using as a primary end-point a composite of death or symptomatic stroke. Furthermore, the ARUBA trial sought to determine whether a conservative strategy would prove superior to any therapeutic modality using a composite of death or disability by the modified Rankin Scale (mRS) score at 5 years as a secondary outcome measure . Patients with previous brain AVM haemorrhage or treatments and those considered untreatable for complete eradication were excluded from the study .
Initially planned for 800 patients to be included over a 30-month period, the recruitment period was lengthened to 60 months and the aim of patients accrued was reduced to 400 because of a slow randomization and accumulation of centres . Enrolment was halted after the second pre-planned interim analysis, when data for 223 patients were available and the predetermined threshold for safety/efficacy was met, as reviewed by the trial’s independent Data and Safety Monitoring Board .
During the period from April 4, 2007 to April 15, 2013, 226 patients were randomised. A further 1,514 patients were screened for eligibility, of which 1,014 were ineligible for enrolment, largely because of evidence of previous haemorrhage or a history of previous treatment. Of the 500 patients deemed eligible, 323 refused participation in the trial, and 117 patients were treated outside the randomisation process.
Of the 226 randomised patients, three were excluded because randomisation occurred after database lock, 114 were randomised to intervention, and 109 were randomised to conservative treatment. The mean follow-up was 33.3 months (IQR 16.3–49.8). The primary endpoint was reached in the interventional group (30 · 7 % [35 patients]) as compared with the medical management group (10 · 1 % [11 patients]), with a statistically significant difference . The cohorts were well-matched for characteristics such age, sex, mode of presentation, functional score, and AVM size, location, venous drainage pattern, and Spetzler-Martin grades. Approximately two out of three patients were in Spetzler-Martin grades I or II, with no patients in grade V.
The main finding of the ARUBA trial is that the risk of death or stroke was more than threefold increased after intervention (HR 3.70, 95 % CI 1.85–7.14) with a mean follow-up of about 33 months . Patients treated conservatively also had a significantly lower risk of death and neurological disability (mRS ≥2) . Untreated patients were found to have a spontaneous rupture rate of 2.2 % per year (95 % CI 0.9–4.5).
Of the proposed 104 centres, only one-third participated (39 centres). Of these 39 centres, 10 centres included only one patient, putting into question whether all of them really are high-volume centres, which the study states was the entry ticket into the study . Conversely, seven centres included more than half of the total number of patients, and two countries, namely France and Germany, alone included more than half of the patients. Is this problematic for external validity?
The physician accreditation to participate in ARUBA was that the department managed at least 10 AVMs yearly, irrespective of treatment modality (including conservative treatment). By comparison, in other large multicentre studies, the physician accreditations were more rigorous [2, 16]. Nevertheless, the problem is not so much the low total number of AVM patients treated at many of the participating centres, but rather that the number of each curative procedure per centre, be it microsurgical, endovascular or radiosurgical, was very low.
Many patients were treated outside the study, as only 226 of 1,740 (13 %) initially screened patients were randomised in the trial. According to the ARUBA protocol, site selection was based on "centre experience with management of at least 10 brain AVMs per year" . But, during a time period where each centre managed at least 60 patients, 10 centres (25 % of total) included only one patient or 1.7 % of all their AVM patients. Furthermore, 22 centres (56 % of total) included only 5 % or fewer of all their AVM patients seen during the study period.
The ARUBA study uses composite outcome measures, both as primary and secondary end-points. This is problematic, as death and minor stroke (ARUBA does not grade stroke) are not equivalent, but both count equally as events.
Of the 1,740 patients initially screened, 87 % were excluded. Exclusion was mainly because of evidence or history of previous haemorrhage or if a patient had received some mode of intervention. The exclusion criteria, therefore, eliminated patients with potentially more aggressive natural histories, incorporating selection bias into the study. However, such patients represent a large portion of those with unruptured AVMs in the community, thereby compromising the generalisability of the study’s findings. Furthermore, as the different AVM treatment modalities have progressed considerably during the last two decades [5, 8, 11, 21, 23], the threshold for proposing a prophylactic treatment of AVMs considered to harbour a high risk of rupture has been lowered, despite the results of different studies regarding the rupture risk factors [4, 6, 8, 9, 20]. Consequently, the patients included in ARUBA had possibly low-risk AVMs with a more favourable natural history. This limitation is similar to that encountered in the International Study of Unruptured Aneurysm , where the high-risk aneurysms were pre-selected out of the study, leaving an enrolled group of patients with low risk for rupture.
The event rate in the therapeutic group is exceedingly high. In fact, it is close to 50 % higher than expected in their statistical power analysis! The event rate in the interventional therapy, assumed to be 22 % after 5 years, was 31 % after 33 months. However, this is not thoroughly explained in the article. On the contrary, the issue seems to be obscured. The ARUBA article reads as follows: "The proportion of outcome events we identified for interventional therapy (30.7 %, as randomised) closely approximate interventional complication rates cited in a systematic meta-analysis (8) of brain arteriovenous malformation treatment for both bled and unbled lesions: surgery (29 %, range 1.5–54), embolisation (25 %, range 7.6–55), and radiotherapy (13 %, range 0–63)" . This passage is extremely problematic as the authors seem consciously to obscure the fact that the event rate in the therapeutic group is exceedingly high by stating complication rates “for both bled and unbled lesions”, whereas the ARUBA trial only deals with unbled lesions.
Reference 8 in the ARUBA paper is the 2011 JAMA article by van Beijnum and colleagues who did a meta-analysis of 142 cohorts containing 13,698 individuals with intracranial AVM who were followed up for 46,314 person-years . The authors compared the case fatality rates and haemorrhage risk per 100 person-years for the combined cohorts and individually for patients undergoing microsurgery (41 cohorts), radiosurgery (76 cohorts), intravascular embolisation (14 cohorts), and combined therapeutic modalities (11 cohorts). In this article, complications leading to permanent neurological deficits or death (i.e. similar to primary end-point of the ARUBA trial) occurred in 7.4 % of patients (range, 0–40 %) after microsurgery, 5.1 % (range, 0–21 %) after radiosurgery, and 6.6 % (range, 0–28 %) after embolisation of unbled BAVMs . These rates are far from that quoted in the ARUBA article!
Another major criticism of the ARUBA trial is that very different therapeutic modalities are lumped together. Again, referring to the 2011 JAMA article by van Beijnum et al., intracranial haemorrhage rates were 1.4 (95 % CI, 1.3–1.5) per 100 person-years overall, 0.18 (95 % CI, 0.10–0.30) after microsurgery, 1.7 (95 % CI, 1.5–1.8) after SRS, and 1.7 (95 % CI, 1.3–2.3) after embolisation of unbled BAVMs . The difference between microsurgery and the other two modalities is by a factor of 10! Continuing on the same track, successful brain AVM obliteration was achieved in 96 % (range, 0–100 %) of patients after microsurgery, 38 % (range, 0–75 %) after SRS, and 13 % (range, 0–94 %) after embolisation . Again there is an enormous heterogeneity between the treatment modalities. However, the ARUBA study did not have a statistical power to analyse any differences in the primary end-point with respect to treatment modality.
When the study was terminated, 53 patients randomised to intervention had ongoing treatment plans and 20 patients had not even initiated therapy. Thus, of the 114 patients were randomised to intervention, only 94 had actually been treated. Patients were treated by neurosurgery alone (n = 5), embolisation alone (n = 30), or SRS alone (n = 31), or using a multimodal approach combining embolisation with either neurosurgery (n = 12), SRS (n = 15), or both (n = 1) . This means that 76 of 94 patients, or 80 % of patients treated, received SRS and/or embolisation. This bias for treatment in the interventional therapy group towards single-method therapy with predominantly either endovascular embolisation (n = 30) or SRS (n = 31), is highly likely to have biased the trial in favour of medical management .
More than two-thirds of the ARUBA study patients were Spetzler-Martin grades I and II. The gold standard treatment of such BAVMs is microsurgery, usually as monotherapy. In the ARUBA trial, only five patients had microsurgery as monotharapy and only 12 patients had microsurgery in addition to other interventions . Consequently, the ARUBA trial is inadequate to draw any meaningful conclusions regarding the gold standard treatment for the majority of the BAVM patients.
The primary hypothesis of the ARUBA trial was that medical management improves long-term outcomes of patients with unruptured AVMs compared to intervention. However, is 5 years long-term? Although the use of a 5-year end-point is typical of prospective RCTs, what is the relevance of such a short time period to clinical decision making in a disease that may play out over 20–30 years? As one colleague stated rhetorically: Would you treat a patient with an asymptomatic AVM who was known to have only a 5-year life expectancy? .
The short follow-up in the ARUBA trial favours the medical management group because all complications occur early with interventional therapy. However, as the event rate was 31 % in the intervention group and 2 % in the medical group, the event-free survival curves will cross after 15 years or so, i.e. the clinical outcome after 15 years should be similar in the two groups. But that begs the question of what to do after that. Continue to wait?
The hypothesis that the event-free survival curves will cross after 15 years or so is, however, conditional and based on a presumption that all the treated BAVMs have been obliterated. If, on the other hand, many were treated incompletely, the long-term event rate in the treatment arm will not be 0 % so that even after 20 years of follow-up, the lines never cross. Regretfully, no information on obliteration rates in the ARUBA trial is given .
Because of the premature interruption and result analysis only at 33 months after inclusion, the trial’s independent Data and Safety Monitoring Board required the ARUBA investigators to continue monitoring the 223 enrolled patients for up to 10 years . This is an appropriate response, as the long-term follow-up results could show a diminishing difference in outcome between the two groups over time, similar to that seen in the ISAT publication . However, the premature interruption decreases the power of this randomized study and limits the generalization of the results .
- The ARUBA study compares no treatment with a pool of treatments that are extremely heterogeneous with respect to obliteration rates (Microsurgery 96 % vs. SRS 38 % vs. Embolisation 13 %) , timing of effect (Microsurgery + Embolisation short/instant vs. SRS 2–3 years), and haemorrhage rate per 100 patient-years after therapy (Microsurgery 0.18 % vs. SRS/Embolisation 1.7 %) . Incidentally, no information on obliteration rates is given .
- The ARUBA study tells us that AVM treatment comes at a price and that not doing anything is less harmful than doing something - in the short term. However, only long-term follow-up can establish whether the disparity in outcome will persist or change, for which the ARUBA study is not designed .
- In the ARUBA study, the complication rates of any treatment were very high: 50 % higher than expected by the investigators and 5–6 times that found in a recent meta-analysis totalling 13,698 patients and 46,314 follow-up patient-years . This is not properly accounted for.
- The ARUBA study essentially compares radiotherapy and embolisation (76 of 94 patients or 80 % of patients treated) of unruptured BAVMs to conservative treatment . These modalities represent alternatives to the gold standard. To use them as the major form of treatment, skews the results towards suboptimal nidus obliteration, and ultimately biases the conservative management group.
- In the ARUBA study more than two-thirds (68 %) had Spetzler-Martin grades of I or II. The ARUBA study is inadequate to draw any meaningful conclusions regarding the gold standard treatment of AVMs of Spetzler-Martin grades I and II, namely microsurgery as monotherapy (5 % of patients treated).
Conflict of interest