Clinical Effectiveness Evidence
Company’s Review of Clinical Effectiveness Evidence
The company undertook a comprehensive literature search, which identified a relatively large volume of evidence within the scope, restricted to English-language papers and English-language abstracts, as per MTEP procedure. The company included 16 publications presenting data from randomised studies comparing TURis with mTURP [6, 11–25]. Of these, four publications were conference abstracts from a single study [16–19] and two were Spanish-language [11] or German-language [25] full papers with English-language abstracts.
The company also included publications from observational studies [26–33]. Of these, three were full papers [29, 31, 33] and five were conference abstracts [26–28, 30, 32]. All but two observational studies [27, 29] had a comparator for TURis, most often mTURP, but sometimes including other surgical procedures.
The company undertook meta-analyses of randomised trials (TURis versus mTURP) to present the clinical evidence relevant to TURis and did not place great emphasis on the data from the observational studies. The company included a meta-analysis of clot retention, an outcome not specified in the scope [8].
Critique of Company’s Clinical Evidence Submission
The EAC performed an independent literature search and identified all of the evidence provided by the company, plus three additional studies published as full papers: two randomised trials comparing TURis with mTURP [34, 35] and one observational study of TURis [36]. The EAC agreed with the company’s focus on meta-analysis of randomised studies and, having also reviewed data from the observational studies, concluded that these did not significantly add to the evaluation of TURis. The EAC considered that eight randomised trials (published as 13 papers [6, 12–15, 20–25, 34, 35]) were eligible for inclusion in its analysis. Table 1 presents the characteristics and published papers of each randomised study included in the analysis by the EAC, and Table 2 presents the studies excluded by the EAC.
Table 1 Randomised studies of transurethral resection in saline (TURis) versus monopolar transurethral resection of prostate (mTURP) included in the analysis by the External Assessment Centre (EAC)
Table 2 Randomised studies of transurethral resection in saline (TURis) versus monopolar transurethral resection of prostate (mTURP) excluded from the analysis by the External Assessment Centre (EAC)
The eight randomised trials (Table 1) all presented data on the patient group and the comparison specified in the scope. None of the studies were undertaken in the UK. Most studies were similar in terms of baseline prostate size, which ranged typically from 45 to 60 g. The first of two studies from the same team in China had the largest baseline prostate size of 78 g [13]. This study also had the longest procedure times for both TURis (88 min) and mTURP (105 min) [13], whereas most of the other studies had average procedure times of <60 min.
The study sample sizes ranged from 40 to 550 subjects. Many studies did not define a primary outcome measure. Only three studies stated that patients were blinded to treatment allocation [13, 14, 34]. Only one study stated that assessors of outcome were blinded to allocation status and that a sample size calculation was performed [34]. It is therefore unknown whether most of the studies had adequate statistical power to detect important differences in many of the outcome measures. Five studies reported the method used to randomly allocate subjects to treatment [6, 14, 15, 20, 21, 23, 29, 34]. No study reported that an intention-to-treat analysis was performed. Therefore, the eight included randomised trials [6, 12–15, 20–25, 34, 35] carried some risk of bias but represented a substantial volume of evidence of reasonable quality to inform the evaluation.
Table 3 summarises the results of the company’s meta-analyses. The company’s analysis did not demonstrate that TURis statistically significantly reduced the risk of TUR syndrome (relative risk [RR] 0.28 [95 % confidence interval (CI) 0.08–1.02]) or clot retention (RR 0.63 [95 % CI 0.21–1.90]), though the risk of blood transfusion was substantially reduced by TURis (RR 0.36 [95 % CI 0.16–0.80]). The company found that TURis did not significantly shorten the procedure time (mean difference −1.68 [95 % CI −4.18–0.8]) minutes) but concluded that TURis shortened the time to catheter removal (mean difference −0.23 [95 % CI −0.38 to −0.08] days) and also the hospital stay (mean difference −0.52 [95 % CI −0.74 to −0.30] days).
Table 3 Summary of meta-analyses of randomised trials conducted by the company and by the External Assessment Centre (EAC)
Additional Work Carried Out by the External Assessment Centre
The EAC studied the methodology of the company’s meta-analysis and reproduced the analyses with checks or modifications as follows:
-
Adding data from additional randomised trials identified by the EAC [34, 35].
-
Removing data that were duplicated in the company’s analysis, because of repeat publication [20, 23].
-
Excluding non-peer-reviewed data available only in abstracts, which did not enable critical appraisal [16–19].
-
Obtaining confirmation from a lead author that two randomised studies conducted at the same centre were separate patient samples [13, 14].
-
Determining whether the data from non-English-language papers [11, 25] were pivotal to the analyses (MTEP procedure is to include such data only when this criterion is met, and with translation by a proper agency), with translation of one paper into English [25].
-
Correcting data entry errors.
-
For outcomes expressed as RRs, excluding studies with zero events in both study arms, as this precluded RR calculation [11, 14, 25].
-
Excluding from the meta-analyses one outlying study conducted in China, which generated heterogeneity for two outcomes (hospital stay and time to catheter removal), because the EAC considered that both outcomes are driven by local practice, which may differ between healthcare providers [13].
The results of the EAC’s meta-analyses are summarised in Table 3, alongside those of the company.
The EAC analysis found that TURis significantly reduced the risk of TUR syndrome (RR 0.18 [95 % CI 0.05–0.62]) and suggested that one case of TUR syndrome was prevented for every 50 patients treated with TURis. The EAC found that the risk of blood transfusion was significantly reduced (RR 0.35 [95 % CI 0.19–0.65]), suggesting that one case of transfusion was prevented for every 20 patients treated with TURis. Like the company, the EAC found no statistically significant reduction in clot retention (RR 0.55 [95 % CI 0.26–1.15]) and virtually no difference in procedure time (mean difference −1.36 [95 % CI −3.70 to 0.98] min) or time to removal of the catheter (mean difference −0.09 [95 % CI −0.25 to 0.06] days). In contrast to the company’s analysis, the EAC’s analysis found no substantial reduction in the hospital stay through the use of TURis (mean difference −0.19 [95 % CI −0.46 to 0.07] days).
The remaining outcome measures specified in the scope were readmission for repeat procedures, healthcare-associated infection, quality of life and device-related adverse events.
The EAC performed a meta-analysis of the rate of repeat procedures due to incomplete resection (Table 3) and found no significant difference between TURis and mTURP (RR 0.76 [95 % CI 0.42–1.40]). The EAC also undertook a meta-analysis of readmission due to haemorrhage (Table 3) and found little difference between the groups (RR 0.53 [95 % CI 0.22–1.25]).
The EAC recorded data on infection, where available from studies, and concluded that there was little difference in infection rates between TURis and mTURP. Likewise, studies that reported either quality of life or functional urological measures after treatment suggested that TURis and mTURP were equivalent [12–15, 29, 34].
Because of a concern over higher energy settings used in TURis, the EAC undertook meta-analyses of the longer-term complications of urethral stricture and bladder neck contracture, and found no difference in risk between TURis and mTURP (Table 3).
The company identified four adverse events from the US Food and Drug Administration’s Manufacturer and User Facility Device Experience (MAUDE) database [37] and the Medicines and Healthcare Products Regulatory Agency (MHRA) database [38]. The EAC identified an additional 13 adverse events from the same sources, but these sources were prone to either duplication or under-reporting of events, and the EAC could not establish that every adverse event was related to TURis. The commonest adverse event was breakage or degeneration of the electrode. Other events were failure to coagulate, urethral burns, bladder rupture and air embolism leading to cardiac arrest. Adverse events should be considered in the context of those that arise during mTURP.
Economic Evidence
Company’s Economic Submission
The company identified three economic studies [39–41], and the EAC identified one additional study, which included an economic estimation [36], but these were not used as evidence for TURis by the company or by the EAC, because of low applicability to the scope.
The company provided a de novo economic model in the form of a decision tree with an NHS perspective and 2013 prices. The model matched the scope in terms of the population (men with LUTS secondary to BPE in whom surgical intervention is indicated), intervention (TURis) and comparator (mTURP). Patients entering the model were treated either with TURis or with mTURP. The following complications were included in the base case: TUR syndrome and blood transfusion. The time horizon of the model was not defined, but it was designed to capture early surgical complications.
No capital cost for mTURP was included, since mTURP capital equipment was assumed to be already in place under standard care. For TURis capital costs, the model considered existing Olympus customers and non-Olympus customers independently, since new customers would require more new equipment, assumed to be three each of a telescope, light guide, inner sheath and outer sheath (total £26,715). Existing Olympus customers had some components already, so their capital cost was £8,800. The model did not consider the capital cost of the generator in any instance, because generators are supplied to customers free of charge as part of a contract to buy a volume of consumables. The company assumed that three sets of TURis capital equipment (excluding the generator) would suffice, enabling up to three TURis operations to be carried out per session, but no more, because the equipment needs to be cleaned before re-use. A discount rate of 3.5 % was applied to the capital equipment cost of TURis beyond the first year.
In addition to the base case, the company included three optional scenarios in the model:
-
1.
Considering the cost of readmission due to clot retention.
-
2.
Including re-operations due to the initial procedure being terminated prior to completion.
-
3.
Assuming that TURis reduces the hospital stay by 1 day in comparison with mTURP.
The base case inputs for the model were drawn from several sources. Clinical parameters were drawn from the company’s meta-analysis. The difference in hospital stay between TURis and mTURP was taken from the company’s meta-analysis, but the mean length of stay for mTURP was taken from hospital episode statistics (HES) data for 2012–2013 [42]. The additional resources required for treating patients with TUR syndrome, as used by the company, were based on a 2-day stay in a high-dependency unit, followed by a 2-day stay in a general ward, utilising the national schedule of reference costs for 2012–2013 [43, 44]. The resources required for a blood transfusion in the company’s model were taken from a published study [45]. The cost of re-operation due to the initial procedure being terminated prior to completion was calculated in the company’s model as the cost of consumables plus the cost of the hospital stay.
The company’s base case analysis showed that TURis is cost saving in comparison with mTURP. For existing Olympus customers, the saving per case was £133.63, and for non-Olympus customers, it was £114.19. The company conducted one-way deterministic sensitivity analysis for ten input variables and found that TURis remained cost saving across the range of values tested. The company also performed probabilistic sensitivity analysis, whereby each model parameter was assigned a statistical distribution and the model was run for 1000 simulations, by randomly sampling the distributions and calculating the results of the model each time. TURis remained cost saving in almost all of the simulations.
Critique of Economic Evidence
The key drivers of the company’s model were the reduction in the hospital stay for TURis patients in comparison with mTURP, the cost of a bed-day and the cost of mTURP consumables. The reduction in the hospital stay of 0.52 days was based on the company’s meta-analysis, in which a single study [13] introduced heterogeneity into the data. After removing this study, the EAC found a difference in the hospital stay of 0.19 days (p = 0.16) and considered this difference to be small in magnitude and not statistically significant. However the committee considered that the point estimate of a reduction in the hospital stay of 0.19 days should be used in the model.
The cost of mTURP consumables was estimated by the company to be 50 % of the cost of TURis consumables (£80.57 per case). The EAC considered that for a key driver of the model, it is more appropriate to use an accurate cost where this is available.
The EAC considered that the company had overestimated the cost of blood transfusion, by including several blood products (red blood cells, plasma, platelets and cryoprecipitate). Clinical advisers confirmed that the EAC proposal of 2.7 units of red blood cells alone was a reasonable estimate for transfusion for mTURP patients.
The EAC considered that the company’s model did not consider the case where non-Olympus customers perform mTURP with their own capital equipment but sourcing cheaper mTURP consumables independently via NHS Supply Chain. The EAC modified the model to explore this scenario.
External Assessment Centre’s Revisions of the Company’s Economic Model
The four most significant modifications made to the company’s model by the EAC were:
-
1.
Changing the cost of mTURP consumables in the base case.
-
2.
Changing the reduction in hospital stay in the base case.
-
3.
Changing the cost of blood transfusion in the base case.
-
4.
Modelling an additional scenario based on limited evidence [15] of a reduction in readmission (due to any cause) following TURis.
These are explained as follows. For existing Olympus customers, the EAC changed the cost of mTURP consumables from £80.57 to £137.75 per case on the basis of Olympus price list prices. For non-Olympus customers independently sourcing consumables for mTURP, the consumables cost per case was changed from £80.57 to £56.84 on the basis of NHS Supply Chain prices. Clinical experts confirmed that this was realistic. The EAC changed the reduction in the hospital stay in favour of TURis from 0.52 days to 0.19 days on the basis of its own meta-analysis (Table 3) and at the request of the committee. The EAC changed the cost of blood transfusion from £920.40 to £329 to reflect the cost of 2.7 units of packed red cells [45].
On the basis of its own meta-analyses, the EAC also made minor modifications to the risk of TUR syndrome in mTURP cases and the likelihood of blood transfusion (Table 3).
On this basis, the EAC base case analysis found TURis to be cost saving at £70.55 per case for Olympus customers and cost incurring at £19.80 per case for non-Olympus customers.
The EAC’s additional scenario considered the only available randomised trial [15] that reported rates of readmission (due to any cause) following TURis (5.1 %) and mTURP (16.1 %). The platform for this analysis was the company’s scenario of readmission due to clot retention, and the EAC used the company’s estimated cost of readmission of £2781, based on an NHS reference cost for admission with urological complications [43]. There is uncertainty regarding this cost, which may not be accurate for all causes of readmission. The effect on the EAC’s base case was that TURis was strongly cost saving by £375.02 per case for Olympus customers and £284.66 per case for non-Olympus customers. This result must be treated with caution because of the uncertainty of the modelled cost and because the rates of readmission due to any cause were based on just one randomised trial [15] and were not reported in the other randomised trials [12–14, 20–25, 34, 35].