The company provided an evidence submission to NICE that summarised the clinical and cost evidence for the Sherlock 3CG® TCS and presented a cost-consequence model . The aim is to evaluate whether the Sherlock 3CG® TCS carries a clinical advantage and/or a reduced cost in comparison to current NHS standard care.
Company’s submission of clinical effectiveness evidence
The company identified and submitted 4 relevant studies [9–12]. None of the submitted studies were peer reviewed or published in full. Two studies were from the USA, one from Australia and one from the UK. The company also identified a study that was pending publication , and subsequently published. Given the paucity of evidence, the EAC widened the scope to include any previous models of the technology that included both ECG measurement and magnetic tracking. This resulted in one presentation being included . The company also submitted information from questionnaires that it had sent to selected NHS users of the Sherlock 3CG® TCS.
Critique of clinical evidence available
Peer reviewed evidence
The only available peer reviewed paper reported a retrospective analysis of the first 250 PICC insertions following introduction of the Sherlock 3CG® TCS to an NHS intensive care unit (ICU) . PICCs were placed by the Vascular Access Team at the bedside and, following tip confirmation with the Sherlock 3CG® TCS, a portable X-ray was used to assess placement accuracy. Eleven out of 250 patients (4 %) were excluded due to difficulties interpreting the ECG (n = 4), other PICC difficulties (n = 3), or X-ray difficulties (n = 4). The authors reported 49 out of 239 (21 %) tip placements were not in the correct position using European guidelines (Pittiruti guideline).
A paper by the same group reported accuracy of blind bedside PICC placement shortly before the introduction of the Sherlock 3CG® TCS . The EAC has presented the results of both papers in Table 1, although the studies do not directly compare identical populations. The malposition rate using the Sherlock 3CG® TCS is significantly lower than the blind placement (p < 0.0001) and the authors conclude that if this definition of tip placement is acceptable, then the Sherlock 3CG® TCS can be used for tip confirmation without routine chest X-ray confirmation.
Results from non-peer-reviewed evidence such as abstracts, posters and presentations are summarised in Table 2.
The accuracy of the Sherlock 3CG® TCS in Johnston et al.  (Table 1) is lower than that reported in five other studies (Table 2); possibly because ICU patients have less clear heart rhythm or are recumbent. It is unclear if the results from this one study are generalisable to other settings (either to other ICU settings or the wider NHS).
Questionnaires completed by six NHS sites using the Sherlock 3CG® TCS highlighted the variation in normal clinical pathways for tip insertion and confirmation, repositioning and reinsertion. At the time of submission, the company reported that nine NHS sites had stopped using routine chest X-rays for tip confirmation, from a total of 16 NHS sites that have introduced the Sherlock 3CG® TCS.
All studies used the Sherlock 3CG® TCS as the intervention; none were comparative. The studies reported the use of the Sherlock 3CG® TCS for tip confirmation, followed by a chest X-ray to determine if the tip was in the correct place. There were no comparators giving information on the blind PICC placement malposition rates in the same setting. The studies all stop at the point of tip confirmation and assessment by chest X-ray. It is not known if PICCs identified as malpositioned had to be adjusted or re-inserted, or what the consequences might be. Similarly, the studies do not give any indication if any patients would have been identified as having misplaced PICCs without the chest X-ray, and what the clinical implications might have been.
Several of the sites from the published abstracts [9–12, 14] report that routine chest X-rays are no longer used to confirm PICC placement. The sponsors report that nine NHS sites using Sherlock 3CG® TCS have had sufficient confidence in the device to remove the requirement for routine chest X-rays to confirm tip location.
Two studies included cost information [9, 10], but contained insufficient information for critical analysis, and had limited relevance to the decision problem. The EAC did not find any additional studies that should have been included. Only two of the six clinical studies identified report the time taken to insert the PICC and complete tip confirmation using Sherlock 3CG® TCS (without chest X-ray), and compare it to the time taken by the previous method. There was no evidence for the effect on staff time, time to deliver treatment, bed days, clinical outcome or patient experience.
The company created an economic model with four branches:
Bedside PICC placement with Sherlock 3CG® TCS with X-ray.
Bedside PICC placement with Sherlock 3CG® TCS without X-ray.
Blind bedside PICC placement, with X-ray.
Fluoroscopy guided placement.
Inputs to the model were based on the studies and NHS questionnaires submitted as clinical evidence. Staff costs were derived from the Personal Social Services Research Unit (PSSRU) , and detailed pathway information and costs for blind bedside PICC placement and fluoroscopy were obtained from Walker et al. [17, 18].
Using this cost model, the company found that the Sherlock 3CG® TCS without X-ray gave a cost saving of GBP25.67 per patient compared to blind bedside PICC insertion, and a cost saving of GBP510.03 per patient compared to fluoroscopy. All costs are based on 2014 values.
Critique of Economic Submission
There are uncertainties in the model structure and inputs due to the lack of data available and the variations in patient groups and service provision. The company carried out extensive sensitivity analysis, although the EAC disagreed with some of the key inputs. Important assumptions and data for the model submitted by the company, along with the views of the EAC, are listed below.
The model stops at the point of confirming PICC tip placement. The EAC considered that a preferable time horizon would include adverse events, complications, the time to starting treatment and the clinical consequences. Unfortunately, there is no evidence available to inform such a model and the impact and direction of any effect is unknown.
The difference in staff time is one of the model’s key drivers. The two studies used in the company’s model were from different countries with different health systems, causing the EAC to consider comparability unlikely. The values used by the company were:
62.49 min for blind PICC insertion with X-ray and the Sherlock 3CG® TCS with X-ray, based on Walker and Todd  bedside placement time.
39.5 min for the Sherlock 3CG® TCS without X-ray .
Reinsertion Rates and Method
The difference in reinsertion rates and cost of reinsertion is a key driver when comparing blind bedside PICC insertion to the Sherlock 3CG® TCS. The proportion of accurate placement the model uses is:
96 % for the Sherlock 3CG® TCS  for trained staff. This study used a comparison to X-ray to judge accuracy. Where the normal pathway does not include a chest X-ray, a malposition will only be detected if it causes a clinical problem.
93.1 % for blind bedside PICC placement .
100 % for fluoroscopy .
The base case of the cost model assumes that all identified tip malpositions will require replacement, and that all reinsertions would be performed using fluoroscopy. Not all malpositions will require reinsertion procedures, and although some services may reinsert PICCs using fluoroscopy, others will use bedside techniques. Because the model includes more malpositions for blind PICC placement than for the Sherlock 3CG® TCS, and fluoroscopy is the most costly option, these assumptions favour the Sherlock 3CG® TCS.
The scope defined by NICE included all adult patients undergoing PICC insertion. The population included in the company’s model was adult patients undergoing PICC insertion who were suitable for ECG tip confirmation using the Sherlock 3CG® TCS. The company assumed 83.5 % of patients were suitable , and did not include costs for treatment of the remaining 16.5 % by an alternative method. This has a low impact on the model where the two comparative arms have similar costs. Expert advice indicated that the proportion of patients for whom the Sherlock 3CG® TCS is suitable was likely to be higher than 83.5 %. The company’s instructions for use state that use of the technology is limited (but not contraindicated) in patients who do not have an identifiable P wave, e.g. those with atrial fibrillation, tachycardia, etc.
The company’s model finds a large cost saving using the Sherlock 3CG® TCS at the bedside compared to fluoroscopy. This is partially due to the high cost attributed to fluoroscopy—much of the remaining cost saving is due to the move to bedside placement and would be equally true of blind PICC bedside placement.
The cost of theatre for fluoroscopy was assumed by the company to be GBP507.18 [17, 18]. The EAC estimated this cost to be GBP101 .
If standard practice was fluoroscopy, and a new bedside PICC placement service had to be set up, there would be costs for nurse training and reorganisation. The model does not include these initial set up costs.
EAC Alternative Scenarios
There are unavoidable uncertainties due to the lack of evidence, and the variations in patient groups and service provision. There are many inputs and scenarios that can change the model from being cost saving to cost incurring—the most accurate representation will depend on the service being examined. The EAC analysis presents an alternative set of assumptions—there is insufficient evidence to allow absolute certainty over which is most appropriate.
EAC Changes to Company’s Model
The EAC changes to the company’s base model were:
Nurse time is equal for blind PICC insertion with X-ray, the Sherlock 3CG® TCS with X-ray and the Sherlock 3CG® TCS without X-ray. The EAC used a time of 62.49 min . X-rays are costed separately.
Where the Sherlock 3CG® TCS is used without X-ray, there is an assumption of zero malpositions being identified within the time frame of the model, since no other tip confirmation system is used.
Replacements for PICCs placed at the bedside are by the same method as the original placement.
In patients for whom use of the Sherlock 3CG® TCS is not suitable, treatment costs by an alternative method were included.
The cost of theatre use for fluoroscopy was changed to GBP101 .
Using these inputs, the EAC found that the Sherlock 3CG® TCS without X-ray incurred a cost of GBP9.37 per patient compared to blind bedside PICC insertion, and gave a cost saving of GBP106.12 per patient compared to fluoroscopy. All costs are based on 2014 values.
EAC Cost Neutral Scenario
The EAC explored a scenario where the use of the Sherlock 3CG® TCS is approximately cost neutral when the following assumptions are made (in addition to the previous EAC changes):
Nurse time for the Sherlock 3CG® TCS is 57.5 min, a 5-min reduction from the original EAC assumption, as there is no requirement for the nurse to interpret the X-ray image.
The cost of X-ray is assumed to be 10 min of radiologist time (GBP5.67, company’s submission, based on Walker and Todd ) with the addition of 15 min for a Band 2 porter (GBP5.25 ) to transport patient to and from X-ray. This gives total X-ray cost of GBP10.92.
Using these inputs, the EAC found that the Sherlock 3CG® TCS without X-ray was associated with a cost saving of GBP1.17 per patient compared to blind bedside PICC insertion, and a cost saving of GBP111.27 per patient compared to fluoroscopy.
EAC Scenario Based on ICU Data
The EAC also explored a scenario based on ICU data [13, 15], changing the accuracy rates to those in Table 1. Sherlock 3CG® TCS with X-ray was used in the model, reflecting the local practice at the time of publication . The limitations of this scenario are:
the data is taken from a single centre and may not be generalisable.
the comparison is with historical data from the same centre.
the study only includes intermediate outcomes—the actual number of replacements is not reported.
In this scenario, the Sherlock 3CG® TCS with X-ray confirmation is associated with a cost saving of GBP41.35 per patient compared to blind bedside PICC insertion, because of the significant reduction in the proportion of malpositioned catheters that needed to be re-positioned.
Impact of Changes
Given current information, use of the Sherlock 3CG® TCS compared to blind PICC insertion using a chest X-ray appears to hover around cost neutral. Staff time and accuracy are key drivers in the model, but evidence for these is sparse and the reality will vary in different situations. If evidence became available for outcomes after the initial insertion, such as replacement, complications and adverse events, the cost implications may change. The direction of this potential change is not known.