Background

Carbapenemase-producing Enterobacteriaceae (CPE), Gram-negative bacteria with resistance to carbapenem antibiotics, are an increasing problem world-wide [1,2,3]. Carbapenems are powerful broad spectrum antibiotics and one of the last effective treatments for multi-resistant bacteria [4]. In some European countries, CPE are endemic, particularly in southern countries such as Greece and Italy [5, 6]. Evidence suggests that hospitalisation, especially when occurring in countries with high numbers of patients colonised or infected with CPE, is a key risk factor for CPE spread and associated deaths [7,8,9]. The United Kingdom has seen a continual increase in incidence of detected colonisations and infections since 2008 with fewer than five confirmed cases between 2004 to 2007 followed by 23 in 2008, 73 in 2009, 333 in 2010, and 561 in 2011 [5].

In December 2013, as part of the response to a number of CPE outbreaks in acute hospitals in England, Public Health England (PHE) published a guidance document, the “Acute trust toolkit for the early detection, management and control of carbapenemase-producing Enterobacteriaceae” and hereafter referred to as the CPE toolkit [10]. Acute trusts are organisational structures of the English National Health Service (NHS), which manage one or more acute care hospitals (secondary and tertiary) with a single management board. The CPE toolkit was produced to offer guidance on CPE, transmission risk factors, early recognition and prevention of transmission to inform the development of trust-specific plans. A literature search was conducted to identify and review relevant publications, documents and guidelines focusing on lessons learned from countries that had experienced CPE outbreaks previously. A question and answer section of the toolkit was based on real questions asked by health care professionals as well as further anticipated questions. The CPE toolkit was formally launched in March 2014 with an NHS England Patient Safety Alert requiring acute trusts to have a local CPE plan in place by June 2014 [11].

The aim of this implementation evaluation was to investigate the awareness of the CPE toolkit amongst frontline staff with specific infection prevention and control responsibilities (IPC), such as IPC nurses or microbiologists, as well as non-IPC specialist frontline staff, including nurses and doctors. Further, the implementation of procedures recommended by the CPE toolkit and the processes by which the CPE toolkit may have influenced clinical practice were examined [12, 13]. Implementation evaluations are particularly important for interventions like the CPE toolkit, which facilitate adaptation to the local context [14, 15]. By encouraging the development of local plans implemented process may vary between trusts, for instance use of risk-based screening of all patients versus limited screening for specific wards. Examining contextual influences, differences between the recommended and implemented processes and identifying key implementation facilitators and barriers from the perspective of frontline staff should help improve and strengthen future versions of the CPE toolkit and inform other policies.

Methods

Implementation evaluation frameworks

The Behaviour Change Wheel (BCW), a comprehensive framework for developing and evaluating behaviour change interventions, was used to theoretically inform the evaluation [16]. It describes seven broad policy level options through which behaviour can be influenced: fiscal measures, regulations, service provision, legislation, communication and marketing, environmental and social planning, and guidelines. To target capabilities, opportunities and motivation to perform behaviours (COM-B model), policy interventions support specific intervention functions, for example education, persuasion, incentivisation, coercion, training, restriction and environmental restructuring. Mapping onto the COM-B model, the Theoretical Domains Framework (TDF) breaks down the capability, opportunity and motivation domains further, outlining 14 areas of influence on professional behaviour, for example improving skills, changing beliefs about consequences or social influences [17, 18]. We examined which influences were relevant to the CPE toolkit implementation and what behaviour change pathways applied according to frontline staff. The evaluation built on empirical findings from a 2016 online survey of executive acute trust staff on organisational level perceptions of the CPE toolkit implementation, which suggested awareness of the CPE toolkit, but a lack of confidence in its practical value [19]. Additional file 1 illustrates how the evaluation frameworks informed the following research process.

Design

A qualitative design was used to gain a contextual understanding of CPE awareness amongst frontline staff and their experiences of the CPE toolkit implementation [14, 20, 21]. The evaluation protocol was reviewed by the University of Bristol Research Ethics Committee, classifying this health service evaluation as an audit of existing NHS procedures [22].

Setting

The evaluation was conducted in acute hospital trusts in England, the setting for which the CPE toolkit was developed. At the time of this research, there were 154 English acute trusts. Trusts that had given their consent to be approached for further evaluation activities (59 of 99 respondents) in the prior online survey [19] were purposively sampled to ensure a diverse sample [23,24,25]. The sampling frame focused on capturing all potential combinations of two main sampling criteria, CPE colonisation rates and time point of introducing local CPE plans, and on including large, medium and small trusts from each commissioning region. All trusts had local CPE plans in place, but the time point of initial introduction varied from prior to the CPE toolkit publication, as early as 2014, to late adoptions (2015/16). CPE colonisation, a precursor for infection [26], correlated with CPE infection rates (χ2 = 116.49; p < 0.001) and ranged from no cases, to some (1–10), and many CPE colonisation rates (> = 11) [19].

A personal access strategy [27, 28] was employed, gaining access through gatekeepers such as the local infection prevention and control teams (IPCT) and antimicrobial resistance (AMR) leads or through executive members of staff that participated in the survey. Following a maximum variation sampling approach to explore different views and experiences and to maximise the chance of discovering differences between staff with a range of roles and responsibilities [29,30,31], gatekeepers identified interview participants from two staff groups targeted by the CPE toolkit: 1) staff with specialist IPC responsibilities (e.g. IPC directors, lead IPC nurses or consultant microbiologists) as key informants [32], hereafter called IPC frontline staff, and 2) non-IPC frontline staff (e.g. staff nurses, ward managers and doctors) caring for patients, hereafter called general frontline staff. Settings included low CPE risk but high patient throughput wards, such as admission and acute medicine, as well as specialised, high CPE risk wards, like intensive care and surgical wards. Depending on trust size and speciality, the number of staff available for interviews varied, ranging from two to five participants per trust (Additional file 2) with at least one person from each group (Table 1). Out of the 14 trusts approached, 12 agreed to participate and 44 interviews were conducted (Table 2). Inability to spare staff time was given as reason for non-participation.

Table 1 Interviewed staff characteristics
Full size table
Table 2 Participating trusts’ characteristics
Full size table

Interview procedures

An interview topic guide (Additional file 3) was developed by the multidisciplinary research team, piloted with participants at two acute trusts and iteratively adapted over the course of the interviews as new insights emerged [21, 33, 34]. Semi-structured interviews ensured detailed information relevant to the evaluation questions were captured while allowing participants to raise additional topics of importance [24, 25]. Participants were asked to reflect on any ‘significant changes’ [35, 36] concerning CPE prevention, management and control since 2014 when the CPE toolkit was launched nationally. Questioning not referring explicitly to the CPE toolkit initially was aimed at identifying changes in the wider context and how much importance participants ascribed to the toolkit, if they were aware of it. Aspects related to potential behaviour change pathways and context factors were explored with questions based on the TDF [18]. As visual stimulus to prompt further insight into participants’ experiences and views on recommended procedures [14, 37], participants were asked to comment on the patient management steps specified in a flowchart included in the CPE toolkit. Interviews were conducted in accordance with data protection and confidentiality regulations, by briefing participants and obtaining informed consent. As this evaluation was commissioned by the same group at PHE who commissioned the CPE toolkit, interviewers emphasised their independence of the evaluation group to counter response biases. Depending on participants’ preferences and availability, interviews were conducted between April and August 2017 either face-to-face during one-day site visits or by telephone over a maximum period of one month (Table 1). Both interviewers (AS, CC) were women with a PhD in health science (health psychology, epidemiology) with prior qualitative research experience. Interview times ranged from nine to 108 min with an average length of 26 min. The average interview length was similar for face-to-face (27 min) and telephone interviews (24 min).

Analysis

The interviews were digitally recorded on an encrypted recording device and transcribed verbatim. The transcripts were imported into NVivo Version 12 [38] and analysed following thematic analysis steps [39] using the constant comparison technique, coding the material line-by-line [40]. To ensure reliability and validity of the analysis [21, 30, 41], the first five transcripts were independently double-coded (AS, CC). Potential discrepancies were discussed to reach consensus and a coding framework was developed using a hybrid approach [42], combining data-driven inductive codes and deductively developing categories based on the Medical Research Council process evaluation guidelines [12] and procedures outlined by the CPE toolkit [10]. The first author applied the agreed coding framework to the remaining transcripts. Themes were converted into thematic networks [43], mapping implementation barriers and facilitators according to COM-B model categories for behavioural influences [16]. Further procedures employed to ensure quality of the data analysis included discussions of findings within the multidisciplinary team, memo writing and keeping reflective and decision logs throughout the data collection and analysis phase, grounding findings in the data through anonymised participant quotes and transparent reporting in line with recognised standards [25, 44].

Results

Table 3 summaries results concerning awareness and reach of the CPE toolkit, the implementation and impact pathway and views, responses and improvement suggestions, followed by corresponding participant quotes.

Table 3 CPE toolkit implementation overview
Full size table

Awareness and reach of the CPE toolkit

The extent to which frontline staff came into direct contact with the published CPE toolkit varied by staff group. All interviewed IPC frontline staff were aware of the CPE toolkit. This was rarely the case amongst general frontline staff (10%) and the few who recalled seeing it, acknowledged that they had not read it. General frontline staff, which referred directly to the CPE toolkit, usually did so in the context of CPE outbreaks. IPC frontline staff often could not remember how they initially heard about the CPE toolkit. If they did, this was typically through their professional networks and PHE mailing lists.

CPE toolkit implementation and impact pathway

All trusts had local CPE plans in place and participants described CPE prevention, management and control procedures that aligned with the CPE toolkit guidance as much as possible considering local circumstances. Figure 1 provides an illustration of the role of the CPE toolkit in the typical development and implementation process of local CPE plans.

Fig. 1
figure 1

Typical CPE toolkit impact pathway

Full size image

The process started with encountering CPE, either within their trust or becoming aware that it was a problem at other, particularly neighbouring trusts. CPE proximity seemed crucial in influencing the implementation process by raising awareness of CPE as a problem (82%). In response, IPC frontline staff would use the CPE toolkit to inform trust practice and translate it into local procedures and policies, making amendments to accommodate their circumstances and demands. Examples of local adaptations were enhancement of CPE precautions for high CPE colonisation rates or CPE outbreaks and large-scale patient cohorting at CPE isolation wards in response to insufficient isolation facilities. The three trusts that had local plans in place before the CPE toolkit was published reported their own hospital CPE outbreaks as triggers for developing CPE policies in the absence of national guidance. When the CPE toolkit was published, they referred to it to check if their policies aligned with recommended procedures, but did not necessarily find them useful.

The process of developing local CPE plans was often described as an interdisciplinary effort. Considering multiple perspectives helped to engage different staff groups and aimed to generate policies that were relevant, workable and acceptable for users. Communication channels for local CPE plans varied, including word of mouth, meetings and training sessions, intranet communications and written information leaflets and checklists. There was awareness that the implementation of procedures recommended in the CPE toolkit or locally-relevant adaptations promoted a general confidence in CPE prevention, management and control capabilities. This, however, did not necessarily co-exist with an in-depth knowledge of CPE, even in hospitals that experienced outbreaks.

The described process (Fig. 1) was iterative in nature without an ultimate endpoint, as responding to implementation experiences and situational factors (e.g. CPE outbreaks, availability of new evidence) local plans and procedures continued to change. Overall, developing local CPE plans was seen as less of a challenge than their implementation.

Implementation facilitators and barriers

Table 4 provides an overview of factors mentioned by participants that acted as facilitators and/or barriers to implementing recommended CPE prevention, management and control procedures. They are grouped under the most relevant COM-B model component and illustrated by verbatim quotations. Factors influencing implementation often interacted as staff capabilities (e.g. knowledge and skills) and opportunities (e.g. social influences and available resources) had an impact on their motivation (e.g. beliefs and intentions) to engage in target behaviours.

Table 4 Factors influencing the implementation of recommended procedures
Full size table

Due to the novelty of the topic, a lack of awareness and knowledge especially amongst general frontline staff prevailed. Expert CPE knowledge went hand-in-hand with concerns about resulting risks and increased efforts to deal with CPE. Uncertainties about CPE risk factors and its management as well as receiving too much and too detailed information were problematic. Implementation was made easier, if new processes such as screening procedures were added to already existing standard operational procedures and routines. One way of doing this was making CPE part of a wider IPC plan. This however posed the risk of not paying enough attention to CPE but prioritising other more pressing issues.

Contextual factors, including resources and social influences, were overall key to implementation. Participants recognised multidisciplinary, trust-wide and across-trusts efforts and knowledge sharing as facilitators, even if these slowed down the process since involving various stakeholders took time. This approach promoted communication of relevant patient information and helped to creatively deal with important barriers, such as insufficient numbers of isolation facilities. A problem mentioned was communication and collaboration with community care providers, where anxieties and a lack of CPE knowledge hampered the discharge of colonised patients.

Further facilitative social opportunity factors included the presence of dedicated IPC staff and support from supervisors, modelling good practice, and trust management support. The CPE toolkit was seen as a credible source of information, which could help to secure management support for financial investments to address resource challenges such as a shortage of isolation facilities or funding for additional training. Comparing their own situation to other trusts seemed to encourage CPE prevention and control efforts as CPE outbreaks at other trusts highlighted the seriousness of the problem; some participants mentioned a desire to outperform other trusts. Patients’ lack of awareness and refusal to participate in recommended procedures, particularly refusing rectal swabs, was a barrier reported by some trusts but could be addressed with communication and reassurance.

Encountering CPE increased staff motivation to develop local plans and follow recommended procedures and led to raised awareness and CPE knowledge. On the other hand, if CPE was already a problem, procedures outlined in the CPE toolkit were often not seen as adequate or practicable, forcing staff to find new ways of managing the problem, including cohorting whole patient groups. Older hospital buildings were a barrier to ensuring appropriate cleaning of facilities and isolation. Staff shortages, time pressures and competing IPC demands complicated matters further. Training members of staff for example, the most frequently named dissemination strategy for local CPE plans, was difficult due to high staff turnover and work pressures. There was increased anxiety in the context of resource shortages as this increased the challenge of managing potential CPE cases. Being extremely concerned and anxious about CPE seemed to evoke helplessness and inaction.

Optimism and confidence in the capability to carry out recommended procedures was both a barrier and facilitator. A certain level was required for successful implementation, for example when carrying out rectal swabs. An over-confidence, assuming current practice based on clinical expertise and experience sufficed, however, prevented staff from acquiring further knowledge and skills. Some staff mentioned that the CPE toolkit helped reduce concerns and improved beliefs in their capacity to manage outbreaks as it provided clear guidance. Continuous improvement efforts, e.g. using monitoring, audit or feedback, helped adapting patient management procedures. IPC nurses typically took charge as they felt responsible for and took ownership of the issue and for passing on relevant knowledge. Overall, it was facilitative to change if staff perceived following recommended procedures and dealing with CPE as an important part of their role and identity.

Views and responses to the CPE toolkit

Participants, who were familiar with the CPE toolkit, often could not remember their initial thoughts or recall its specific content, but reported views were mixed (Table 3). Some thought it was comprehensive and useful and much needed at the time, especially the flowchart, while others perceived the recommended procedures as impractical to implement locally. Participants criticised the CPE toolkit for not being specific enough, sometimes acknowledging this could be due to lack of a clear evidence base. The lack of scientific evidence led others to question the credibility of the information provided and to voice distrust in or confusion around the recommended procedures. While the toolkit did not include enough information for some, other participants criticised it for being too long, making it inaccessible. There was also some uncertainty about the CPE toolkit target group.

CPE toolkit improvement suggestions

The most commonly voiced improvement suggestion was that the current version of the CPE toolkit should be updated regularly (23%). Participants were keen that an updated version would include new evidence and further clarifications on the difference between CPE and Carbapenemase-producing Organism (CPO), risk factors and screening procedures, including different implementation options to fit clinical practice. One way to address concerns around the evidence base of the CPE toolkit and strengthen its credibility could be linking it to related guidance and research publications or providing contact details of experts for further queries. A better fit for clinical practice might be achieved building on previous involvement of experts “on the ground”.

Participants felt it needed to be more explicitly highlighted that the CPE toolkit encouraged local adaptations and to provide tailored information to meet the needs of different staff groups and types of trusts, for example trusts with low and high colonisation rates. To improve accessibility and reduce information overload for general frontline staff, an abbreviated version was suggested. Participants also expressed a desire for further visualisations, particularly in the form of graphs and figures concerning epidemiological information. Those figures would not necessarily need to be provided in writing, but could be accessible online and should again be updated regularly. Table 5 summarises recommendations for improving the CPE toolkit based on participants’ suggestions and outlined implementation facilitators and barriers.

Table 5 Recommendation summary for future CPE toolkit versions
Full size table

Discussion

In this qualitative implementation evaluation, frontline staff representing a variety of roles and different hospital settings, in terms of CPE colonisation rates, time point of introducing local CPE plans, trust size and geographical region, were interviewed to examine in-depth their experiences concerning awareness and uptake of the CPE toolkit and recommended procedures. A minority of frontline staff had seen the published CPE toolkit and of those who had almost all were staff with specific IPC responsibilities. The majority of frontline staff was exposed to the information in the CPE toolkit indirectly via local CPE policies, which were in place at all participating trusts.

Interviewed IPC frontline staff typically used the toolkit as a resource or were aware that it had been used to inform the development of their local CPE plan. In cases where outbreaks preceded the CPE toolkit, it was used to check whether existing plans aligned to recommended procedures. The previously conducted survey with executive-level staff found that out of 99 represented trusts 64% said they had used the CPE toolkit to inform their local CPE plan whilst 32% reported having used it as provided [19].

Proximity to CPE in terms of local experiences or outbreaks at trusts nearby seemed to be a crucial facilitator of change via increased awareness and motivation. Specifically trusts with high colonisation rates and outbreak experiences were quick to have management and control procedures in place, even before the publication of the CPE toolkit that was not necessarily seen as useful anymore. This is in line with findings from the executive staff survey and research on regional CPE prevalence at English hospitals [19, 45]. The development of local plans and the introduction of CPE prevention, management and control procedures based on the CPE toolkit in response to such trigger events represent a multilevel implementation process typical for healthcare interventions in complex settings [46, 47]. It was not a strictly linear, rational and clock-workstyle pathway to change standard operational procedures. As found in research on introducing new procedures and systems in primary care [48], pathways to implementation were complex and both organisational and interpersonal factors, such as senior management support, a culture of knowledge sharing and multidisciplinary collaborations, mediated the process.

The evaluation captured the multidisciplinary nature of both, local plan development and their implementation. Continual efforts were required to adapt procedures in the light of changing conditions, such as increased experiences of CPE. The conceptualisation of the CPE guidelines as a toolkit allowed for differences between trusts and shifted the implementation focus from fidelity to local adaptations, which could be tailored to the specific contexts and built on local expertise. Such multicomponent approaches that build on and increase existing infection control measures have been found to be particularly effective in controlling CPE outbreaks, whilst feedback loops involving various stakeholders within the organisation are important to translate guidelines into implementable procedures [3, 46, 47].

Overall, writing local plans was not a problem, but difficulties arose implementing them. This is not unusual as even evidence-based guidelines are frequently not implemented in practice [49, 50] and therefore implementation research is crucial to understanding the reasons for this to improve uptake [46]. This implementation evaluation highlighted implementation barriers and facilitators that should be considered in future revision of the CPE toolkit. Furthermore, as epidemiological studies assessed CPE as a representative example for new emerging AMR threats [45], current findings could be helpful when developing and implementing other AMR focused policies.

Social and resource factors were the most important implementation facilitators and barriers. Organisational cultures were found to be particularly important for the implementation of new policies in complex organisations such as hospitals where many different groups of people interact [51]. Interview participants commented, for example, on the facilitative influence of senior management support, which is in line with the prior executive staff survey [19]. Acute trusts that placed a low priority on CPE prevention, management and control also had poorer compliance at the frontline.

Organisational factors are even thought to outweigh the impact of the quality of the evidence supporting new policies or procedures in respect to implementation [52, 53]. This may be partly because frontline staff did not necessarily have contact with the actual guidance, but rather with the local interpretation of them. Staff commented on the benefit of knowledge sharing and multidisciplinary efforts, similar to evidence on successful CPE outbreak control measures, which included dedicated IPC frontline staff and hospital-wide contact tracing as part of a multifaceted intervention [54]. Delegating overseeing the implementation of local CPE prevention, management and control procedures to specific IPC frontline staff seemed easier in large trusts with extensive IPC teams, while multi-disciplinary collaborations were facilitated in small trust by staff being familiar with each other and existing work relationships. Cross-trust coordination of IPC activities within regions, e.g. by sharing information about patients’ colonisation status between health-care institutions, seems particularly important in England and other countries where hospitals are connected by patient movements and inter-hospital spread is likely, and where there is the possibility of community associated CPE [26, 45, 55, 56]. A big resource-related challenge was staff having to negotiate competing demands with little scope to be released from day-to-day duties for example for relevant training. Keeping up with training was an even bigger issue if there was high staff turnover and employment of agency staff.

In the absence of cases, CPE concerns amongst participants were often low and so was the perceived risk to patients’ safety, hindering compliance with recommended procedures. This is supported by research that has found a poor compliance with guidance at the frontline if staff deemed their patients not at risk [57] and if there was a lack of fit between clinicians’ experiences and recommended practices [58]. Some participants at trusts with currently no or low numbers of CPE cases felt confident in their abilities to manage small numbers of cases but were less optimistic if they expected an increase in the future. This mirrors the perception of executive trust staff, who also voiced a lack of confidence in the value of the CPE toolkit when dealing with outbreaks [19]. Interview participants had general concerns about the effectiveness of the CPE toolkit guidance and questioned its evidence-base.

Although the CPE toolkit suggests a multifaceted approach in line with best practice evidence [3] and its overlap with other guidance based on expert consultations and systematic reviews [59, 60], evidence gaps remain. There is uncertainty for example with respect to CPE risk factors, cost effective surveillance strategies and the general robustness of the available evidence. It is important to address these gaps, although improving the evidence base alone will not be enough to strengthen guidance implementation [52, 53], which also depends on the described motivational and organisational factors.

Regular intervention revisions and updates are required to achieve sustainable change, but excessive guideline revision frequencies should be avoided as they may undermine implementation [46, 47, 58]. In the case of the CPE toolkit and in light of the influence of CPE proximity on implementation, it may be that maps providing epidemiological data on the geographical distribution of CPE positive isolates could be produced and updated frequently online, while the CPE toolkit in its entirety should only be updated when a considerable amount of new evidence is available [45, 58]. The need for comprehensive summary information should be considered when presenting new evidence and references and additionally information sources should be clearly signposted to accommodate further information needs. It would also be helpful to link the CPE toolkit to related guidance such as the CPE toolkit for non-acute and community settings [61], which would be important for issues around transfers to community settings.

The CPE toolkit contained information targeting different groups, including hospital frontline staff, board and executive-level staff and patients [10]. The evaluation found that it was used mainly by IPC specialist staff and disseminated to other frontline staff was via local plans and processes such as on the job training. It may therefore be useful to target the CPE toolkit for use by the IPCT, and provide them with recommendations and resources on how to disseminate its content to the hospital workforce. This approach, providing implementation guidance, would harness the responsibility felt by IPC staff to inform others. Additional staff groups such as microbiologists or pharmacists may also be involved in dissemination. A section for board level executives should remain due to the importance of garnering management support. The CPE toolkit could function as a comprehensive reference document, but with sections and resources clearly signposted to relevant audience (information for general frontline staff, microbiologists, pharmacists, management etc.) and with information specifically prepared for each target group to reduce information load and improve processing. A modular design could also consider different or incremental CPE incident levels as trusts with none or low colonisation rates require different action plans than those with widespread outbreaks. For low-prevalence CPE settings enhanced infection control measures even without active surveillance could suffice whereas hospital-wide contract tracing would be required to control outbreaks [54, 62].

Implementation guidance could be included in the CPE toolkit to facilitate the process of incorporating recommended procedures into clinical practice. A task force or implementation team consisting of different target group representatives and tools for conducting stakeholder analyses or leadership support assessments are promising options [63, 64]. Leadership support could further be encouraged, if necessary, by providing information on cost savings due to outbreak prevention measures [65, 66]. Overall, acute trust representatives should be involved in the revision of the guidelines and multidisciplinary collaborations at trust level should be encouraged to revise local policies. Involving end-users promotes implementable recommendations and a sense of ownership and commitment to adherence [58].

Drawing on behavioural science theories, this implementation evaluation contributed to addressing a lack of rigorous, theory-based implementation evaluations and provided an evidence base for updating the CPE toolkit [17, 50]. The in-depth, qualitative research approach allowed for flexibility to explore emerging issues and new points of views, which was particularly important to understand the implementation in context and to uncover not only the intended, but also unintended intervention consequences [46, 47]. Taking into account participants’ responses and following up on their specific experiences and implementation concerns meant that topics outlined by the interview guide were not assessed in a standardised, quantifiable but holistic manner [39, 67].

A personal access strategy and entering the field during data collection [27, 68] resulted in good participant engagement rates with 12 out of 14 trusts and general as well as IPC frontline staff participating. Purposively recruiting trusts and frontline staff using a maximum variation sampling approach [29,30,31] provided insight into wide range of frontline experiences in different settings. The recruitment was based on the previous survey sample [19] and required consent to interviews, possibly selecting trusts already being more engaged in the CPE toolkit implementation than non-survey participants. Considering late adopters, trusts that introduced local CPE plans only with a delay of up to 3 years, might counteract this potential selection bias to some degree. With telephone interviews being conducted over multiple days, answers concerning awareness of the CPE toolkit could have been affected due to trust internal information exchange, but remained overall low amongst general frontline staff.

The CPE toolkit was nationally disseminated and there were therefore no examples of acute trusts not provided with it to be considered for comparative purposes. Due to the multicomponent nature of the CPE toolkit, the evaluation focused on general, top-level implementation facilitators and barriers to provide generic improvement suggestions. Analysing the implementation of all target behaviours suggested by the CPE toolkit would have been beyond the scope of this research and would not have been reasonable considering the guidelines allowed for local adaptations. This should, however, be kept in mind when interpreting the findings and it would be useful to triangulate them with experimental research investigating the impact of the guidelines on specific target behaviours [69]. It would have been beneficial to conduct this implementation evaluation sooner after the national launch of the CPE toolkit, accompanying the implementation phase, to avoid potential memory biases. This approach would fit with a continual cycle of evaluation, feedback and implementation as subsequent adaptation is important for future guidance.

Conclusions

The CPE toolkit was found to be helpful as a reference document for IPC frontline staff using it to inform and check their own CPE prevention, management and control plans in particular in the event of an outbreak. Frontline staff felt there was a need for an updated version of the CPE toolkit, including new evidence and guidance on how to tailor it to different contexts. Greater consideration should be given to implementation when developing national guidance and policy.