Development, implementation and evaluation of the digital transformation of renal services in Wales: the journey from local to national

Background Treatment for people with kidney disease is often associated with complicated combinations of medicines. Logistical challenges with traditiona paper-based prescribing means that these patients are particularly susceptible to medication-relation errors and harm. Aim To improve the quality of care that people with kidney disease receive across Wales through a Value-Based digital transformation programme. Setting Renal units within the National Welsh Renal Clinical Network (WRCN). Development A novel Electronic Prescribing & Medicines Administration (EPMA) system, integrated into a patient care record and linked to a patient portal was developed in South West Wales (SWW) region of the WRCN, enabled by the Welsh Government (WG) Efficiency Through Technology Fund. National upscale was enabled through the WG Transformation Fund. Implementation EPMA was designed and rolled out initially in SWW region of the WRCN (2018). A dedicated delivery team used the blueprint to finalise and implement a strategy for successful national roll-out eventually across all Wales (completed 2021). Evaluation A multi-factorial approach was employed, as both the technology itself and the healthcare system within which it would be introduced, were complex. Continuous cycles of action research involving informal and formal qualitative interviews with service-users ensured that EPMA was accessible and optimally engaging to all target stakeholders (patients and staff). Results confirmed that EPMA was successful in improving the quality of care that people with kidney disease receive across Wales, contributed to Value-Based outcomes, and put people who deliver and access care at the heart of transformation. Conclusion Key findings of this study align directly with the national design principles to drive change and transformation, put forward by the WG in their plan for Health and Social Care: prevention and early intervention; safety; independence; voice; seamless care.


3
• Clear implementation strategy, incorporating a project initiation period, with a dedicated team of clinical staff, experienced in technology.

Barriers to best practice
• Time for staff to be released for training-overcome by: • lean training material in the form of short videos; • real time one-to-one training in the care setting.
• Across Health Board information technology (IT) barriers-overcome by a dedicated renal IT engineer to coordinate roll-out with local IT teams. • Limited engagement from some key staff as a results of work pressures-overcome by: • identifying local enthusiasts and agile use of local skill mix; • have a multi-professional central Delivery Team agile to fill local gaps in skill mix.

Background
In 2018, following a Parliamentary review, the Welsh Government (WG) published their long-term plan for Health and Social Care [1]. 'A Healthier Wales' described the need for change in order to meet the needs of the Welsh population. It set out a vision for a 'whole system approach', integrating health and social care, with an emphasis on using new technologies to support people in managing their own health and wellbeing, and to enable them to live independently. In Wales, renal services are coordinated from three regional centres in the South West, South East and North of the country. The Welsh Renal Clinical Network (WRCN) commission and plan specialised renal services. With a growing elderly and comorbid population the demand for specialised renal services is increasing. Current estimation in Wales is that around 6% of the adult population has chronic kidney disease (stages [3][4][5], when renal services are required to support the specialist treatments and care of the patient [2]. The South West Wales Renal Service (SWW-RS) cares for people with kidney disease over a large geographical area, including people with progressive kidney disease, kidney failure, and those who require dialysis or kidney transplantation. For these patients, treatment is often associated with a complicated combination of medicines that are prescribed and monitored by specialist prescribing pharmacists, prescribing nurses, and renal physicians. The supply is done by renal pharmacy professionals [3]. SWW-RS has a dedicated renal pharmacy, which is professionally-integrated into the regional multi-professional service. This has created the culture to think differently to redesign services, to provide equitable, accessible, and high-value treatments and care to people with kidney disease [4].
Logistical challenges with traditional paper based prescribing in the regional SWW service, which covers a large geography with a number of rural satellite dialysis units, often resulted in treatment delays, missed medication doses and mismatches between paper prescriptions and electronic records, as reported elsewhere [5] with patient safety compromised. In addition, clinicians wasted time on administrative tasks and unnecessary travel and relied on fax machines.
It became evident that delivering value and capacity to deal with the growing pressures in the healthcare system, whilst also aligning with the World Health Organisation's (WHO) Global Patient Safety Challenge: Medication Without Harm [6], meant doing things differently.

Aim
To improve the quality of care that people with kidney disease receive across Wales through a Value-Based digital transformation programme, in line with WG and WHO strategy.

Development
As part of a national transformation programme, kidney care in Wales has been digitised through a single platform: the renal electronic patient record (rEPR). rEPR is a single platform for information sharing, allowing patients (via a patient portal) and healthcare professionals to easily access information online. The purpose of this digitisation was to improve the quality of care for all patients who access renal care services in the country and support them with self-care [7]. SWW-RS identified the need to further digitise their service and develop a new Electronic Prescribing and Medicines Administration (EPMA) module within their rEPR.
EPMA computerised systems have been designed to address some of the problems associated with paper-based drug charts, and have the capacity to facilitate both the prescribing for patients and the subsequent administration of these medicines within hospitals [5]. Several studies have built the evidence-base supporting the positive clinical use of these systems, with results confirming reductions in prescribing and administration error rates [8], reduced illegibility issues and hence decreased likelihood of medication errors [9]. In addition, they demonstrated other benefits 1 3 such as cost-savings to the NHS, staff effectiveness and workflow, accountability and auditing, formulary compliance and out-of-hours support, freeing up clinicians' time from duplicating information in more than one records [10]. Routinely collected data of EPMA systems have also been used to generate actionable knowledge to improve medication safety [11].
However, EPMA systems do not come without challenges and realising the benefits that digitisation can bring is contingent to addressing several considerations. It has been argued that EPMA can disrupt workflows and work patterns, and have a negative effect on communication between members of the multidisciplinary team [12][13][14][15]. In some cases, no impact on medication errors has been found and even an increase in discrepancies has been reported [16]. Issues such as incorrect drug picking, population of dosages and remote prescribing have been identified as contributing to these errors [17].
Despite this, EPMA is established in many NHS Trusts in England and in other countries. Unlike many of these hospitals, SWW renal EPMA is fully integrated with the renal central electronic patient record. The team were conscious that digitisation should be more than simply implementing new technology, it should reconceptualise the service delivery model and must put people who deliver and access care at the heart of transformation. Hence, the development, implementation and evaluation of EPMA was completed in phases, informed at all stages by the updated Medical Research Council framework for the development and testing of complex interventions [18]. This included identifying the desired digital approach, developing it, assessing its feasibility ensuring we noted uncertainties and refining points, continuous evaluation and implementation. It was a dynamic process involving an adaptive design, whereby we aimed to explore how the system works, what the direct and indirect impact was and how it impacts on system change, and what the balance of the resources versus outcomes was. It was decided that a 'retrofitted' commercial EPMA product would not be the solution; instead a fit-for-purpose product was designed in-house (insight from nurses available at: https:// youtu. be/ vOTRu urgwgA): it was simple to use but elegant to deal with complexities of renal treatment, affordable, integrated within the existing rEPR rather than overcoming interoperability issues, with the team having control over its evolution based on feedback. The product also allowed all users the professional autonomy to undertake auditable clinical decisions when at patient's best interest, and the professional decision to act was not prohibited by a 'computer says no' permission or functionality issue.
The pharmacy team were involved in all parts of the process from securing funds and budgetary responsibility and appropriate approvals, through to concept design and technical development, providing training materials and face to face training, ongoing support and ongoing development. The digital intervention was co-designed with service-users in an iterative approach (core development team of four renal pharmacists, one pharmacy technician, one nephrologist, one IT technician, one project manager and one nurse in the SWW-RS), using multiple cycles of action research [19,20] and following principles of the person-based approach to development of digital health-related behaviour change interventions [21]. This ensured EPMA was accessible and optimally engaging to all target stakeholders, including a wide range of healthcare staff (pharmacists, nurses, dietitians, physicians etc.) and patients, and that promoting change and problem-solving was embedded throughout the whole journey. At the heart of the strategy was to make information available to patients via the digital patient portal, or to automate the outputs to paper, for patients who preferred to assess their information in this way. Patients were supported to understand and act on this information to be active partners in managing their health.
Early cycles of action research in the development phase involved qualitative think-aloud techniques [22] and verbatim extracting positive and negative comments about the system, observing and noting reactions to the different functionalities. The bespoke EPMA module was rolled-out to five dialysis units across SWW by 2019 ( Fig. 1), providing equitable care for people living in rural and urban areas (development journey available at: https:// youtu. be/ L2KaM C41RIg). In this stage, cycles of action research involved formal interviews with staff and patients to supplement the initial, informal feedback. The approach facilitated informed discussions and constant improvement, with documented proposed changes, their rationale and priority [23]. It also allowed identification of enablers for the wider roll-out, from one dialysis unit in one hospital, to the whole of Wales. Some of the evidenced-based adaptations, tailored to service-users, were the training model, including short videos for training of nursing staff rather than relying on a manual; continuing co-creation with service-users to make the system intuitive and improve user-friendliness; improved reports on medication use and simple business continuity procedures in case of system failure. Collaboration with the commercial partner chosen to develop the system and WG funds (Efficiency Through Technology Fund (2016), Transformation Fund (2020)) facilitated both the system development and the service transformation.
The core EPMA development principles were: • a paper-free tool where treatments prescribed from any renal centre are instantly available for a patient wherever they are receiving care; • system housed within the established single rEPR; • bespoke touch-screen medicines module designed inhouse by the end users; • integration with the patient portal; • incorporating business and clinical intelligence in real time, including medicines surveillance dashboards.

Implementation
Having created the blueprint with the successful regional implementation, a strategy was developed for national implementation. Government funds allowed for the original SWW development team to be designated as a Transformational scale-up Delivery Team, consisting of renal health professionals, renal IT engineer and a newly enrolled programme manager who engaged with the commissioner and renal colleagues across Wales. This experienced team enabled the successful national roll-out at scale, at pace and within budget. Non-staff costs included hardware and software development, and external evaluation. The EPMA module was designed with scalability and flexibility in mind, to accommodate warranted variation in practice across the regions; however, a clear implementation strategy for roll-out was essential. EPMA was implemented in 13 dialysis units across the two remaining Welsh renal regions between 2020 and 2021. The core delivery team introduced EPMA in each unit and outlined the process of change with renal physicians, nurses and pharmacists as described above in advance of each unit's 'go-live' date. Media, such as videos and infographics, were used for engagement to tell the story of the project and transformation; these resources were made accessible to those unable to attend events.
Digital hardware, including touchscreen computers and combination medication and computer carts were purchased by the Delivery Team and provided to each dialysis unit to standardise the model across the country. The team worked with local IT departments in each locality (Health Board) to prepare the dialysis unit for EPMA, ensuring good WiFi connectivity and robust business continuity procedures in case of system or network failure.
Each dialysis unit was assigned with up to one week with onsite support from the Delivery Team to transition from paper-based prescribing to paper-free EPMA. Pharmacy professionals, an IT engineer and project manager from the Delivery Team travelled to each dialysis unit to support local clinicians. This included the transcribing of paperbased medication records onto the new electronic module, followed by digital authorisation by the unit's prescribing pharmacists or physicians. Prescribers and nurses were provided with face-to-face training and competency was assessed by parallel running paper medicines charts with the replicated e-chart. The core delivery team provided ongoing clinical and technical support remotely and provided virtual pharmacy surveillance (using real-time clinical intelligence reports) to each dialysis unit after EPMA implementation, to ensure quality and patient safety.

Evaluation
Evaluating EPMA required a multi-factorial approach, as both the technology itself, as well as the healthcare system within which it would be introduced, were complex [24,25]. Table 1 outlines the on-going projects that go beyond the  question of whether the intervention works; as outlined by Skivington et al. [18], a broader set of questions is being asked related to understanding the impact of the system on different health and social care outcomes, and the financial implications, as well as the preliminary findings. In this section the detailed findings of interviews with patients will be presented. It is important to note that multiple team meetings have already taken place to triangulate results from the different parallel projects, using a formalised protocol [26].

Study design, method and data analysis
As part of the wider project consisting of action research cycles to evaluate the staged national digital transformation across renal units in Wales, we wanted to explore patients' experiences of treatment in the renal units using the new EPMA system and how the system has impacted patient experiences and outcomes. A qualitative phenomenological design was used for exploring patient views, employing semi-structured interviews. Further details on design and ethical considerations are provided in Table 2.
Following transcription and familiarisation with the interviews, transcripts were coded and mapped against a combination of two different frameworks, to find out the impact that the digital system has had on patients' quality of care and their wider lives more generally: Table 2 Details on the data collection, sample and recruitment (including inclusion/exclusion criteria), sample size, sampling technique, recruitment and consent, and ethical and regulatory considerations for the patient evaluation strand of the study

Methodological aspect Detailed information
Data collection Interviews were conducted by the researcher over the phone or through Skype/Teams with or without the use of video (whatever was preferred by each participant, guided by a semi-structured interview schedule and lasting approximately 30-40 min. Written consent was sought from participants prior to the start of the interview. The interviews were audio or video recorded for transcription, with consent, and once verbatim transcription was complete the recordings were deleted to ensure data anonymisation Sample and recruitment Patient/guardian/carer Inclusion criteria Population Individuals or carers of individuals who have been initiated or on maintenance HD therapy, or require treatment for chronic kidney disease, since the introduction of the EPMA system Over the age of 16 Experiences Individuals who have been initiated or on maintenance HD therapy since the introduction of the EPMA system Communication Can communicate effectively in English (does not have to be first language), to be assumed upon response to the information leaflet Ability to provide informed consent for themselves or the patient they care for Exclusion criteria Population Individuals or carers of individuals who require HD treatment or treatment for chronic kidney disease, and access this through a renal unit in Wales without the EPMA system Under 16 years of age Communication Unable to provide informed consent Sample size Using the principles of information power, it was estimated that between 7-15 participants from all patient populations would be needed to gain valuable data across the whole SWW region, to account for a representative sample Sampling technique Potential participants were selected using a mix of non-probability purposive and convenience sampling a. Purposive: Potential participants were screened against the eligibility criteria presented above and approached by the gatekeepers b. Convenience: Participants were self-selected, as they chose to respond to the information leaflet Recruitment and consent Designated members of staff acted as gatekeepers, identified patients that met the inclusion criteria, and provided them with a copy of the information sheet and consent form about the study (in English and Welsh) with the research team's contact details, so they could contact the researcher directly if they were interested to find out more about the study. In the case of children, patients with difficulties in communicating or who lacked capacity to give informed consent, their carer would be approached. All the basic elements of the consent process were followed and no incentives were offered for participation Ethical and regulatory considerations Participation was voluntary, and all participants were informed of their right to withdraw at any point. The study was carried out in accordance with the Research Integrity and Governance Code of Practice, laid out by the NHS and Cardiff University; the project was registered as a service evaluation with NHS Research and Development department, and ethical aspects were assessed by Cardiff University, who granted a favourable opinion (ref: 2021-01). All participant information and characteristics were anonymised, de-identified and given pseudonyms in the final report, all audio or video recordings of interviews were deleted immediately after transcription. At this point there was no way to connect consent forms with individual comments ensuring complete anonymisation of data 1. The three dimensions of health, as stated in the definition of health by WHO: "…a state of complete physical, mental and social well-being" [27]. 2. The six domains of healthcare quality, by the Institute of Medicine [28].
To ensure no other concepts have arisen during the interviews that have not been captured with the framework analysis, inductive analysis followed the approach outlined above.

Results
Eleven patients were recruited and interviewed between 23 rd April 2021 and 30 th November 2021 ( Table 3).
The combined framework, consisting of the six domains of health care quality and the three dimensions of health, provided nine themes for analysis-data was mapped across all domains, highlighting the holistic impact of the digital system on patients (Fig. 2, Table 4). No additional themes were found using inductive analysis.

Safety
Patients believed that the digital system had made care safer by allowing healthcare professionals to view the same, upto-date information. They particularly highlighted how using Patient View, the patient-portal interface of the system, helped ensure that any treatment patients received in nonrenal settings would not cause any harm. Examples were provided of cases where circumstances prevented patients from sharing results on PatientView [29], leading to safe transfer of care being compromised.

Effectiveness
The patient portal was raised as an extremely effective resource in supplementing patients' care, which allowed patients to actively manage their own condition by checking blood test results and medication changes live, on their phone, and also to input their own data, such as blood pressure and volumes of dialysis fluid after home dialysis. They felt this was effective in enabling them to self-care and be independent.

Patient centredness
The opportunity that patient portal offered to patients to access their own information was considered important, empowering them to feel involved and at the centre of their care, increasing their awareness of their condition and their treatment. They felt in control as they could monitor trends in their blood results to see whether their condition was improving or deteriorating, and used the parameters that were given to know what to aim for. Patients also gave examples of how the digital system being used by staff had allowed their preferences and needs to be taken in to account, and how it increased bedside interactions, which was welcomed and appreciated.

Timeliness
All patients discussed how this functionality of entering and viewing blood results digitally increased timeliness of several aspects of their care. Staff had instant access to all information at the point of care, on devices that were easily portable, and this allowed fast action when needed, for example when blood test trends indicated suboptimal treatment.  . 2 Key quotes from framework analysis of interviews with patients, mapped against the three dimensions of health [27] and the six domains of healthcare quality [28]. Figure created using Canva Education edition 10 days later there will be medication changes put up on PatientView, so I'll have a look at those as well." PP3: "I go and look at my blood results, especially my monthly blood results, and I look at the trends there." A3: Patient centredness PP1: "I rely on it…..as soon as the message comes and sometimes it comes late at night, I will look at it whatever time…You know, it's important to me, perhaps it isn't to other people but to me it's the way I live now." PP3: "I can look at the trends going back years and I notice how, for instance, how GFR has deteriorated over the years.
Which is a bit of an interest…for me." PP1: "Because I don't like…..blood infusions because I've always got a funny taste in my mouth…..I get a little bit fed up, when the nurse is by your side the iPad computer [means] she doesn't have to leave me, you know….there's always somebody on hand….I'm comfortable going there." A4: Timeliness PP1: "If there's a problem I've got instant access to people…to a text…to emails, you know, there's always something I can do." PP1: "When they looked at my blood results there were other minerals and other vitamins I was deficient in, so they prescribed Remevit for me…..they were able to prescribe that there and then for me, you know." PP11: "When I was with [a different renal unit outside of Wales], I'd have to wait…up to a couple days for some results to be, um, actually looked at by a consultant…Whereas with PatientView, I can view them in the evening and if there is any issues that I see, personally I can phone up that night or the next day and just, uh, go over my concerns." A5: Efficiency PP1: "I like a text I do or otherwise I forget." PP2: "For me the phone app it works well. I think it helps to cut down on paper as well so I'd rather that over a letter." PP3: "I'd rather online. It's quicker, more direct so I'm not waiting for a letter so online is great." A6: Equity PP5: "Oh I find it great because every morning when I come off dialysis I put all my details, my treatment on Patient-View…..I just go into….dialysis treatment and then just put all my blood pressures, how much I've taken off the machine…the pulse." PP1: "If I've got a question which I have had a few times, for example I'll just text 'my potassium is high is that okay?'….and they always reassure me." B1: Physical health PP3: "There's a menu sheet…so if I've come across some food that I'm not sure that is high in protein or high in potassium or whatever I can got to that and that's a reference point for me." PP9: "If I see my creatinine is high, but I mean I'm really good with my water so I don't need really to, to be careful, but obviously if it's come a bit high up, then I will make a bit more effort, like I will be conscious, you know, and, and do it a bit more." B2: Mental health PP9: "I know I'm going to have the result between 9:00 and 11:00 and I, I just wait for that and then…I can be relieved or I won't be worried because I know there is always a solution. Um, so it really, really helped me to say 'Oh brilliant, I'm good'." PP9: "I love it. I absolutely love it. Um, I don't think I can do without it…it makes me feel really, really good to know my blood results after a blood test." B3: Social health PP6: "I was working for a private ambulance service….I did show them my blood results [on PatientView], saying "look I'm fine, my bloods are normal." PP3: "If we're going out for a meal we look at the menu and I can always access it to see, you know, if there's a possible problem with me eating a certain food." PP9: "When we apply to, to be part of the race, basically [organisers] want to know all the medicine and…what my condition and what they can [do]." PP4: "You go on PatientView and just click on medication and everything that you're on is on there….I went on holiday and they asked me if you're on any medication and I went on there and printed it out."

Efficiency
It was perceived that the digital system was very efficient in receiving information about their treatment from staff, for example via the patient portal or via the secure messaging function on the app, as opposed to receiving paper letters or phone calls. Texting prescription requirements also made collecting prescriptions more efficient for patients.

Equity
The digital system supported equity of care provision and monitoring to all patients, regardless of whether they dialysed in the main hospital, a satellite unit or independently at home. Patients on home dialysis felt reassured that they would receive the same standard of monitoring as if they were in hospital, because of the direct link between the patient interface and the digital system that staff have access to, which allowed regular review of electronic versions of these records. Transplant patients also described using email and text functions for contacting their care team, not feeling disadvantaged by their self-care treatment plan any more.

Physical health
The digital system has impacted patients' wider lives, aside from the quality of their health care, through also improving the pillar of physical health. Examples were provided by different patients of how the patient portal was used to adjust diet according to blood test results (e.g. abnormal potassium levels), to find out whether certain foods were appropriate for their diet via the 'menu' function, and to continue with hobbies that involved exercise, such as paddle boarding.

Mental health
The ability to instantly contact staff at their renal units, through the secure messaging function on the patient portal and to receive their blood results in a timely manner, supported patients to feel reassured and alleviated any anxiety, providing peace of mind and having a positive impact on their mental health.

Social health
Participants gave examples throughout the interviews of how the patient portal helped with several aspects of their wider life and social health. This included a) their careers, by enabling them to provide list of current medications/blood test results to prove that they were within 'normal' ranges (i.e. that they were healthy enough to work); b) their hobbies and social lives, by helping them to inform the organisers about their condition and continue with their hobbies in a safe way, and also by enjoying meals in restaurants; c) their holidays, as they could inform others about their current medications when needed.

Discussion
Key findings from our multi-factorial evaluation align directly with some of the national design principles to drive change and transformation, put forward by the WG in their plan for Health and Social Care [1]: 1. Prevention and early intervention: access to information on patient portal allows patients to maintain healthy lifestyles, e.g. dietary choices, to prevent worsening of their conditions or development of co-morbidities. 2. Safety: EPMA ensures that everybody involved in the care of a patient, including the patient themselves, shares the same knowledge about them and so harm is minimised. 3. Independence: EPMA, and its integration with the patient portal, supports self-care e.g. for patients on home dialysis treatments or those involved in the management of their own medicines, such as transplant antirejection mediation. 4. Voice: patients are informed about their condition and empowered to take an active part in managing it and can easily contact those involved in their care to express concerns they may have about their health. 5. Seamless care: access to information through EPMA/ patient portal and its outputs by all healthcare services (renal and non-renal) enables seamless transition of care between them.
The findings of improved safety and seamless care by enhanced communication between the renal units and other primary, secondary or emergency care settings have been reported elsewhere [8,30].
The idea that EPMA, particularly the patient portal linkage, had a positive impact on patient centredness and effectiveness of care was prevalent in the results of this study. Similar digital patient portals, used across the world, have had the same effect of promoting patient involvement in their own care. Preventative health behaviours, such as receiving flu vaccinations, blood pressure checks and lipid level screens were substantially higher in users of a patient portal than non-users [31]. Patient portal usage has also been associated with a significant increase in patient knowledge regarding health conditions, lower conflict in making decisions, higher self-efficacy and better collaborative relationships between patients and healthcare providers [32]. Additionally, several studies found that patient portal usage was linked with better medication adherence [33][34][35].
Our study did not investigate the impact of EPMA on specific health outcomes, however it was suggested in the results that physical health may be improved as a result of using functions of the patient portal. The impact of eHealth interventions on outcomes such as these in patients with chronic kidney disease was evaluated in a study by Stevenson et al. [36]. This study, a review of randomised control trials (RCTs) and quasi-RCTs, measured the impact of these interventions on dietary intake, behaviours and nutritional status among many other patient-centred and clinical parameters. Even though included studies had methodological limitations, findings showed the potential benefits that selfmonitoring using electronic applications can have on dietary and fluid intake in patients.
Critical success factors during development of EPMA were the co-creation with service-users patients and staff, and the development of an in-house product, tailored to suit the needs of the service. This is in line with literature, whereby a system-wide service redesign, taking into account prescriber attitudes, patients and the existing workflow is recommended [17] and where understanding of the service specific constraints is highlighted as essential [37]. A clear implementation strategy was pivotal to national scale-up, incorporating a project initiation period, with a dedicated team of clinical staff (Delivery Team), experienced in technology. Engagement of key stakeholders at all time, with demonstrations, events, and creation of publicly available videos helped to maintain momentum and create enthusiasm about the transformation, and a suite of digitally accessible, lean and paper-light training materials supported flexible learning and multi-professional training, as suggested in literature [17]. In line with literature, staff concerns about IT skills or limitations of the system pre-implementation, including safety back-ups, were alleviated with this approach [30]. On-going monitoring and adjustment of the system post-implementation was also crucial, in line with other evidence [37].
During all phases of development, implementation and evaluation the team continued to reflect on the six core elements of the updated MRC framework for evaluating complex interventions [18], a process that informed progress throughout. This also enabled the team to anticipate and prepare for key challenges during national roll-out, including: • Time for staff to be released for training-overcome by: a. Lean training material in the form of short videos; b. Real time one-to-one training in the care setting.
• Across Health Board IT barriers-overcome by a dedicated renal IT engineer to coordinate roll-out with local IT teams.
• Limited engagement from some key staff due to workload pressures-overcome by: a. Identifying local enthusiasts and agile use of local skill mix; b. Having a multi-professional central Delivery Team agile to fill local gaps in skill mix.
In addition, rollout during the COVID-19 pandemic meant extra service demands and periods of staff shortages. Even so, the experience and flexibility of the Delivery Team meant that implementation was completed. It is important to note that to ensure sustainability, all of these barriers remain and whilst addressing them at the point of implementation ensured this phase of the project was completed successfully, each centre needs to optimise the system's use so it can achieve its full potential. A benefits realisation toolkit may support centres going forward and may reduce regional variation [38].
While this article has concentrated on EPMA, this is one of five elements of the Transformation Programme for recognising, preventing and managing kidney disease across Wales [39].
Limitations of the arm of the evaluation presented in this article include the small number of patients, which may mean that opinions may not be fully representative for all patients. This limitation was mitigated by using principles of information power to determine sample size, so that the final sample included patients receiving all of the different types of treatment (dialysis at hospital, dialysis at home and transplant recipients).

Conclusion
EPMA was successful in improving the quality of care that people with kidney disease receive across Wales, contributed to Value-Based outcomes, and put people who deliver and access care at the heart of transformation.

Conflicts of interest None.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.