Background

Paediatric early warning systems (PEWS) [1,2,3] and PEW scores [4,5,6] are based on a child’s vital signs and other factors such as parental concern [7, 8]. They alert health professionals (HPs) to signs of a child’s deterioration, with the intention of triggering an urgent review and escalating care, as needed [9], and reducing emergency transfer to critical care, cardiac arrest and death [10,11,12,13,14]. Recording PEW scores can be paper-based or electronic [3, 9, 15]; e-scoring has advantages in comparison to paper-based scoring in terms of enhanced safety benefits including greater time efficiency, reduction in human error and instant availability of the recorded data [16, 17]. There is a drive for PEWS to be embedded in the care of children in hospital [13, 18], but in the UK their use is not consistent [3]. The evidence base for the effectiveness of PEWS and PEW scores is ambiguous [18, 19].

Defining clinical utility and acceptability

We adopted a narrow definition of clinical utility; does the technology do what it is supposed to, and does it perform its designated function [20]. We chose a multifaceted definition of acceptability as proposed in the Theoretical Framework of Acceptability (TFA) [21] since we appreciated that implementation, adoption and assimilation of technology in healthcare systems is inherently complex [21,22,23,24,25]. The TFA is composed of seven component constructs: ‘affective attitude’, ‘burden’, ‘ethicality’, ‘intervention coherence’, ‘opportunity costs’, ‘perceived effectiveness’ and ‘self-efficacy’.

The DETECT study

The Dynamic Electronic Tracking and Escalation to reduce Critical care Transfers (DETECT) study [17] implemented a proactive end-to-end deterioration solution (the DETECT surveillance system) across a tertiary children’s hospital with the aim of screening children for early signs of serious deterioration or sepsis and reducing complications and emergency transfers to critical care following deterioration in hospital. The DETECT surveillance system is supported by System C’s CareFlow Connect and Vitals (paediatric version) apps. These apps were modified for the study and are known as DETECT e-PEWS. DETECT e-PEWS is used by HPs to document vital signs on iPods (Supplement 1) and escalate concern (Supplement 2) and to respond to alerts of deterioration triggered by the system using iPods, iPads or by personal mobile device (Supplement 3). Alert thresholds for the study were set to signal children whose deterioration trajectory suggested potential transfer to the high dependency unit (scores 6–9) or paediatric intensive care unit (scores 10+).

Within the DETECT study we chose to measure both clinical utility and acceptability as this meant that, across both concepts, we were measuring a broad range of important factors. The main focus of the qualitative study, reported in this paper, was on the acceptability so we chose a multifaceted definition of acceptability as proposed in the Theoretical Framework of Acceptability (TFA) [21] since we appreciated that implementation, adoption and assimilation of technology in healthcare systems is inherently complex [21,22,23,24,25]. The TFA is the first robustly developed framework, that provides conceptually distinct constructs that reflect the key dimensions of acceptability. We selected the TFA as it offers a coherent evidence-based to defining and assessing acceptability in healthcare [21]. The TFA is composed of seven component constructs: ‘affective attitude’, ‘burden’, ‘ethicality’, ‘intervention coherence’, ‘opportunity costs’, ‘perceived effectiveness’ and ‘self-efficacy’. However, we were also interested in health professionals’ perceptions of clinical utility so we adopted a narrow definition of clinical utility; does the technology do what it is supposed to, and does it perform its designated function [20]; other parts of the larger study addressed wider and different aspects of clinical utility.

In this paper we present the findings of a qualitative, interview-based sub-study of HPs who were in the initial months of using the DETECT e-PEWS as part of the DETECT surveillance system.

Methods

The aims and research question were: What are the experiences and perceptions of HPs about the acceptability (primary aim) and clinical utility (secondary aim) of DETECT e-PEWS and what factors influence acceptability?

Study design

This descriptive qualitative design was selected as this allows the study to be informed by naturalistic principles, aims to generate a clear description of the phenomenon under study but does not aim to develop theory [26, 27]. The study was reviewed and approved by the North-West, Liverpool East Research Ethics Committee (IRAS ID: 215339). This study followed the Consolidated Criteria for Reporting Qualitative Research (COREQ) guideline.

Participants and setting

HPs working at Alder Hey Children’s Hospital, a tertiary setting in the UK, were invited to participate in the interviews. Recruitment occurred between February 2020 and February 2021. Any HP who used DETECT e-PEWS was eligible to participate.

Convenience sampling was used to recruit participants either by expression of interest at the end of an associated DETECT study survey (paper in submission), by research nurses on the wards or snowballing by participants. Initial contact was by email with an information sheet; then a mutually convenient time was arranged for a telephone call where any questions about the study were answered, consent was gained, and the interview undertaken.

Data collection

Single, semi-structured audio-recorded interviews were conducted by telephone (further details-Supplement 4). The interview schedule (15 questions) covered key demographic data, relevant experience with vital signs, raising a concern and/or responding to a child’s potential or actual deterioration and questions about the acceptability of DETECT e-PEWS (Supplement 5).

Analysis

The interviews were analysed (BC, HS) using the five stages of thematic analysis [28]; familiarisation, generating initial codes, searching for themes, reviewing themes and producing report (details-Supplement 4).

Results

Fourteen HPs participated (n = 12 nurses, n = 2 doctors); a further 25 expressed initial interest but failed to respond to texts/emails (after three invitations spaced over a few weeks with no reply, no further invitations were sent). Participants could be broadly categorised into two groups according to DETECT e-PEWS role:

  • Documenting Vital Signs (D-VS) group: assessed and recorded vital signs (Assistant Nurse Practitioner (n = 1), Staff Nurses (n = 6)); and

  • Responding to Vital Signs (R-VS) group: reviewed patients and responded to tasks and alerts (Nurse in Charge (n = 2), Advanced Nurse Practitioners (n = 3), and Doctors specialising in general paediatrics (n = 2)).

HPs noted that the implementation of DETECT e-PEWS was just one of a series of recent changes e.g., updates to electronic patient record (Meditech), a new bleep (paging) system and a new Acute Care (ACT) response team. Although implementation of the documentation component started 6 months pre-COVID-19 pandemic, implementation of the response component occurred close to the lockdowns and HPs had to accommodate changes required due to the COVID-pandemic.

Three key themes and eight sub-themes were identified (Fig. 1): Complying with DETECT e-PEWS; Circumventing DETECT e-PEWS; and Disregarding DETECT e-PEWS. Detailed quotes appear in Table 1. These themes relate to the clinical utility and acceptability of DETECT e-PEWS and reflect how HPs responded to and engaged with the technology.

Fig. 1
figure 1

Three main themes reflecting the ways in which health professionals engage with DETECT e-PEWS

Full size image
Table 1 Overview of themes and sub-themes with supporting quotations (by group)
Full size table

Theme 1: complying with DETECT e-PEWS

Overall, this theme addresses the ways in which HPs positively engaged with DETECT e-PEWS, despite some challenges to utility, and how they complied with the intervention and followed the processes appropriately. However, it also shows that even those who complied and engaged experienced some tensions within the use of DETECT e-PEWS.

Initial impressions

Initial impressions of DETECT e-PEWS were generally positive and this supported compliance. Nurses (D-VS and R-VS groups) perceived it to be “systematic” (D-VS5), “quick and easy to carry” (D-VS4), and “real-time” (D-VS2). Some HPs (D-VS and R-VS groups) found using DETECT e-PEWS a “big change” (D-VS5) and initially “confusing” (R-VS2), although this was overcome as they gained confidence. One HP noted “the more we have used it the more we appreciate the advantages” (R-VS2).

Key issues related to clinical utility

There was agreement across both groups that the technology did what it was expected to and could perform its designated function. One HP explained “without the DETECT device our job would be harder” (R-VS-5). However, some aspects challenged its utility. In the D-VS group, the main complaint was that although vital signs and other data (e.g., blood sugar) could be recorded onto DETECT e-PEWS, fluid intake and output had to be inputted separately onto Meditech via a computer. The main objection within the R-VS group, and more of an issue for the doctors, was the desire to just have one device to carry as some were already carrying “one or two baton bleeps [pagers] and …. an on-call bleep as well” (R-VS7).

Some problems challenged utility in the early weeks of implementation, e.g., user errors such as forgetting the password, or entering data for wrong patient, and interface errors such as vital signs not being saved to Meditech. Other issues included poor internet signal, not hearing the alert alarm in noisy environments, and non-recurrent alert alarm. In settings where each bed had a computer the value of DETECT e-PEWS seemed less convincing as it was “like having to use an extra device” (D-VS3). Problems, where they occurred, were said to be “resolved quickly” (R-VS1).

It’s systematic, real-time, faster and it’s got my back

The aspects of DETECT e-PEWS that were seen to be the most positive and contributed to acceptability were those related to speed, “it’s faster” (D-VS1), accuracy of data input, real-time availability of vital sign data, triggering of alerts, quality of the graphs and ability to easily review patients; “it’s all just there in front of you. And you can pick it up as and when you need it” (R-VS5).

The D-VS group often referred to the benefits of DETECT e-PEWS over the previous system which involved inputting delays due to having to find an available computer. Via DETECT e-PEWS they “recorded straight away … anyone can see them … in real time” (D-VS2).

The immediacy of real-time data entry was beneficial for the R-VS group who stated that DETECT e-PEWS meant that information was “considered more quickly” (R-VS7), it had improved bedside handovers, improved the speed of response as it “helps us to get there [to child] quicker” (R-VS4) and had resulted in improved care of deteriorating children, “it was a controlled step up to critical care … we were there at the time they needed us” (R-VS5).

DETECT e-PEWS helped build confidence in newly qualified nursing staff, and the automated scoring was positively evaluated. All HPs in the D-VS group talked about the system acting as a back-up as it created an audit trail of their actions (e.g., documenting vital signs, creating tasks, sending messages, escalating concern) as it “gives a good timeline if a child does deteriorate, because then you can say I did an obs at this time … I’ve got that as a backup” (D-VS4).

HPs perceived that data completeness was improved, as DETECT e-PEWS used a systematic approach and ensured HPs “don’t skip past things because they can’t” (R-VS4) and the charts were easy to access and review. The data could also be “access[ed].. from anywhere” (R-VS6).

An aspect liked by some of the R-VS group related to managing workload via the use of ‘tasks’ (e.g. requesting cannulation) which, when used effectively, reduced the number of times doctors were getting bleeped as they could “pin tasks and … reply to the task rather than getting bleeped” (R-VS7).

It improves situational awareness

A key reason for the acceptability of DETECT e-PEWS was improved situational awareness about serious deterioration and it was thought to have “created a culture of everyone taking [deterioration] very seriously …. [it’s] putting information to the forefront” (R-VS7).

Alerts triggered by a high PEW score or sepsis concern meant the HP at the bedside was “automatically made aware … and... can escalate any concerns” (D-VS6). HPs in the R-VS group were aware of such concerns and, through tagging patients vulnerable to deterioration, HPs had “a vast view on which patients we need [to review]” (R-VS5).

It can create distance from and support closeness with patients

An aspect that was commented on mostly by the D-VS group was that an alert could be triggered, or a concern raised at the bedside. This was seen as particularly important with a deteriorating child as “you are able to stay with your patient but also to alert the staff with your concerns without leaving the room” (D-VS4).

However, two HPs mentioned that they felt that using the iPod sometimes impacted on the social relationship aspect of nursing care either because parents “might not want to speak to [and disturb] you” (D-VS2) whilst entering the data or because the remote nature of reviewing patients meant that you felt “distant from your nursing care” (R-VS5).

It accommodates clinical judgement

DETECT e-PEWS had standardised algorithms underpinning the alert system which meant that alerts would trigger at a given threshold. This sometimes occurred in a patient who had a deterioration management plan in place and was no worse than previously. Staff could annotate alerts (6 and above, reflecting their real-time clinical judgement: “a lot of our [patients] that have a PEW of 4 anyway just because of their oxygen levels and things like that [so a 6 is not so high]” (D-VS3). Some HPs (R-VS group) suggested a high number of alerts about children with underlying health conditions could cause alert fatigue.

Theme 2: circumventing DETECT e-PEWS

This theme addresses the ways in which HPs circumvented the processes associated with DETECT e-PEWS.

It’s easy to fall back to old ways

Even HPs who were positive advocates for DETECT e-PEWS sometimes circumvented the system; this reduced the system’s effectiveness and could reintroduce error. Typically, HPs (DV-S) entered data onto the iPod at the bedside, but some reverted to the old habits; in one instance this appeared to be routine: “I probably use them at the bedside less than 50% of the time … I still tend to …. write them down on a bit of paper and then fill them in when I come out of the cubicle” (D-VS5). In settings where a member of the R-VS group was readily accessible, staff subverted the task messaging aspect of the system as they made direct contact with the clinician and “just grab me rather than generate a task” (R-VS3).

It’s not reliable in terms of getting a response, we still need to bleep

An issue with the implementation was that whilst it was mandatory for vital signs to be documented via DETECT e-PEWS, it was not mandatory for responders (R-VS) to engage with the system. This meant that HPs (R-VS) did not always log in to the system so ward staff raising tasks were not always confident that they would get a response to tasks or alerts as “the medics … just got out of the habit of [responding to task]” (R-VS5). Thus, even committed HPs could be “nervous …. a bit paranoid” (R-VS6) if they were unsure a task had been picked up or felt they had waited too long for a response and would bleep a doctor anyway. Some felt more comfortable with the bleep system as it meant speaking to someone rather than just messaging as “you have heard them talk to you” (D-VS5).

Theme 3: disregarding DETECT e-PEWS

Although nurses within the R-VS group engaged and valued the system, there were reports that many doctors in the hospital disregarded DETECT e-PEWS and did not commit or engage with the system. This meant that the review and response component of DETECT e-PEWS failed to consistently work effectively throughout the hospital as “the doctors not using them is then leading to other people not using them” (R-VS5). A typical response from the D-VS group was uncertainty whether doctors “take [DETECT] as serious as we [staff nurses] do because they …. don’t count on it as much as we do” (D-VS6). There was also a sense that some of the senior clinicians lacked commitment to the system and that this prejudiced the perceptions of the more junior staff, including any new starters; “if the people teaching [more junior doctors] don’t use it, the juniors are not going to use it” (R-VS5). When senior clinicians either resisted or stopped engaging, the result was that “everyone’s [doctors] just gone, “Oh, well there’s no point using them now” (R-VS4).

In addition to any speciality resistance (e.g., medical doctors perceived as being better at engaging than surgical doctors) and DETECT e-PEWS not being mandatory for doctors, its acceptability in the R-VS group was probably influenced since the response component was introduced around the time of the first lockdown from the COVID-19 pandemic. Problems related to “a lot of information overload [from other changes] (R-VS6) may also played a part. However, a further “push” (R-VS5) was planned in the future.

Discussion

The study aimed to explore the initial experiences and perceptions of HPs about the acceptability and the clinical utility of DETECT e-PEWS and what factors influence acceptability. The discussion draws on the seven constructs of the TFA to explore acceptability (see Fig. 2). The DETECT system aimed to resolve the key barriers that impact on identifying the deteriorating child, including incomplete documentation [29], data input not being in real time [30], incorrect calculation of scores [31], lack of/over-confidence [32], poor communication and/or delayed/non response to alerts [33, 34].

Fig. 2
figure 2

Domains of the Theoretical Framework of Acceptability [21] as applied to findings

Full size image

Overall HPs had positive ‘affective attitudes’ towards DETECT e-PEWS, welcomed its introduction and most overcame the initial problems they experienced. Nurses (D-VS and R-VS) had the most positive initial, and ongoing affective attitudes, whereas resistance by doctors to the use of DETECT e-PEWS was reported by all HPs; similar resistance is evident in other studies [16, 35]. Nurses’ positivity about DETECT e-PEWS reflected attributes reported in other studies including PEWS are systematic [36], reduce errors [9, 35], save time [9] and provide a safety net [32] via an audit trail. It demonstrated good clinical utility (it did what it was supposed to) with those who used it regularly.

The perception of ‘burden’ depended on the HPs’ level of engagement with DETECT e-PEWS and how much it benefited them in daily practice. The initial effort required to learn to navigate and use the system was quickly overcome for HPs in the D-VS group; the benefits accrued were clear cut and real time documentation was faster and not burdensome as seen in other studies of adopting PEWS [35]. However, the burden of carrying an extra device (R-VS group) was perceived as problematic by the doctors although not by the nurses.

‘Ethicality’ was evident. DETECT e-PEWS supported HPs values in relation to generating real-time information that could be easily shared, could reduce the risk of children deteriorating, and aligned with other values (e.g., accommodated clinical judgement, ability to stay at a sick child’s bedside). Implementation of change is enhanced when it fits HPs values) [37].

All HPs understood the purpose of DETECT e-PEWS and how it worked (‘intervention coherence’). HPs talked about how an effective whole system approach [38] such as DETECT e-PEWS can improve situation awareness (SA) (through alerts, raising concern, response to concerns) on both an individual level and at an organisational level (a culture of “everyone taking deterioration more seriously’), as seen in other studies [39, 40]. However, even confident HPs who expressed a clear understanding of benefits for children sometimes circumvented the system, deliberately choosing to revert to ‘paper towel’ (e.g., writing observations on paper towels and later transcribing into the device) techniques; other studies reveal adoption can be inhibited by persistence of paper-based approaches [34, 41].

Doctors were resistant to change, as reported in other studies [35]. The ‘opportunity costs’ were most evident for the doctors (D-VS group) who talked of having to deal with overlapping systems and alert fatigue which is an established barrier [42] particularly relating to patients whose vital signs are typically outside of parameters [32]. Doctors whose engagement was not mandatory may never have been driven to overcome opportunity costs; as seen in other studies, different professional groups can perceive change in different ways [25] and different motivators may be in play [43]. A tipping point exists for change where “evidence of change becomes evidence for change” (43p1); this may not have been reached for the R-VS group.

The ‘perceived effectiveness’ of DETECT e-PEWS was most evident in the reports by nurses (D-VS and R-VS) who valued the system’s effectiveness and clinical utility (e.g. real time reporting, responding quickly, providing accurate data). The system was clearly less effective due to lack of buy-in by doctors; there is insufficient evidence from doctors as to whether the system has clinical utility. All HPs believed that if DETECT e-PEWS was mandatory for everyone, it could be more effective. HPs (R-VS group) were keen to relaunch DETECT e-PEWS. This attitude perhaps acknowledges that implementation is complex and requires sustained effort to overcome barriers [1, 42].

Acceptability in relation to ‘self-efficacy’ was good as DETECT e-PEWS was not considered difficult to use and all HPs were confident in their use, after the initial period of gaining skills. This suggests that clinical utility was robust in this construct.

Acceptability is clearly multifaceted and using the TFA to examine acceptability has shown that a wide lens needs to be adopted to try and understand factors that influence implementation of e-PEWS. Implementation might be better supported by the use of sociotechnical [12] or sociocultural frameworks [44] that take account of the human-interface behavioural aspects of implementing technological change as these are perhaps best placed to support the complexity of change and some of the human barriers encountered in DETECT e-PEWS.

Limitations

Despite efforts to recruit participants the sample size is small when compared to the number of HPs using the DETECT system. Technology acceptance is a staged process [23] and the interviews reflect that staging; the D-VS group had more experience with DETECT e-PEWS than the R-VS group. Doctors are underrepresented in the sample, reflecting both the COVID-pandemic pressures that existed during recruitment and the lower level of engagement with DETECT e-PEWS. The perspectives of the doctors presented in the paper are not directly reflective of the wider population, and do not present the views of those who resisted/failed to engage with DETECT e-PEWS. However, we did ask those who participated for their perceptions of the opinions of other doctors to try and gain a breadth of understanding. A follow-up evaluation, using the same methods, is planned to be undertaken 12 months after this evaluation.

Conclusions

This qualitative study demonstrates how initial acceptability can differ across different disciplines. and how the mandate for change can make a difference in how comprehensively technology is embedded in practice. These differences need to be considered when planning future implementations of this nature. Streamlining the number of devices that need to be carried, making the use of the system mandatory for all HPs, and having strong clinical leadership across disciplines that encourages the use of the system are all ways that could facilitate the embedding of DETECT e-PEWS. The DETECT e-PEWS has generated benefits, but these will remain constrained until HPs in both the D-VS and R-VS groups are committed to the system.