Introduction

Emergency care is characterised by short patient-clinician encounters, at times critically ill or severely injured patients and considerable uncertainty for patients as well as the care team. For clinicians, emergency care provides a complex, dynamic work environment. The emergency care staff is faced with patients suffering from a wide range of medical conditions, rapid decisions need to be made by prioritising and communicating within the professional team, all the while respecting patients’ autonomy and supporting patient participation [1, 2]. In various healthcare settings, patient participation is widely advocated [3, 4]. It has been shown empirically to improve patient outcomes, satisfaction, and safety [5,6,7] and is viewed as an important component of high-quality healthcare [8].

Although clinicians strive to provide the best of care and aim to involve their patients, their actions may not always fully reflect their intentions. Wiman and Wikblad observed an ‘instrumental behaviour’ in staff working with emergency patients, which was explained as a ‘lack of emotional involvement but not insensitivity’ [9]. Knowledge and attitudes among clinicians, as well as time constraints and organisational factors, have previously been reported to influence patient participation [10, 11].

For patients, the situation in emergency care is uncertain as they do not know what to expect or how to act [12]. Elmqvist et al. used the metaphor of a ‘game board with hidden rules’ to illustrate the uncertainty inherent in this situation [12]. In emergency care, patient experiences have been reported to be highly dependent on interactions with the care team concerning communication, information, and involvement [13,14,15].

Initiatives to improve patient participation in emergency care have often focused on shared decision-making [16], which has been claimed as essential to meet ethical obligations towards emergency care patients [17] and has therefore been suggested as the default choice for medical decisions, whenever possible [18]. In addition to shared decision-making, patients receiving emergency care have described other important aspects of patient participation, such as getting attention, being respected, being informed in a manner understandable to the patient, and having a sense of control [14, 15].

However, there is no universal definition of patient participation and related concepts (e.g., patient-centred care, patient engagement) in the literature, and these concepts are often used interchangeably. Common elements in descriptions of the related concepts to patient participation include empathy, respect, engagement, relationship, communication, shared decision-making, holistic focus, individualised focus, and coordinated care [19,20,21]. In our research, we have been guided by the theoretical framework of Cahill [22], who argued that ‘patient participation’ requires that 1) a relationship between the patient and the clinician exists, 2) the gap of knowledge/information between the patient and the clinician is reduced, 3) the clinician transmits a degree of power to the patient, and 4) the patient takes part in intellectual or physical activities, and 5) benefits from these activities. Cahill also illustrated the hierarchical order of the neighbouring concepts ‘patient involvement’, ‘patient collaboration’, ‘patient participation’ and ‘partnership’ as a pyramid: ‘patient involvement’ and ‘patient collaboration’ form the basic level, ‘patient participation’ constitutes the middle, while ‘partnership’, the ideal and most equal form of a patient-clinician relationship, forms the top.

In this framework ‘patient involvement’ and ‘patient collaboration’ seem similar to ‘patient participation’, but do not meet all defining criteria for ‘patient participation’. Eliciting the patient experience is highlighted as relevant in ‘patient involvement’, but compared to ‘patient collaboration’, ‘patient involvement’ does not stress a two-way information flow or the need of reducing the knowledge gap. ‘Patient collaboration’ implies an engagement in discussions on the patient’s part, but in contrast to ‘patient participation’, ‘patient collaboration’ means co-operation in the care process, while ‘patient participation’ does not necessarily imply an agreement between the patient and the professionals [22].

An important criticism of the concept of patient participation has been that it increases the risk of ‘off-loading’ professional responsibility on a patient [23], which would be considered deeply unethical. Patients in emergency care are sometimes unable to participate actively (e.g., due to pain or a deteriorating condition). However, often this situation may be transient and reversed when the patient feels better. Thus, although many patients may be less capable to participate in an acute phase of illness, they need to be involved by being informed about what is happening and by having an opportunity to express their wishes [12, 14, 15]. We argue that even in a situation in emergency care where full patient participation is not achievable or appropriate, at least as a minimum standard, the foundational level of patient participation, i.e., patient involvement and collaboration [22] should be considered.

Enhanced patient participation have been called for decades already, especially in regards to patient safety [8, 24]. But scientific methods to evaluate the level of patient participation are still missing in several domains of healthcare. Previously published instruments for assessing clinician-patient interaction have mainly focused on one-to-one interactions, e.g., consultations in primary care [25,26,27,28,29,30,31] or long-term care relationships, such as elderly care [32] and cancer care [33,34,35]. In emergency settings, however, instruments for assessment of team behaviours that focus on patient involvement and collaboration are still missing, especially with regards to this specific environment that includes medical examinations and/or procedures.

To effectively support clinicians in changing their behaviour towards increased patient involvement and collaboration, interventions such as guidelines or trainings might be developed and tested. Such behaviour change interventions will require a scientifically grounded way to assess the behavioural components of patient involvement and collaboration.

For complex, multidimensional behaviours such as patient involvement and collaboration, reliable and valid performance assessment is challenging, especially in dynamic work settings such as emergency care. One established way for performance assessment is the use of behavioural observation systems that have been developed systematically and can thus serve as a basis for valid and reliable assessment and feedback. Within healthcare behavioural observation systems that can help to identify performance gaps and document behaviour change have successfully been developed in various clinical settings for clinical skills [36, 37] as well as for social and cognitive skills that underpin effective performance [38,39,40,41].

The aim of this study was to develop and test a structured observation tool for research purposes of emergency teams’ behaviour regarding patient involvement and collaboration.

Methods

Theoretical and methodological frameworks

The general approach for developing behavioural observation systems has been described in a condensed manner by Schmutz et al. [41] and has successfully been applied in different fields of healthcare since. This approach contains the following steps: 1. Developing a draft of the behavioural observation system based on available literature (reviews), guidelines, and other scientific sources by subject matter experts, 2. Establishing expert consensus on the content of the draft observation system by performing a Delphi review, 3. Designing and pilot testing the final behavioural observation system, and 4. Defining item weights, again based on expert consensus.

When developing the content of the observation tool, elements included were inspired by, but not limited to Cahill’s model [22]. Thus, we adopted an inductive approach. Tool development was guided by the systematic methodological approach for developing checklists for clinical performance assessment as described by Schmutz et al. [41] and Brogaard et al. [36]. As recommended by these authors, we drew on various data sources in different steps of the development process such as published literature, interviews and observational data and expert feedback. The development and testing of the tool are illustrated in Fig. 1.

Fig. 1
figure 1

A schematic illustration of the development and testing of the behavioural observation tool for patient involvement and collaboration in emergency care teams (PIC-ET tool)

Full size image

Development of prototype: content and rating scale

In the first step of the tool content development, we identified observable behaviours by clinicians regarding patient involvement and collaboration in the following data sources:

  1. 1)

    Published observation tools (n = 16) for other care settings addressing competencies in the domain of patient participation [25,26,27,28,29,30,31,32,33,34,35, 42,43,44,45,46]. We included the observation tools which were described in a review from 2020 by Ekman et al. [47], and did not find additional observation tools in the literature to include.

  2. 2)

    Qualitative studies representing both the patient perspective [15, 48] and the provider perspective on patient participation [49]. We found it important to incorporate qualitative studies in the content development phase and selected three studies that provided particularly rich descriptions of patient participation from different angles.

  3. 3)

    Transcripts from group interviews (n = 13) with health care professionals (n = 39) in emergency settings. This interview data was collected within a parallel on-going research project where teamwork and patient participation in emergency teams in northern rural Sweden is being studied. The objective was to evaluate if there are differences between behaviours in teams that are physically co-located and teams that are geographically dispersed and communicating via telemedicine solutions. Descriptions from the interview transcripts of team behaviour related to patient participation in emergency care on a more general level were extracted and used in the content development phase. Behaviours related to the specific ‘tele-emergency’ situation was omitted for this study, however included in a recently published paper [50].

  4. 4)

    Video recordings (n = 4) of simulated emergency team training scenarios which were based on real-life emergency cases and reviewed by a multi-professional group of experienced clinicians. This data was also collected within the parallel project described above. The video-recordings are included in other studies that have not yet been published.

The video recordings show multi-professional emergency care teams of three, during in-situ simulation training sessions at their workplace caring for a standardised patient (actor) presenting scripted scenarios of urosepsis or myocardial infarction. In both scenarios, the patient is fully conscious but deteriorating. She expresses opinions and questions about her medical condition, treatment and need for transfer, and displays concern, e.g., about a next of kin who is depending on her.

The second step involved the identification of behaviours, including behaviours both promoting and hindering patient involvement and collaboration. Authors HD and TM inductively searched and charted possible observable behaviours in the different data sources that could be of relevance for the tool development. Then the behaviours were sorted and grouped according to conceptual similarity. The behaviours were consequently thematically grouped into categories and phrased as items capturing the essence of each category. The categories were generally consistent with previous literature on patient participation and its related concepts [19, 20, 22, 51]. For instance, common denominators described in patient participation literature are the trustful relationship, information exchange, and shared decision-making [20, 22, 51] and inspired the naming of items and categories in the observation tool prototype.

To allow for the assessment of team behaviours, behavioural anchors were defined for each item using a scale ranging from ‘no patient involvement and collaboration’ to ‘high patient involvement and collaboration’. The behavioural anchors were inspired by the descriptions that were included in the previously developed chart of observable behaviours.

Delphi rounds: expert consensus on prototype content and rating scale

A first prototype of the observation tool was submitted to an international expert panel (with members from Sweden, Norway, Denmark, Switzerland, and Australia) in an email-based Delphi technique [52]. The expert panel (Additional file 1) consisted of researchers with expertise in patient participation (n = 3), clinical emergency experts (n = 3), and patient representatives (n = 3). They were identified by approaching a patient association and through our professional networks. The clinical experts and the patient participation experts all had substantial research experience. The three patient representatives all had experience of being patients in emergency departments. Their professional background and medical experiences varied.

We performed a total of three rounds, where the experts were asked to assess the content and rating scale of the prototype (Fig. 2). The Delphi rounds were mainly e-mail based. However, the patient representatives expressed a need for additional support in completing the Delphi rounds. Thus, their contribution was collected over the phone or by in-person support. The experts were asked to review and revise each category and individual item (content), including the behavioural anchors (rating scale). They were encouraged to comment, remove, add, merge, or edit the content and behavioural anchors of the rating scale in the prototype. During the rounds, all experts remained anonymous to one another, however they all consented to their names being published in the reporting of our work.

Delphi rounds: expert agreement on the weight of importance scores

In addition to reviewing the content, the items in the prototype were scored by the experts for their weight of importance to patient involvement and collaboration in emergency care on a 5-point Likert scale (1=‘not important at all’, 5=‘very important’). The aim was to achieve agreement on all scores. As a definition of agreement we used Brogaard et al’s model [36] i.e., that agreement is achieved when all scores of the experts’ for one item fall within three neighbouring ratings on the Likert scale.

Testing prototype feasibility

To test the feasibility of the prototype, we invited two raters (i.e., clinicians with experience in emergency care settings and familiar with the use of behavioural observation tools in research who had not been involved in other parts of the development or testing process) to independently use the prototype to assess two recorded simulated emergency scenarios with a standardised patient (described above). They provided feedback on the phrasing of the items and the rating scales, which informed the last modifications of the prototype. We also asked them how easy it was to use the prototype and to assess the tool’s potential for research.

Evaluating PIC-ET tool reliability

When the content development, the Delphi rounds and the feasibility testing, were finalised, the prototype was named the PIC-ET tool (patient involvement and collaboration behaviour in emergency teams). To assess the PIC-ET tool’s reliability, two clinicians were selected based on their clinical and research experience of the emergency care setting. They received a half-day rater training where we thoroughly explained and answered questions about each item and the behavioural anchors, and then by practising and discussing the assessment using two video-recorded simulated emergency scenarios (not the same videos used for the actual reliability testing). The reliability raters then used a prototype of the observation tool to rate patient involvement and collaboration in 17 video-recorded simulated emergency scenarios. The videos were 14:09 min in average (range 09:49 min-19:31 min).

Statistics

Inter-rater agreement was calculated by using statistical software Stata 16.0. Due to the two different types of rating scales, i.e., four or two-level ratings, different indicators for the inter-rater agreement were used.

  1. 1)

    Intraclass Correlation Coefficient (ICC) was calculated for items with four behavioural anchors. Both, absolute agreement and consistency of agreement were calculated by using a two-way random-effects model [53].

  2. 2)

    Cohen’s kappa [54] was calculated for the binary data, i.e., items with two levels, as well as for the categories that included binary items.

  3. 3)

    Per cent agreement was calculated for two of the binary items where Kappa values could not be used, due to lack of variability in the data.

ICC < 0.5 was considered ‘poor’, 0.5–0.75 ‘moderate’, 0.75–0.9 ‘good’, and > 0.9 ‘excellent’ [53]. Kappa values < 0.4 were considered ‘poor’, 0.41–0.6 ‘fair’, and > 0.61 ‘excellent’ [54]. P-values < 0.05 were considered significant.

Results

First prototype: content and rating scale

A first observation tool prototype was composed based on various data sources and a thematic sorting of observable behaviours related to patient involvement and collaboration (i.e., items, n = 29, and categories, n = 6). Depending on the content of the item, a rating scale included either four or two behavioural anchors. For most items, a four-level scale was found appropriate.

Content validity based on expert opinions

In a Delphi technique, where nine experts with different perspectives participated, the first prototype was edited, and the PIC-ET tool took form. The content validity of the tool was assessed. All nine experts completed the three Delphi rounds. The Delphi rounds are illustrated in Fig. 2.

Fig. 2
figure 2

Development of the observation tool prototype by using an expert panel in a Delphi technique procedure

Full size image

Based on the feedback from the first round, categories and items of the observation tool prototype were edited, merged, or deleted. This revision of the prototype resulted in 5 categories and 22 items. During round 2, additional comments concerning item formulation were made as well, and changes in the prototype were made accordingly. In round 3, after slight revisions, the experts all agreed on the prototype content. Thus, content validity was assessed by the international expert panel: all categories were found relevant for patient involvement and collaboration, and all items were found to fit well in the respective categories. The categories in the final prototype were: ‘Relationship’, Sharing power’, ‘Information Exchange’, ‘Safe and caring environment’, and ‘Social Circumstances’.

Expert agreement on weight of importance scores

In Delphi round 1, the nine experts assigned all items a ‘weight of importance’ score for patient involvement and collaboration in emergency care (i.e., 1–5), which in round 2 were presented to the experts as group means and medians. In round 2, the weight of importance scores were reviewed and revised, and agreement was reached on 15 of the 22 items’ scores during this round.

In round 3, the experts answered whether they could agree with the group means of the weight of importance scores for seven of the remaining items, where agreement had not been achieved in round 2. For one item (# 5 ‘Physical positioning’), agreement was not achieved as one of the experts did not agree on the group mean. Thus, the group mean for that item was recalculated by keeping the disagreeing expert’s rating. Agreement of opinion on the items’ scores between the nine experts was achieved after the three Delphi rounds. The weight of importance scores are presented in Table 1 and the Delphi rounds’ consensus procedure is illustrated in Fig. 3.

Table 1 Categories, items, and weight of importance scores in the final PIC-ET tool
Full size table
Fig. 3
figure 3

Illustrative examples of the expert agreement on weight of importance scores during Delphi rounds

Full size image

Feasibility

After having used the tool to assess team behaviour in video-recordings, the feasibility raters provided useful feedback which was implemented by slightly rephrasing two items for clarity, however without changing the meaning or content. As indicated in Table 2, the tool was found quite easy to use and relevant for its different purposes.

Table 2 Feasibility raters’ assessments of the utility and relevance of the observation tool prototype
Full size table

The feasibility raters had also been encouraged to give us further opinions on the tool, and their reflections from the feasibility testing were used to develop the recommendations for using the PIC-ET tool (Additional file 2).

Reliability

The results from the reliability evaluation with two independent trained raters indicate ‘fair’ (Kappa 0.52) reliability across all items. Some items reached ‘good’ and ‘excellent’ reliability. Half of the 22 items obtained significant ICC or Kappa values. Reliability for two items (#7 and #8) could not be calculated due to a lack of variability in the data. However, the per cent agreement for these two was 100%.

ICC and Kappa values for categories and items of the observation tool are presented in Table 3.

Table 3 Inter-rater agreement: ICC (consistency of agreement and absolute agreement) and Kappa values on categories and items
Full size table

Discussion

In this paper, we introduce the PIC-ET tool (see Additional file 3) – to the best of our knowledge, the first observation system that focuses on emergency team behaviour related to patient involvement and collaboration. The PIC-ET tool was developed for observing teams caring for an adult patient without severe cognitive impairments. For patients, the emergency setting differs from many other contexts by the sometimes frightening and unpredictable experience and the short-term care encounter with a medical team. As context may matter for the patient’s possibility to be active and involved, the PIC-ET tool has been developed for the contextual factors of emergency settings, including the encounter with a team, instead of a one-to-one clinician-patient consultation.

The PIC-ET tool is divided into five categories and 22 items reflecting key elements of patient involvement and collaboration. It is important to note that the items and categories are often interrelated. For example, item #8 ‘Clarifying with the patient his/her preference concerning the level of information and involvement/collaboration’ has a natural influence on how item #14 ‘Information and discussion about diagnostics, treatments, procedures, and plan ahead’ is handled. Understanding a patient’s preferences and wishes is central to providing opportunities for patient involvement and collaboration [55]. Thus, it should always be done with respect for the patient’s preferences and ability in the moment.

Ekman et al. (2020) reviewed previous assessment tools and checklists within the area of person-centered care [47]. As pointed out by the authors, a majority of the assessment tools lacked a clear conceptual framework to guide the construction of their tools [47]. We relied on the theoretical framework on patient participation, as described by Cahill [22]. By choosing to focus on the foundational levels in this framework, ‘patient involvement and collaboration’, expectations of what can be achieved in an emergency can be kept on a realistic level, without setting an upper limit to patient participation, or even partnership.

Cahill’s theory, as most other theories that include patient involvement and collaboration, was not developed to fit the special context of short-term patient-team encounters. In addition, the emergency setting challenges the conditions for reciprocal interaction. In addition, Cahill’s theory does not describe manifestations of the concept in terms of behaviours. Therefore, we found it necessary to adopt an inductive approach to find relevant content for the tool in this setting.

Another criticism on previously developed checklists and assessment tools within this domain is that patients have not been involved in the development of the instruments [47]. In the PIC-ET tool, the patient perspective was included both by the selection of literature in the early development phase, as well as by including patient representatives in content validation phase.

The development of the PIC-ET tool followed methods described in previous work [36, 41]. The rigorous process of identifying and systematically including team behaviours related to patient involvement and collaboration in different data sources, including perspectives of patients and healthcare professionals, as well as observations of actual emergency teams during simulated patient encounters in an emergency care setting should be viewed as a strength of our study. In addition, an international expert panel contributed to the content development in a Delphi process, which showed promising content validity of the PIC-ET tool. Comments from both the experts and the feasibility raters indicate that the PIC-ET tool has qualities suitable for research purposes.

Limitations

Our study is not without limitations. Developing an observation tool for interpersonal behaviours in a team context is a complex undertaking. We acknowledge that there are issues that raise questions and may limit the applicability of the PIC-ET tool. At this stage, the PIC-ET tool could be viewed as a promising early version or a pilot.

Although we used international and peer-reviewed literature, as well as an international expert group, the PIC-ET tool was developed and tested in a Swedish context. Therefore attention to assessors’ cultural and contextual competence for the observed setting should be given, as it may impact the perceptions of patient involvement and collaboration. Also, legal requirements, what is considered polite/rude, and what treatment options are available may differ between countries and cultures.

A limitation in our study was the relatively small sample of scenarios and teams included in the development and testing phase. We chose to use simulated scenarios since it is a feasible method and provided us the opportunity to observe and record team behaviour in relation to an emergency patient, without intruding on real patients’ integrity. When designing the cases, we could include opportunities for interaction and patient involvement and collaboration in multiple occasions during the scenarios. We believe that ‘live emergencies’ should be a future step in the validation of this instrument, although this may be more challenging regarding ethical and regulatory aspects to carry out.

We used a Delphi technique in the development and validation of the PIC-ET tool’s content. Delphi groups are often of different sizes and there is no gold standard for the number of experts. Although other studies using expert groups have reported even smaller groups [41, 56], it is fair to say that nine experts is a relatively small group. We however carefully chose the experts and found them highly motivated to provide useful feedback and to complete the rounds. As stated by Boulkedid (2011), a heterogenous expert group can be considered a strength [56]. We found that the experts’ different perspectives enrichened the feedback and contributed to the progress of our work. Also, we assured anonymity between the experts during the Delphi rounds, as this is considered one of the key characteristics of the Delphi technique [57]. Although a larger group could have been valuable, we found the Delphi technique a well-functioning step in the development process.

The reliability test, which was performed on data supplied by two independent raters, showed a ‘fair’ (Kappa 0.52) overall interrater reliability. Although partly inconclusive results, we found the overall interrater reliability as acceptable considering this is a first version of a complex behavioural observation tool. Nevertheless, the reliability assessment of the tool could benefit from further testing, e.g., by using additional clinical scenarios in different settings.

Since this observation tool is newly developed, and this is the first report on its reliability, we suggest caution when using the PIC-ET tool, as its scientific robustness would benefit from being further established. We welcome others to contribute to the validation of this observation tool.

Future research

For an improved scientific value, the PIC-ET tool’s psychometric properties could be further evaluated. The PIC-ET tool may have the potential for short-term care contexts other than the emergency care setting (e.g., pre-hospital care) and would thus need further testing and potential adaptation in such settings. It could also be argued that patient involvement and collaboration cannot fully be assessed without the contribution of the patient. Thus, another important step in validating the PIC-ET tool would be correlating actual patient experiences to team assessments in ‘live-emergencies’.

Although the aim of the study was to develop an instrument for research purposes, throughout the process it became clear to us that there may be other future potential areas for its use. Patient participation is highlighted in modern curricula in health professionals’ education [58]. Instruments for assessment and feedback for both students and professionals have been developed to raise awareness, ultimately to strengthen patient participation in clinician-patient interaction [25, 42, 43]. Education and training may, thus, be a future area of the PIC-ET tool. The PIC-ET tool could provide an opportunity for an outside view grounded on a 360-degree perspective. The tool could be valuable in education or for feedback on quality improvement projects, as well as a basis for fruitful discussions among clinicians to further improve patient involvement and collaboration. Of course, also for this the PIC-ET tool may need adaptation before validating it in the field of education and training.

We regard the domain of patient involvement and collaboration in emergency settings undertheorized and we present an opportunity to study behaviours in this context, which in turn may contribute to further theoretical development.

Conclusions

The PIC-ET tool, an instrument for observing emergency care teams’ behaviour for patient involvement and collaboration is introduced for research purposes. The tool has been developed systematically and content validity, as well as feasibility, were found to be high. Overall inter-rater reliability was fair. The scientific value of the PIC-ET tool should be established by further testing and validation. Potential future areas of use for the PIC-ET tool that have been identified are education and training, with the ultimate goal to contribute to improved patient involvement and collaboration in emergency settings.