1 Introduction

This paper investigates participants’ experiences of participating in a multi-disciplinary, researcher-practitioner design project. The goal of the project was to design, develop and test a simulation-based training environment for prehospital emergency care.

Simulation-based approaches are important for training and testing critical tasks and situations that can be challenging to conduct in real emergencies [1], and hence very useful for pre-hospital care settings. Current training approaches have a number of limitations, and richer and more holistic approaches and simulation technologies are needed. This can be attained through a combination of different technologies and design solutions [2, 3]. The work within the project propose a richer model for simulation that, compared to current training, provides higher realism and the ability to train a wider range of situations, tasks, and processes in several different simulated physical environments.

Technology design for healthcare settings is challenging. It is characterized by high degrees of complexity, and might include a wide range of use locations, stakeholder organizations, rules and regulations, healthcare personnel, tools and technologies. In prehospital work settings, a number of challenges are added, e.g., concrete factors such as time constraints, weather, portability requirements and communication infrastructure, but also to formal guidelines such as work protocols; security and safety aspects; different organizational and thus funding structures; and, the need for “hard evidence” in terms of improved patient outcomes [e.g. 4, 5]. This calls for multidisciplinary, holistic design and development approaches that take domain knowledge from multiple perspectives into account. Hence, it is important to involve end-users as well as other stakeholders during the whole technological development cycle.

The relationship between user participation and system success in terms of user satisfaction is strongest when the task and system complexities are high [6]. UsersFootnote 1 can participate in a wide variety of way along several dimensions, e.g., as purely informants, full members of the project team, in parts of, or during the whole process [7]. One way of doing this is through participatory design (PD) [8].

The project presented in this paper applies the views of PD as stated by [9]: “every participant in a PD project is an expert in what they do, whose voice needs to be heard; that design ideas arise in collaboration with participants from diverse backgrounds; that PD practitioners prefer to spend time with users in their environment rather than ‘test’ them in laboratories” [9, p. 213]. Hence, user participation should take place in a way that gives high value to the project and the final product as well as makes everyone involved feel comfortable in the situation [7].

PD have been used since the 1980 s’ [10], and also in prehospital care settings [11, 12]. While there is an extensive body of research on PD, few studies have explored how project members experience their participation as full active members of a PD project [13, 14]. This is one of the key issues, emphasized by [15] that need to be addressed by the research community. Thus, this paper aims to investigate how participants experience the work process and participation in a multi-disciplinary, research-practitioner design project in the context of designing simulation-based training for prehospital emergency care.

2 User Participation

User participation is defined as “a set of behaviors or activities performed by users in the system development process” [16, p. 149]. The importance of taking users’ perspectives into consideration has been known for more than 30 years, and ever since Gould and Lewis [17] introduced the three fundamental principles of user-centered design, usability experts and others have struggled to address them, not least the principle “early focus on users and tasks” [17, p. 300]. Failing to understand the users has repeatedly been reported as a reason to failed system development projects [6, 7, 16, 18].

A widespread concept to address this is to let the users participate in the design process; user participation has been identified as one of the most important factors for system success [19]. Involving users is also important for other reasons, such as democratic empowerment that allows them to take active part in decision-making that influence them and their work, as well as competence development and learning [7, 20].

However, if not managed properly, user participation can negatively affect the development process, e.g., not achieving the intended objectives, becoming more difficult and less effective [6, 16, 19]. Wu and Marakas [21] investigated the impact of different aspects of user participation on perceived system implementation success. Aspects that influenced were, e.g., the users’ perceived extent of participation, overall responsibility, top management support, user attitudes, system initiation, as well as congruence between user participation and user status in terms of functional expertise.

A movement that provides an approach for user participation is participatory design (PD) whose roots are participatory democracy [9], where cooperative design work is emphasized and users are viewed as co-designers [22, 23]. Olsson and Janson [24] have identified several values that are important to create a good atmosphere between users and designer: mutual respect; active participation on equal terms; common goal; common language; to listen; refrain from immediate implementation thinking; simple tools that reveal what work is about; and to document findings.

3 Research Setting

3.1 Project Initiation

The focus of this case study is on the work that was conducted in relation to the development of a new emergency training facility for a Prehospital Care Center (PCC) in western Sweden. In order to further develop their prehospital training, the PCC had acquired an advanced Laerdal patient simulator.Footnote 2 They wanted, however, to improve and develop the overall training approach, and the use of the patient simulator, so that the training better would reflect the full prehospital process and its complexity with respect to physical contexts and diversity in tasks, and they initiated the work to plan and build a new emergency training center.

Hence, the project initiative came from the practitioners at the PCC who contacted a group of informatics researchers at the local university. Together, they outlined an initial idea and set up the research project. The purpose of the facility is to enable advanced realistic simulations, covering the entire prehospital work process from call-out to delivery at the emergency care unit, including different work dimensions, e.g., medical treatment, transportation, communication and caretaking [2, 3].

They also invited experts in prehospital emergency care and in information science from another university. A joint project proposal was developed and partially funded through regional support, so that the project could start in January 2014.

3.2 Project Scope and Composition

The scope of the project was a joint effort to design, prototype, and test a holistic approach to training the entire care chain through rich simulation and serious games-components.

The project has an action design research approach [25] where all participating researchers and users are full members of the project group. Purposeful interventions are conducted in the users’ organization, and an artefact is developed.

The project group consists of 15 members from two different universities, three academic fields, and users from (and related to) the PCC. The researchers include: informatics and information science researchers specialized in serious games, HCI/UXD, and, ICTs for emergency care; prehospital care researchers specialized in emergency care, pre-hospital training and simulation, and ICT for prehospital care. One of these had also been working in parallel as a paramedic for 15 years up until the project started. The participating users included paramedics, ambulance training officers; ambulance training administrators; an emergency physician; and a healthcare strategist. Thus, the users have been active participants during the whole process.

The authors of this paper (researchers in informatics and information science), are active participants in the project and therefore have personal and inside experience of it.

3.3 Work Process

The project started with field studies conducted by information science and informatics researchers in order to get a thorough understanding of the prehospital work process as well as the current practices of emergency training. This was followed by iterations of design and prototyping of simulation environments, tools and scenarios. All project members contributed to the evolving simulation-based training concept and prototype.

The project was led by an informatics researcher, with a collaborative, inclusive, PD inspired strategy. In the project, all experts (users as well as researchers) had influential power on areas within their own domain and/or specialization.

The work took place in several ways; during grand meetings with all project members, functional meetings between members working on different aspects, e.g., scenario building, and construction of the technical and physical parts of the prototype, as well as frequent communication via phone, email and Skype.

The first part of 2014 was devoted to conceptual and physical design iterations, including user testing and a small pilot study. The autumn of 2014 mainly focused on preparing for and conducting a pilot test, an experiment where the simulation environment was going to be evaluated by 24 paramedics. The research presented in this paper focuses on the process up until the experiment, that is, about half-way into the project.

4 Research Method

In order to investigate project participants’ experiences of working in the project, open-ended interviews were conducted with 12 of the participants (total n = 15). Three project participants were omitted: one prehospital researcher could not be interviewed due to time constraint issues; and, the two paper authors (one informatics and one information science researcher).

A semi-structured interview guide was used. This approach helps in making certain that the basic parts of the interviews remain the same and reminds the interviewer of the main subject areas. A conversational interview style was established. The questions covered participants’ perceptions of the project’s direction and design, meeting culture, possibilities to influence, as well as general experiences of pros and cons of the project overall. Examples of questions were: If you were asked by an uninitiated to describe our project, how would you describe it? What is your viewpoint of the project goals? What is your opinion of the content of the meetings? What have you been able to influence so far? What have you not been able to influence so far, but that you wanted to?

All interviews were conducted at the participant’s work place, and each lasted for about one hour. All interviews were recorded and subsequently transcribed. In the first stage of the data analysis the transcripts were coded in three high-level categories: a) expressions for that the project works (it was obvious during the interviews that everyone was very satisfied with the project so having a reverse category was not meaningful), (b) possible reasons for why the project works, and (c) problems or potential problems that the interviewees experienced or were worried about. All coded text segments was provided a condensed description. From the condensed data categories emerged, e.g., resources and time; roles, interpersonal relations; and, personal commitment. Further analysis was then made within each category participant group: (a) users, (b) prehospital care researcher, and (c) informatics researchers.

5 Results

The all-pervading theme is that all participants express strong satisfaction with the project so far. Some even stress that the project exceeds their expectations and their previous experiences of collaborative projects.

There are, however, also reports of problems and difficulties. These illustrate a continuous balance between intertwined experience-affecting aspects, primarily related to four main themes: (1) project organization and leadership; (2) interpersonal relationships and project vision; (3) lack of resources and support; and, (4) user-researcher collaboration.

5.1 Project Organization and Leadership

One of the themes that most clearly emerge from all participants’ perspectives is their appreciation of how the project is organized. They felt that there is a clear and effective organization, reasonable meeting frequency, and an open, constructive and pleasant discussion climate. The users feel included in the dialogue, they feel that they both contribute and are being listened to.

All participants express that even though there are large differences in background, knowledge and competence; there is both an openness and willingness to understand each other and also to make an effort to explain things and be understood. Although the users find the academic and research-related aspects that now and then are discussed in the meetings are somewhat irrelevant, several of them consider the meeting discussions to have expanded their view on simulation-based training of prehospital emergency care and increased their understanding of its complexity and possibilities.

Participants in all categories express uncertainty about the formal overall goals of the project. Moreover, when asked to describe the project, they provide fairly different views of the project scope and its goals. Still, they consider everyone involved to be working in the same direction. The project leader acknowledges these differences; his strategy is to align the different goals and motivations for participating, and make them work together side-by-side.

Thus, in spite of both the inconsistencies and uncertainties concerning how the actual project goals are understood, and the differences in project members’ backgrounds, the project is considered as running smoothly and is experienced as highly positive by all the participants.

5.2 Interpersonal Relationships and Project Vision

Participants’ high engagement in the project was one of the most striking themes that emerged from the data. Across all participant categories expressions like “enjoyable”, “interesting”, “exciting”, “motivating”, “fun”, “win-win”, was used in relation to the different interview themes.

This seems to be strongly related to two factors. First, the interpersonal relationships between the individuals are reported to be very rewarding. The participants stress the joy of working together. They perceive each other as highly committed and easy to cooperate with. Second, the project vision is considered to be of high societal importance and all participants strongly believe in the new training concept. The potential benefits and usefulness of the new training approach are stressed; however, noteworthy is that the character of its importance and potential vary depending on participants’ backgrounds. For instance, some emphasize the project’s immediate relevance for improving the current training, while others stress more long-term potential for technology development or improvement of healthcare in a wider sense.

However, some participants, mostly in the informatics researcher category, think that their primary interest has been somewhat diminished due to limitations in technical resources. Still, they enjoy participating in the multi-disciplinary setting and find it worthwhile. In particular, the users and the informatics researchers express that they enjoy their collaboration and the mutual learning.

Thus, personal engagement and feelings of delight is salient, which can be interpreted as closely related the positive interpersonal relations and leading spirit of the project vision. Together, these seem to have significant weight in the balance between positive and negative overall experience of the project, not least considering the fact that there are a number of problematic aspects of the project, namely limitations in funding, time and resources.

5.3 Lack of Resources and Support

There are some intertwined problematic aspects that worry the participants; aspects that potentially could overturn or harm the project. These include: lack of financial resources; and, lack of explicit support for the long term goal, i.e., building a new emergency training facility for the PCC. To build such a center requires decisions from many levels; political, regional top management, and PCC management. Although there are many strong advocates that support the idea of a new emergency training facility, no final decisions are made at any level. This means that the future for a fully equipped training facility is unsecure, which is a source of worry for many of the participants.

Furthermore, the regional financial support that was awarded for the project (to support the design, prototype, and a pilot test of the suggested training concept) does neither allow funded project time for all participants, nor full coverage of all project-related costs. Thus, several members report that they had to “find” other ways to create possibilities to participate, i.e. by using their spare time, competence development time, and/or ask for funding from other sources to manage some of the project’s expenses.

These fairly severe difficulties affect the participants negatively, but - interestingly enough - not to such an extent that they seem to have an effect on the permeating positive experience of the project. In spite of these problems, the participants find the collaboration rewarding, are highly committed, invest their own time, and strongly believe in the project vision and its potential societal benefits.

5.4 User-Researcher Collaboration

The collaboration between users and researchers is working satisfyingly according to the participants. However, there are indications of potential pitfalls that could undermine the feelings of satisfaction in the project.

There are signs of tensions in the user-researcher relations, e.g. the users express hesitation regarding some of the research-related activities in the project; the researchers emphasize scientific rigor and the users stress the importance of “getting things done” and taking a concrete form. The researchers need to publish the results appear a bit strange to the users. Nevertheless, in this project the users respect this need in spite of this “peculiarity”, and the users found the academic perspective meaningful overall. For example, as expressed by one of the users; it feels empowering that academics are interested in our daily work and what we do.

Initially users found the lack of prehospital care knowledge among informatics researchers surprising. However, this tension vanished as the informatics researchers participated more in the daily activities at the ambulance center, engaged in meetings and design workshops, and thus acquired a better contextual understanding. After these initial problems, the users perceive the work with the informatics researchers as well-functioning and mutually beneficial. One of the users believes that the absence of a “gap” between them is because they are from diametrically different fields and thus already know that there are obvious differences between them - which make everyone extra careful to be clear, explain things and not assume healthcare or informatics-related expert knowledge.

There is however some tension between the users and the prehospital care researchers that seems to be stemming from them being from the same realm. Based on previous experiences, the users feel that healthcare researchers in general often use fancy language, and talk about their (the users’) work in ways they do not recognize. There might also be a certain amount of competition in how training should be done, what does work, what the real current problems are, and how to improve training by new innovative approaches.

The prehospital researchers on the other hand, highly value the multi-professional project constitution and the different perspectives this includes, stating that all are dependent on each other’s perspectives and competences. They are however also aware of the potential problems. In particular, one researcher who previously worked as an ambulance nurse acknowledges that the users might feel that there is too much of talk about research design and publications.

Nonetheless, regardless of the described tensions the positive overall experience of the project remains. A probable factor is the reported mutual dependence, with complementary competencies and little rivalry and prestige. Our interpretation is that these feelings are strong and to some extent compensate for or help to overcome differences and tensions.

6 Concluding Discussion

In this paper, we explore participants’ experiences half-way into a project in which design, prototyping, and testing of a simulation-based training environment for prehospital emergency care has been conducted. The project is a joint multi-disciplinary, user-researcher effort, where the views of PD as stated by [9] have been applied. The aim of the study reported in this paper was to investigate users’ experiences of their project participation in order to identify and understand what makes such a project be running well.

The results show that this is a much more complex pursuit than identifying a list of “success factors” or “best practices”. Instead, there are several aspects, such as interpersonal relations, project leadership, and user-researcher collaboration that vary along a pros- and cons-dimension or axis, where pros can be viewed as reasons that evoke and increase positive feelings and cons can be seen as occasions that impose and amplify negative feelings. What seems important in order to achieve a positive overall participation experience is that the total pros and cons balance; the “sum” of pros outweighs cons that under other circumstances severely might harm the project.

However, many questions remain. Future studies should investigate and concretize the interplay between these aspects and dimensions, as well as the reasons and occasions that influence the participants’ experiences in these types of projects. The next step for us is to conduct another round of interviews with the project participants to conclude the now (February 2015) finished project. In that study we intend to deepen and expand the investigation and our understanding of the findings reported in this paper.