1 Introduction

Research methods are at the heart of research practice: they underpin the process that delivers research outcomes. Researchers learn from, build on, test and refine the findings of others; it should also be possible to learn from, build on, test and refine the methods applied by others. There is an established tradition of presenting and reflecting on research methods in healthcare (e.g. Malterud 2001) and in social sciences (e.g. Braun and Clarke 2006), and increasingly in HCI and Human Factors (e.g. Millen 2000; Rode 2011; Furniss et al. 2011b). By reporting on methods explicitly, the research community can learn from and improve on those methods. In this paper, we focus on strategies for conducting research into the situated use of technology in hospitals.

Healthcare is evolving in many ways, including becoming increasingly reliant on interactive technologies. Yet many current technologies are difficult to use or not fit for purpose, compromising patient safety (Blandford et al. 2014). To address these challenges, an increasing number of HCI and Technology Human Factors (THF) researchers are moving to work in this area, bringing their skills in human factors and in observational studies, but with little prior experience of the healthcare domain. The focus of this paper is on strategies for HCI/THF researchers conducting situated studies in healthcare, with the aim of improving patient safety, and patients’ and clinicians’ experiences, by making future health technologies more fit for purpose.

1.1 Why focus on healthcare?

Situated studies have a long tradition across work settings (e.g. Heath and Luff 1992; Hughes et al. 1993; Harper and Sellen 1995), and the challenges of conducting situated studies are widely recognised. For example, Grudin (1988) noted that “Evaluation of groupware ‘in the field’ is remarkably complex due to the number of people to observe at each site, the wide variability that may be found in group composition and the range of environmental factors that play a role in determining acceptance, such as user training, management buy-in, and vendor follow-through”. Blandford (2013) presents a framework for planning and conducting qualitative studies across a variety of settings. In this paper, rather than discussing generic strategies for conducting situated studies, we focus on those that are particularly pertinent for studying technology use in hospitals. Attributes of healthcare that are shared by few other environments include the following:

  • In many countries, there is a requirement to obtain formal ethical clearance to conduct any study in a hospital or involving patients. The precise details vary by location (and are beyond the scope of this paper), but most require the submission of a detailed research protocol prior to commencing a study, making it impractical to evolve the study design significantly as the study proceeds.

  • Hospitals are populated by both professionals and lay people, young and old, often in very vulnerable situations. Respect for their privacy and confidentiality constrains what data it is possible to gather.

  • Staff are working under high pressure, conducting work that is part-planned, part-reactive, in an environment that is complex: organisationally, physically, and professionally. The work is safety–critical, and there is a widespread culture of accountability and blame that can make people wary of being observed by outsiders.

  • Health is a caring profession, and for the vast majority of staff, technology use is a means to an end; the design of that technology is not of personal interest (beyond it often being a source of frustration in their work).

Healthcare delivery is becoming increasingly reliant on interactive digital technologies, and those technologies need to be fit for purpose. This can only be achieved by developers and hospital management having a deep understanding of user needs and the contexts in which systems are to be used. As discussed by Debono et al. (2010), users typically have limited (semi-tacit) awareness of the details of how they use technology, including workarounds and inefficient interactions. Situated studies are needed because new interactive technologies are being developed and adopted without taking sufficient account of human factors or the ways systems are used in practice. This results in systems where people have to develop new strategies to keep the system safe (e.g. Holden 2011; Perry and Wears 2012), compromising clinical effectiveness and patient safety.

Situated studies seek to understand not just how clinicians and patients work with technologies, but how they adopt, adapt, and appropriate the technologies they engage with and to inform future design. HCI/THF researchers in healthcare need to:

  • make sense of complex (and often inscrutable) structures: organisational, physical, and social;

  • understand sophisticated, knowledge-based work of many different professional groups within the same space; and

  • be in emotionally demanding situations, often with little support.

Although there are challenges to conducting situated studies in any domain, our aim has been to focus on those that are particularly pertinent in healthcare. Typically, situated technology studies in health environments present clear methodologies, implying that everything was well planned from the outset and smoothly conducted. There is little discussion of challenges that were difficult (or sometimes impossible) to overcome and how studies were adapted to be successful in practice. Such accounts embody important lessons. In this paper, we describe lessons learnt in conducting situated studies in hospitals and propose strategies that other researchers can use.

1.2 Why do situated studies?

It is possible to conduct useful studies of healthcare technologies in the laboratory (e.g. Garmer et al. 2002), or in simulated ward settings (e.g. Trbovich et al. 2010); such studies are essential for studying the details of interface design and delivering repeatable results. However, it is impossible to fully comprehend how a technology is used or experienced without accompanying it into the wild. Only there is it possible to really understand how it is used, appropriated, and shaped, and in turn how it shapes the work and the lives of the clinicians and patients who interact with it.

Situated studies help the researcher to focus on the aspects of technology design and use that really matter to people and that have an impact on practice. For example, Furniss et al. (2011a) originally set out to study errors, but quickly realised that in practice there are many minor deviations from the “golden path” and that there is no clear definition of “error”, leading to a discussion of “unremarkable errors”; this had not been anticipated at the outset of the study, but shaped our subsequent thinking on the nature and significance of “error” in the use of interactive healthcare technologies. Likewise, Mentis et al. (2012) originally set out to introduce the Kinect sensor as a time-saving mechanism for surgeons to manipulate images themselves as opposed to finding and instructing a second party to manipulate a mouse or touch screen on their behalf. In fact, surgeons began to view and manipulate images during surgical procedures even more often than they had before, indicating that the value of this touchless mechanism was an interleaving of the images into the work itself to improve decision making and ultimately patient outcomes, as opposed to saving time.

2 Background: What do we know about the pragmatics of situated studies?

Many studies of technology use in hospitals (e.g. Perry and Wears 2012) have reported using ethnographic methodologies, with a focus on observations and interviews. Studies have reported using observational methods such as time study and flow process charting (Carayon et al. 2005b), process-tracing to construct behavioural protocols (Cook and Woods 1996; Seagull and Sanderson 2001; Nemeth et al. 2007), and cognitive task analysis (Cook and Woods 1996). Authors report that notes were taken by hand in all of these studies and later transcribed; in some cases, special observation sheets were developed during the first few observations; in others, a photographic record was created. Carayon et al. (2005b) report that the observers in their study were “complete observers” who did not participate in any way in the process being observed, while Randell (2003) reports that her observations were “unobtrusive”. However, few papers discuss particular challenges in observing technology use or how they were addressed. Most present the studies as if there were no obstacles to access, no difficulties in engaging with participants and no issues about what to observe or how to record it (other than observation sheets being adapted). Any issues are presented in the context of the study results or as an afterthought, in the ‘Limitations’ section, rather than bringing them to the foreground in the methods.

Some studies have made use of audio-visual recording to capture complex, dynamic safety–critical healthcare work (Catchpole et al. 2006; Catchpole 2011; Sarcevic et al. 2012; Mentis et al. 2012). Wilcox (2012) argues that video is essential for ethnographic research. While it may be feasible to gather video data in some hospital spaces (e.g. the operating theatre), it is much more difficult in less structured spaces, and many healthcare organisations prohibit the gathering of video data for fear of compromising patient confidentiality or demand timely deletion of data. For example, Sarcevic et al. (2012) report that video recordings had to be deleted within 96 h, giving little time for analysis. Although Seagull and Sanderson (2001) reflect on the value that audio-visual recording would have had in improving the richness of data, they do not discuss in detail what barriers prevented them from gathering such data.

As well as observations, researchers (e.g. Gurses et al. 2009; Rajkomar and Blandford 2012; Adams et al. 2005; Sarcevic et al. 2012) report conducting interviews with participants to: improve their understanding of particular events in a complex setting; supplement or validate observational data; or seek clarifications on the reasons for actions performed by participants or devices that appeared out of the ordinary. Some studies (e.g. Holden 2011; Miller and Xiao 2007; Ross et al. 2014) are based entirely on interview data. Some of the challenges of setting up interviews are discussed by Rajkomar and Blandford (2012) and also by Sarcevic et al. (2012), who note that “Informal interviews with trauma team members usually took place during down time. Interviewing immediately after the events was difficult because physicians and nurses followed the patient to the next hospital unit”. This discussion of difficulties is the exception rather than the rule in reporting.

Researchers without a clinical background need to make sense of what they are observing; some report having consulted system manuals (Cook and Woods 1996) and other medical documents related to the procedures being observed (Seagull and Sanderson 2001; Mentis et al. 2013). But again, little attention is paid in reporting the overall learning curve that is needed when entering an environment as complex as a hospital to conduct observational studies. Indeed, with few exceptions (e.g. Husch et al. 2005; Pennathur et al. 2013 who both made use of multi-disciplinary teams for data gathering), the academic background of researchers is rarely discussed.

Siek and Connelly (2006) is unusual in reporting lessons learnt while working with haemodialysis patients in a hospital, where they gathered user requirements and deployed an early prototype system for tracking dietary and fluid intake. They note the need to check regulations (e.g. about data recording) and deal with practicalities (e.g. how to organise research instruments within the study space). They also report on the challenge of ensuring that paper prototypes conformed to infection control procedures (being laminated and routinely disinfected). Finally, they note the need to be sensitive to patients’ needs, to dress appropriately, and to be flexible and adaptable in data gathering.

To build a repository of experiences, a recent edited volume (Furniss et al. 2014) brings together case studies written by researchers conducting studies in healthcare; each case study reports on the experiences of researchers in one project, exemplifying themes such as managing the emotional demands of working in hospitals or addressing the practicalities of deploying a novel technology with nurses in a developing country. A companion volume (Furniss et al. forthcoming) synthesises guidance for researchers under the themes of: ethics and governance; readying the researcher; establishing and maintaining relationships; practicalities of data collection; practicalities of implementing novel technologies in healthcare; and achieving impact through research.

In summary, the vast majority of research reports follow the classical experimental practice of presenting the final “cleansed” method with minimal reference to the learning process of the researchers, or any difficult decisions they had to make about how to conduct the study. In this paper, we discuss lesson learnt in conducting situated studies of technology use in hospitals, with a particular focus on strategies for the researcher who is a hospital outsider: neither healthcare staff nor patient.

3 Methods

This paper is not the outcome of a single study, but of the authors reflecting on experiences of conducting situated studies in healthcare. It has hallmarks of a confessional ethnography (Rode 2011): where researchers reflect on their relationship with people in practice, how they conducted themselves in practice, and how this affected data gathering and analysis, to demystify the fieldwork process.

This paper has had a long gestation period. Themes for this paper were initially identified through discussion between Blandford, Furniss, and Rajkomar at UCL, early in 2012, including both broad themes (e.g. gaining access, respecting privacy, and managing potential patient and clinician concerns) and vignettes (stories of incidents that illustrated or teased out important points). As we recognised important gaps in experience, we recruited further authors: Berndt, O’Kane, and Owen were junior researchers studying technology use by clinicians and patients, bringing their recent experiences as novices in both situated studies and healthcare; Catchpole, Mentis, and Randell are established researchers in healthcare who brought years of experience of conducting situated studies across different clinical settings in the UK and the USA; and Mayer is a consultant oncologist with experience of facilitating studies. The expanded team of authors iteratively validated and extended the narrative through both discussions and exchanges of partial documents. It became apparent that there were important differences between:

  • studies that involved clinical professionals, where patients were involved by virtue of their treatment, and studies where the main engagement was with lay people;

  • studies of novel technologies in clinical use and of clinical practice with existing technologies; and

  • the experiences of HCI/THF researchers and of clinicians as observers.

In order to be concise, this paper focuses on HCI/THF researchers studying the use of existing technologies by clinicians in hospital contexts.

While working on early versions of this paper, it became clear that many other researchers also have valuable experiences to report. This encouraged us to propose a workshop on this theme at the CHI 2013 conference; the output of that workshop is two books (Furniss et al. 2014, forthcoming) authored by the workshop participants. We have intentionally kept the development of this paper and the books separate, so that each offers a different (but complementary) perspective on strategies for conducting situated studies of technology use in healthcare.

Through iterative development, we aimed to eliminate generic themes pertaining to situated studies across contexts (though we recognise that the themes that remain do generalise in many ways to other settings). We also removed themes that related to generic human factors studies in healthcare, including obtaining ethical clearance and informed consent. The remaining themes were gradually refined through discussion of experiences and the identification of more focused challenges and solutions.

We have structured the paper according to key themes that were identified, including vignettes of experience to illustrate the issues and approaches to addressing those issues: getting started; managing relationships with clinicians and patients; being in the healthcare setting; and cultural divides in reporting.

4 Getting started

One set of issues concerns how to get a study “off the ground”: how to identify a good match between research question and study setting; get institutional buy-in from key individuals; and work with constraints such as time.

4.1 Matching a research question with a study setting

There are many triggers for HCI/THF research projects in healthcare. Some may focus on a particular setting and have clear questions that seek to improve clinical care through the design and use of technology. Others may have particular driving questions that will contribute to improved healthcare in the longer term (e.g. understanding errors or multi-tasking strategies around technology, or user experience of a particular system), but not have immediate access to a suitable study setting.

Without sufficient preparatory investigation, there is a risk of focusing attention on phenomena that occur rarely, making the study time-consuming or yielding limited data. For example, O’Connor (2010), a Masters student with Blandford, was particularly interested in workarounds and keen to conduct a study in Accident and Emergency [A&E, also known as the Emergency Room (ER)]. His was a short (3 month) project. Blandford and O’Connor worked with the Matron in A&E in one of the local hospitals to identify the focus, keen to identify a project that was of interest to staff as well as to the researchers. At the time, the A&E was replacing the blood gas analyser and the Matron wanted to know whether the replacement model was easier to use than the earlier model. In practice, the new model was so much easier to use that staff did not develop many workarounds with it, and less data was gathered than expected. Despite our prior discussions with staff, the study setting turned out to be poorly suited to the proposed research question (though the Matron was very happy with the findings!).

Healthcare is very variable, with differences across wards, conditions, types of technology, and hospitals. One risk mitigation strategy is to study multiple settings. This helps highlight aspects of design or use that might otherwise go unnoticed. For instance, to address the issue of limited data, Blandford and O’Connor extended his study to an Intensive Care Unit. This highlighted important differences in use between the settings, which enriched the analysis of causes and effects (O’Connor 2010). Similarly, Randell and colleagues’ study of handover practice across a variety of clinical settings (Randell et al. 2011b) revealed not only that the term “handover” captures a variety of collaborative practices that vary in both form and content, but also how the level of heterogeneity amongst those participating in the handover impacts how the handover takes place.

It is best to engage interested and engaged clinicians and/or patients early on to ensure you are addressing a productive and useful question. Clinicians and patients are embedded in the study context and may have a good sense of where HCI/THF issues and opportunities exist in the clinical environment. For example, Catchpole was approached by an intensive care clinician who had the idea of learning from auto-racing pit-stop crews about how to improve handovers. The team approached Ferrari to understand their techniques and adapted the approach to the clinical context (Catchpole et al. 2007). Similarly, Randell et al. (2013) gave demonstrations to patients and members of the public of a workstation for viewing digital histopathology slides, a tool for supporting clinical work. Patients suggested that digital slides could be used as a tool to support communication between clinicians and patients about cancer, which the team had not previously considered.

4.2 Getting institutional buy-in

Hospitals are complex organisations where it can be difficult for an outsider to identify lines of responsibility and where changes in management and structure occur frequently. It is necessary to identify suitable areas to conduct research and who manages those areas.

When a clinician has a direct personal interest in a project, and engages as a project partner, that can smooth the path to setting up studies that work for clinicians, patients and researchers. For instance, final access to the ER (Mentis et al. 2013) is a story of networking and fortuitous events. The search for a research site began by asking acquaintances whether they had any contacts in an ER. Planting the seeds early led to people remembering the request when an opportunity did present itself. Within a year, Mentis had contacts at three different hospitals. One was a friend of a friend: a resident who was finishing her emergency medicine rotation, who introduced Mentis to an ER physician with an interest in information systems, which led to a productive collaboration and paved the way for gaining ethical clearance, hospital approval, and ultimately access and support.

4.3 Working with the available time

In healthcare, time for preparation is coupled with complications in negotiating access and gaining ethical approval. Timescales and deadlines mean that extended periods of preparation are not always possible. For example, many UK Masters projects (e.g. O’Connor 2010; Gant 2011) have to be completed within 3 months, and other studies have pressures to deliver results on a short time frame.

It is important to anticipate delays in gaining formal access and to use that time effectively in preparation: for example, doing extensive background reading and planning. Sometimes, it is possible to negotiate informal access to gain familiarity with the study setting. For instance, Mentis volunteered in a local ER, working a weekly 4-h shift where she was responsible for activities such as patient transport, laboratory runs, and assisting nurses and doctors in medical procedures. During this time, she could observe the relationship and coordination activities between staff. This allowed her to become more comfortable with observational methods and more knowledgeable of ER work, local practices, and culture. From these preliminary observations, she discovered the central role of the charge nurse station, where the majority of interactions and discussions occur, and that a lot of information is written on temporary documents such as napkins and scraps of paper. This understanding helped focus her subsequent studies.

5 Managing relationships with clinicians and patients

However much rapport is developed with healthcare professionals, the HCI/THF researcher will always be an outsider. Staff may initially be suspicious of them, concerned that they are actually observing on behalf of hospital management or an outside body, and patients may be curious about who this person (who is not staff, patient, or visitor) is. As social scientists, Van Der Geest and Finkler (2004) propose that the ethnographer can choose between three roles: staff, patient, or visitor, but the HCI/THF researcher studying technology use is none of these: they are bringing their knowledge of technology and the study of interaction to the situation.

5.1 Overcoming resistance to being observed

Senior doctors are typically more comfortable with being observed than junior staff or nurses, as they are frequently observed by medical students. Some staff may be resistant to participating in observational studies; for example, individuals may feel threatened by studies of human error when interacting with technology.

Sensitivity to those being observed is a key skill. It is important to anticipate and allay concerns, making it very clear what the purpose of the study is and giving well-founded assurances of confidentiality. We have found it beneficial to make it clear that we are there to learn from staff rather than observe them, following the apprenticeship stance advocated by Beyer and Holtzblatt (1998). Even nuances in the words used (e.g. learn vs. observe) can affect how people receive the researcher.

In some of Blandford’s and Furniss’ studies, they have been particularly interested in errors, and these need to be explored tactfully. A naïve view is that when anything untoward happens, the researcher interviews the people involved about it as soon as possible. According to their protocol, every error had to be brought to the clinician’s attention, since non-clinicians are not qualified to judge the severity of any error. However, for minor matters, this was found to be counter-productive because mistakes are embarrassing and drawing attention to them disrupts the flow of work. Rather, minor errors were discussed after the nurses had recovered from the situation and long after the occasion had passed, and then reported (Furniss et al. 2011a) at a level of abstraction that aimed to respect the confidentiality of individuals or teams, accepting that this may leave some issues unresolved.

It is also important to be aware of the effects of one’s actions as an observer. For example, if data recording (e.g. writing notes) is directly related to particular actions, communications, or errors, those being observed may become sensitised to the behaviour of observers and adapt their own behaviour accordingly. In order to make the nurses feel at ease, Mentis showed her notes when they asked her what she was writing. This not only engendered a sense of trust, but also helped them understand what she was interested in and encouraged them to provide her with further information.

5.2 Building rapport with staff

Staff may have different degrees of interest in and engagement with the research. In our experience, nearly all clinicians are cooperative and loosely supportive, but it can nevertheless be difficult for HCI/THF researchers without a clinical background to build rapport with healthcare professionals. Staff may value their own work over that of researchers; for example, Catchpole was asked by a clinician: “have you done anything useful recently?”. Staff may be too busy to talk, and non-clinical researchers often cannot help with tasks because they are technical or sensitive, or for infection control reasons. Although small talk is often effective in building rapport, it can occasionally backfire. Furniss once asked a nurse where they go on their break; she smiled and said that they did not get breaks. When he laughed to reflect her mood, she turned serious and said it was not funny that they do not get breaks.

While being an apprentice has its place, as discussed above, it also has limits, and there are times to contribute HCI/THF expertise. For example, Furniss was working with a nurse who had tried everything she could think of to silence an alarm for a critically ill patient; he identified for her further actions that she had not tried that eventually achieved the desired effect. There are also times to simply be useful; for example, when Randell was observing in a paediatric ward, the nurses seemed to appreciate the fact that she took the time to talk to the children, particularly those whose parents were not able to visit often. Other small actions might include moving chairs, getting water for a patient, making an internal call, answering surgeons’ pagers, and getting a replacement for faulty equipment.

Where possible, it is valuable to join staff in common rooms during lunch and breaks. This facilitates informal chats, small talk, discussions other than work, and further opportunity to explain the study and ask research questions. This helps build rapport, but it is necessary to be invited in. It is also helpful to fit with local practices; Furniss was invited to sleep in the treatment room for a few hours, an unofficial practice that helps nurses get through the night, when studying night work on a ward. He accepted because, as well as being very tired, this fitted with community practice.

It is important to recognise how stressful and demanding patient care is: clinicians may not want to participate when tired. Consent processes should make it easy for people to opt out and in again. If a whole ward had had a bad day, it can be best to simply abandon data gathering, even if times when fatigue and stress are heightened would be interesting for understanding work and interaction under adverse conditions. Tomorrow is another day.

5.3 Engaging with patients and visitors in hospital settings

HCI/THF studies in hospitals most commonly focus on the work of healthcare professionals. However, building rapport with patients is also important—if only to make them feel comfortable with the researcher’s presence and obtain their consent to observe activities that take place around them. To patients and their visitors, the researcher may be hard to make sense of: not a member of staff (clinician or support worker) and yet working in the hospital.

One approach is to ask the clinical staff to introduce the researcher to patients and visitors (establishing the legitimacy of the researcher). If staff are too busy, an alternative is to introduce oneself. Seeking consent can be a way to engage with patients initially. In doing this, it is advisable to ask nursing staff which patients to approach. For example, if a patient is suffering from confusion, they will be unable to provide informed consent.

Our collective experience is that patients are typically happy to take part in research: many patients appreciate being able to chat, and like the fact that the researcher is someone they can talk to who is not a healthcare professional. When a patient declines to take part, that is a sign that the consent process works: that those who do not want to take part feel free to say so.

6 Being in the healthcare setting

Healthcare presents challenges that are not experienced as powerfully in other research settings. These include making sense of the complexity of clinical work; being in the right place at the right time; interviewing people whose time is rarely diarized; and being an outsider in an intimate space.

6.1 Getting familiar with a new and complex work environment

As noted elsewhere, healthcare is a complex system, characterised by various interrelated parts, fast pace, and a high degree of unpredictability. Entering any system can be overwhelming at first (Wong and Blandford 2003), but healthcare is particularly complex. Whereas experts might see a clearly defined structure, lay analysts often start with a chaotic image (Norman 2010). Consequently, an observer will be unable to develop a coherent understanding of what he or she observes, not to mention revealing the underlying patterns. Inevitably, expertise develops over time; in most cases, though, the initial learning curve is very steep.

It is useful to prepare well prior to data collection. Useful resources include clinical descriptions of conditions, device manuals, and support forums for patient experience. For example, before studying clinical handover, Randell read existing literature on clinical handover, including research studies and guidelines; this highlighted the need to look at the work of preparing for handover as well as informal discussions between staff before the official handover. A preliminary visit to the site helps to develop a broad understanding of the setting and provides insights to the medical paradigm with its special features and language. It also facilitates an appropriate choice of research focus (Jirotka and Wallen 2000), and the choice of data capturing method (Fisher and Sanderson 1996). Depending on the duration of a project, it may be beneficial to treat the first few days of data collection as a pilot study: an opportunity to test the methods of data collection before formal data collection begins.

It may also be helpful to manage complexity by focusing on a particular theoretical perspective. For example, Bardram (2009) based his Activity Based Computing approach to understanding clinical interactions on Activity Theory (Kaptelinin and Nardi 2007), while Pennathur et al. (2013) based their observations and analysis on the SEIPS model (Carayon et al. 2005a). One method Blandford, Furniss, and Rajkomar have developed and used is the DiCoT framework (Furniss and Blandford 2006), a way of structuring Distributed Cognition (Hollan et al. 2000) analysis that has proved successful in handling complexity in the healthcare domain (e.g. Rajkomar and Blandford 2012). Sarcevic et al. (2012) also report basing their analysis on Distributed Cognition.

A third strategy, adopted by Gant (2011), a Masters student with Blandford, was to embrace naivety regarding both correct use of the device and the corresponding clinical practice, to avoid having preconceptions regarding what would be observed. This can help with noticing details that might be taken for granted by someone with more experience. This can be uncomfortable, as the researcher will inevitably come across as naïve within the study setting, but can also enable the researcher to observe aspects of technology use that would go unnoticed by someone with more experience. This will ideally be followed by a second phase of data gathering with knowledge of correct device operation, to pick up on complementary observations based on deeper understanding of intended use.

6.2 Being in the right place at the right time

As an individual researcher in a hospital, a further issue concerns knowing where to be, what to observe, and who to work with at any given moment, particularly in a large, multi-roomed ward with many staff and patients around. This arises due to the unpredictability of many activities that might be the focus of study, as Gant (2011) describes in relation to her study of infusion device use in a haematology ward:

It was sometimes difficult to follow through with a whole sequence of preparation and infusion. I might find a nurse with a set tray ready prepared and follow them to a room straight away, missing the preparation. The nurses might divert from their intended activity and carry out a new task when the first was not possible (patient in the toilet, for example). This meant some of my notes are of incomplete sequences, having been interrupted between preparation and infusion.

One approach is to shadow particular clinicians, but this only works well if they frequently perform the activity that is the focus of the research. If the study focuses on the use of a particular technology or on particular activities, it is often necessary to be patient: there can be a lot of time while waiting to observe the phenomenon of interest; this can be a good time to talk informally with staff, building rapport and developing a richer understanding of the context within which the behaviours of interest take place. Another approach is to engage staff in the research so that they inform the researcher when the activity of interest is going to be performed. For example, while nurses’ handovers typically take place at a regular time and in a regular place, medical handovers may be harder to track down: time and the location may vary depending on ongoing work, and might take place in corridors or cafes. Randell handled this by sometimes shadowing the people whose handovers she wanted to observe and at other times relying on clinicians to inform her when and where the handover would take place (Randell et al. 2011a).

It is usually beneficial to work with, rather than trying to ignore or change, the constraints of the situation. For example, O’Connor (2010) occasionally had to leave the study site because it was too busy and crowded:

The resuscitation room could go from a state of quiet isolation to distributed and organised action in a matter of two or three minutes. This may happen when two patients enter in close succession and an emergency case follows shortly afterwards. At these busy times the medical team consisting of doctors, nurses and observers are added to the core team of nurses who are responsible for ensuring the resuscitation room is always ready for patients. These additional staff would quickly fill the room and require space to work. […] These enforced breaks enabled the researcher to reflect on the recently observed interactions and to consider the aspects of interest to be investigated on resumption of observations.

It may be necessary to accept that data gathering is partial, and to plan the analysis accordingly.

6.3 Scheduling interviews

Scheduling interviews can be challenging, as many clinicians are over-worked, with little time to spare for other activities. Interviews may be cancelled at the last minute due to pressures of work. Staff may not be available during their breaks: when they have a break, they need a rest, not to talk more about work.

There can be a tension between a relaxed “being there” approach and a more formal questioning approach. The latter is more “scientific”, and easier to describe in papers; it is easier to gather evidence and demonstrate that the work has been conducted systematically and objectively to give confidence in the validity of the findings. However, it may not be possible to get to the heart of the matter to the same degree as “being there”: the more relaxed and informal interactions with nurses and patients are, the more natural they are, and hence the more open and honest they are likely to be, but the data is more ephemeral and difficult to report clearly.

We have found it most effective to adapt our interviewing style to suit the situation, accepting that this might be less easy to describe cleanly when reporting the study. For example, Rajkomar and Blandford (2012) conducted ad hoc, opportunistic interviews, asking nurses a few questions at the bedside whenever possible. These interviews were different from conventional interviews in that there was not sustained attention from the participants, and they were different from contextual inquiries in that the questions could not be asked during the activity; rather, questions had to be noted down to be asked at opportune moments, which might be minutes or hours later. To make the most of these small pockets of time, in terms of getting as many questions answered as possible, Rajkomar and Blandford (2012) maintained a spreadsheet to keep track of outstanding questions and selected questions from it to ask. This has some of the strengths and limitations of “deep hanging out” as described by Rode (2011), while also having some of the strengths and limitations of more direct and systematic (and more easily reported) data gathering.

6.4 Being an outsider in an intimate space

HCI/THF researchers are not medically trained and are therefore outsiders. The work tempo does not fit, there is often nowhere convenient to sit or stand without getting in someone’s way, it is hard to help out in such specialised work, patient spaces can be very intimate and circumstance can change quickly, e.g. a nurse setting up an infusion pump might also change a patient’s incontinence pads or need to undress them to check intravenous access points.

In everyday life, we move into different spaces that have different social norms and unwritten rules about how to act. It can be difficult to determine the unwritten rules in different areas of the hospital, often leaving the researcher feeling uncomfortable. The researcher has to learn when it is acceptable to interrupt a nurse to ask a question by being sensitive to the ebb and flow of their work. Different hospital contexts, e.g. surgery, wards, and out-patient areas, have different activities and atmospheres. Even within one space the atmosphere can change quickly, e.g. when a patient arrives or leaves on an open ward or when someone has received bad news (Harrison and Dourish 1996). On top of this, there are seriously ill patients, which is emotionally demanding, with scenes that can disturb the squeamish. Any one of these everyday contexts would prove challenging, but even more so when they are lumped together.

Engaging with patients and their stories can be one of the most challenging aspects of working in a hospital, even if engagement with patients is limited. For example, Randell remembers crying in a hospital car park after learning that a child on a paediatric ward, who she had played with and developed a fondness for, had died from a hospital-acquired infection. In another situation, Catchpole observed a paramedic crew attempting to resuscitate and de-fibrillate an extremely sick patient while he sat in full view of the family, looking anxiously to him for hopeful clues in a difficult situation. While the nurses have practices for responding to these events, non-clinicians are not always included in these practices.

Researchers in healthcare need maturity and resilience; those who are unprepared to deal with such difficulties are best advised not to work in challenging healthcare settings. Even for researchers who feel prepared to work in such contexts, it is important to anticipate and plan for difficult situations. This does not concern just the researcher out in the field but also the rest of the research team.

For the person supervising the research, important first steps are bringing it into the open as a topic for discussion and establishing strategies to deal with difficult situations. For example, if the researcher is working late and is faced with a distressing situation, is it okay to phone and discuss it? Where practical, it is helpful to have two researchers involved in data collection, so that they can discuss things together. Visiting a university counselling service before starting data collection can be beneficial, to talk about the possible situations that will be faced and how to deal with them. For the person undertaking the research, it is important to reflect on the emotional toll of the work, to discuss this with the supervisor, and to seek support as needed, whether from a formal counselling service or drawing on personal support networks.

The researcher may also receive support from healthcare professionals when faced with such situations, and while the emotional aspects of the work can be challenging, they can also be humbling and enlightening.

7 Reporting the findings

There are many cultural challenges in interdisciplinary working. These can emerge early on in the process (when writing a proposal to conduct a study) as well as later (when writing up the research). Here, we focus on reporting. It is important to have due regard for the readership (which might be, for example, HCI, medical informatics, or clinicians) and to have due regard for participants.

7.1 Addressing the readership

Healthcare has an established tradition of Randomised Controlled Trials, which is widely agreed to be the most appropriate method for testing the efficacy of new therapies. This methodology is perceived as being objective and scientific, and those trained in it can often be hostile to qualitative methods, which are inherently subjective, lack a pre-determined hypothesis, and may also lack generalizability (Concato et al. 2000). Consequently, some clinicians consider observational studies to be less credible than quantitative, outcomes-based research. Amongst clinicians, positivism is still a dominant philosophy. Nevertheless, qualitative studies have an essential role in understanding and improving health service delivery and technology design: RCTs yield no insights into process, practices, or experience. To maximise impact, it is important to report findings to clinical audiences as well as technology/HCI audiences.

It is important to justify the value of the questions being addressed and the legitimacy of the approach being taken. It can be helpful to author with clinicians, who understand that audience better, as well as providing a clear account of the research method and its claim to rigour and validity. It is likely that as observational research becomes better established and more widely recognised in healthcare, there will be a larger body of the literature to invoke and to model new research on.

We should be aware not only that our presence as individuals in hospitals may be alien, but our very philosophies are challenging the history, current teaching, and perceived status of many professionals. In this sense, learning to communicate our findings across this divide—and ultimately publish our research in respected clinical journals—is essential to bring our worlds together and have long-term impact on practice.

7.2 Respecting participants

Professional sensitivities may be unearthed when we expound the relationship between humans and systems to an audience who have been taught to believe that only bad doctors or nurses make mistakes. The idea that equipment, processes, teams, systems of work, and organizations can have effects on performance that are beyond conscious control can be disturbing for professionals whose confidence is paramount to their work, and who believe it all comes down to what they do. Realising that they do not have control over things that might make them have an adverse event is personally and professionally challenging.

In one conference presentation, Catchpole was heckled by a clinician with a self-proclaimed expertise in Human Factors who, when presented with a photograph of an error-inducing design, proclaimed that they “had never made that mistake in 20 years” even though it had been brought to his attention by clinical colleagues who had made exactly that mistake.

We find it a useful check on our writing and presentations to assume that participants will read our work, maybe recognise themselves in it, and reflect on how we have represented them. Will they feel that we have been fair and respectful? Will they feel that their contributions to the study have been valued? Would they wish to take part in any future study? If the answer to any of these questions is ‘no’, then the presentation needs revising (while remaining true to the data and analysis). Where possible, we encourage participants to read and comment on the reports of the work with them. Occasionally, it may be possible to involve participants as co-authors, depending on the theme of the paper and the kind of involvement they have had with the work.

8 Discussion

We have described lessons we have learnt about issues that need to be considered in conducting technology studies in healthcare and presented strategies that other researchers can use. In this discussion, we reflect on three themes that cut across those issues and strategies: managing the gulf between intention and practice; benefits of engaging in observational studies in healthcare; and an agenda for the future.

8.1 Intention and practice

One of the consequences of addressing the issues outlined above is that a gulf can emerge between what is intended and what is done. This may arise in the initial study design and gaining access, through the writing of “methods” sections of papers which typically outline what is intended, to the reality of taking the context on its own terms.

In ethnography, the term “study design” is often avoided, researchers instead proceeding with research by negotiating objectives with relevant stakeholders and making preliminary observations in order to devise a tailor made research method depending on the objectives set and the particularities of the work setting (e.g. Wisner 1995). However, given the requirements to obtain formal clearance for observational studies in many countries (Van Der Geest and Finkler 2004), such an approach is rarely possible within a healthcare setting, where it is difficult to avoid having a preliminary study design. However, it is only on entering the setting that the researcher can test the feasibility of the proposed study design. When data collection begins, it is necessary to review the proposed methods as the researcher’s understanding of the realities of the setting improves. This can include amendments to protocols that have been approved by ethics committees. Although we all have contact points with the healthcare system (when we, friends, or relatives are ill), it is difficult to anticipate the complexity and practical challenges of working in different healthcare contexts ahead of time. So there is inevitably a co-evolution between planning and practice in any observational study in a new healthcare context. This co-evolution can be eased by early visits to the context and expert input in the planning phase so that what is negotiated is practical in that context.

Negotiations also take place beyond the planning phase. Taking the context on its own terms involves developing and applying a repertoire of approaches to respect the constraints (physical, temporal and affective) of the study environment while also exploiting opportunities that the environment affords. For example, in Rajkomar and Blandford’s (2013) study of nurses’ use of infusion pumps in the ICU, it was necessary to adapt the data gathering methods as the study progressed, based on what was found to work in practice. Doing so allowed them to glean useful findings while minimising disruption to the setting being studied. Adaptations will often have to work within protocols that have been approved by ethics committees, but there can be room for manoeuvre just as long as the details and intent of the protocol are respected. For example, interviewing nurses in short bursts rather than longer periods should be fine as long as they still consent. Adding interview topics well beyond the protocol, and interviewing patients when this has not been agreed, can break the conditions on which access was agreed.

Each healthcare context is likely to be different from others. For example, studying medical devices that are used in the operating room where patients are unconscious poses different challenges compared to studying use with patients on wards who are conscious; whether wards are open or patients have single rooms will affect how the researcher engages with them; whether technology remains in predetermined locations and is used in predictable ways or not influences how easily the researcher can plan a technology-focused study. Each context and study focus presents its own challenges, to which the researcher has to respond in the planning phase and while in the context.

8.2 The benefits of doing situated research in healthcare

We have discussed many challenges to conducting HCI/THF research in hospitals, and yet all authors of this paper have chosen to work in healthcare. There are good reasons for this.

Firstly, and most importantly, studies in healthcare have the real potential to improve lives—most obviously those of patients, but also those of healthcare professionals. Without a good understanding of clinicians’ and patients’ needs and practices, the design of future technologies will remain “hit and miss”; HCI/THF research can make an important difference.

Secondly, the participants are a pleasure to work with. Whether they are clinicians who care deeply about their work or patients who generously share their experiences and insights, people are generally eager to engage, within the limits of the time or energy they have available. It is a real privilege to work with people who welcome you into their worlds, which are often very different from your own. Over time, you build an understanding of these rich and complex environments and work with people who bring an interesting complementary expertise, and have a real need for usable, useful technology that gives a positive user experience. Further, healthcare is a domain that is extreme in many respects—not so much in the physical setting, but in the variability of settings, in the fact that it is embedded in society and in all of our lives, that it involves both professionals and lay people, and that technology is essential but not generally the focus of attention. These factors all make it a fascinating and rewarding domain to work in.

8.3 Looking to the future

Although, as described above, there is an established practice of conducting situated studies of technology use in healthcare, there has been little discussion on how to conduct such studies. Consequently, there has been no strategic discussion of best practice in conducting such studies. This demands designing the future: establishing best practice across different study contexts and, where possible, creating infrastructure that makes it easier for future researchers to do good research in this challenging area. We have identified three areas for development, as follows.

The first requirement is that researchers share best practices. This paper is, itself, an attempt to share our own learning regarding challenges and strategies for addressing them, based on pragmatic and practical experience of conducting research in healthcare. We are, however, aware that our experiences are based on our practices in four institutions across two countries and may only generalise to a limited extent to studies on other continents and in other cultures. As discussed above, a workshop at CHI 2013 has resulted in a set of case studies (Furniss et al. 2014) and a guide for graduate students preparing to conduct fieldwork in healthcare (Furniss et al. forthcoming). We advocate that future papers reporting on situated studies in healthcare should, where possible, include discussion of particular challenges that were faced and how they were addressed, so as to build up an informal repository of knowledge and best practices in this area.

If the situated study of technology use in healthcare is to grow, there needs to be more explicit cross-disciplinary training, rather than people having to “pick it up” as they go along, as generally happens at the moment. Although there are a few courses on human factors in healthcare (e.g. http://humanfactors.ca/aboutus/hf101/), we are not aware of any with an explicit focus on HCI/THF and technology design for healthcare. This should include advanced-level courses: for example, cross-training MD and PhD students to develop an international cohort of people who have appropriate interdisciplinary skills and understand the cultures of both communities.

Finally, as noted above, we have omitted any discussion of formal ethics processes. However, ethics in practice goes beyond locally defined procedures. For example, it is important to consider what the benefits of participation in a study are. It is rarely possible to make an immediate intervention, or design change, to improve the experience of clinicians or patients within a study setting, so benefits are usually indirect. It has not been our practice to pay participants (Grady 2001) who are simply being observed doing their work, but benefits of participation should be proportionate to the costs of participation, where costs might include time to participate in interviews or a sense of being intruded upon. Other motivations for participating in studies that have been identified in other contexts include altruism and having an opportunity to tell your story (Gysels et al. 2008). None of these motivations is necessarily compelling in the context of medical devices. So it is important to identify something that is valued by participants, to reciprocate for their participation, such as personally useful or interesting feedback. It is also important to manage expectations in terms of how quickly the research may result in practical outcomes: healthcare is a large and complex system where the outcomes of research take time to influence practice. While it is necessary to understand current practices to make informed changes, it is not necessarily the case that understanding always leads to immediate change.

9 Conclusions

In this paper, we have highlighted challenges and provided strategies for conducting situated studies of technology use in hospitals. We have emphasised the importance of building rapport with participants and of valuing their expertise that complements our own. We have also emphasised the value of good preparation together with a flexible attitude, identifying and working with opportunities (such as times of enforced reflection) that the nature of the work affords, rather than fighting against situational constraints. This involves studying activity in the setting on its own terms, as also reported by O’Brien and Rodden (1997). This eliminates power relationships between researcher and situation. Ultimately, as a researcher, you create a dialogue with the situation, working with it, responding to it and picking up on opportunities, not expecting to impose your will or your plan on it.

Being responsive to the situation can sometimes conflict with cultural expectations of planning, conducting, and reporting studies, which typically expect precision about processes and outcomes. We have drawn out differences between planning and practice and highlighted strategies for exploiting opportunities in unexpected places. Our aim has been to make visible many of the challenges faced by HCI/THF researchers conducting observational studies in healthcare and propose an initial repertoire of strategies for addressing those challenges.