Keep your coats on: augmented reality and sensework in surgery and surgical telemedicine

Abstract

In Norway, the health sector has recently been looking to the petroleum industry for inspiration with respect to innovative solutions for telemedicine and patient safety. In this article, the potential for and challenges associated with augmented reality (AR) tools and practices in surgery and surgical telemedicine are investigated. Work practices in co-localised surgical operations in a neurosurgical operating theatre are investigated and analysed using central organising principles for distributed collaborative work as envisioned by Integrated Operations in the petroleum industry. Digital representations are found to take on a central role in surgical work, and they show a promising potential for the future inclusion of neurosurgery into the portfolio of telemedicine. However, the article warns against organising telemedical work processes according to theoretical principles for division of labour that are not rooted in actual practices. In line with a constructivist approach to ontology, there are many realities that may be augmented, and inadequate work processes may cause construction and augmentation of inadequate realities and hence suboptimal outcomes of surgical procedures. This possibility of AR enabling both desired and undesired outcomes is in the article referred to as the Janus face of augmented reality.

1 Introduction

Surgery has traditionally been associated with craftsmanship. With ever more tools and technologies being introduced and acquiring integral roles in the practice, however, the nature of surgery is changing. Just as the work of representing and visualising is so central to scientific work in general (Coopmans et al. 2014), so it is gradually becoming for surgical work. Traditionally, access to the patients and their bodies has traditionally been acquired through visual inspection. This inspection has required that the body be physically opened and that flesh, bones and organs be moved. However, the development of representation and visualisation technologies and new operating instruments and techniques has made access to the internal body possible without radical intervention and damage of bodily structures.

Representational technology in surgery has a long history. The microscope has been around for a long time, making visible smaller details particularly important for operating on finer structures. Still, the microscope does not affect the need for invasive methods to access the objects of interest.¹ With X-ray technology, it became possible to investigate internal bodily structures, exploiting the different behaviour (with respect to radiation absorbance) of the different bodily structures. With support from images produced in X-ray examinations, one could decide whether an operation was necessary, what methods would be adequate and the location of the object of interest. Computer tomography (CT) builds on X-ray technology, using the same basic physical principles, but produces cross-sectional and more fine-grained images of bodily structures. Using a completely different physical principle, magnetic resonance (MR) can also be used to give information about internal bodily structures, but instead of exposing the patient to harmful ionising radiation and exploiting absorbance differences, MR exploits the difference in natural oscillation of different tissue types when exposed to radio waves under the presence of a strong magnetic field. Ultrasound imaging uses yet another principle: different tissue types reflect high-frequency sound signals differently—reflections that may be used for producing images in two or three dimensions. In the last 20 years, ultrasound apparatuses have become smaller and smaller. Instead of the large drums in which patients must be placed for CT and MR, ultrasound devices are hand-held, making them especially useful for producing interoperable images that may support diagnosing and navigating work in real time.

Further, using minimally invasive operation methods, live video recordings through dedicated probes make it possible to “see” and operate inside the patient merely through a few tiny holes in the patient’s body. Other methods for image support include the use of X-rays in endovascular surgery.

Image-guided, minimally invasive and robotic interventions are changing the nature of surgical work—from being associated with craftsmanship where what one could do with one’s hands and tools was limited by what one could see with the eyes, it is gradually turning into a profession where highly sophisticated apparatuses make “visible” phenomena that beg for more sophisticated practices of seeing and interpreting and more sophisticated practices and tools for intervention. Robotic surgery illustrates this: working through robots that regulate the surgeon’s movements gives new opportunities for working on particularly fine structures that are visible or accessible by virtue of imaging technologies and minimally invasive methods. In this paper, we will sometimes refer to specific techniques and sometimes be more general, but it is the sum of the tools, techniques and practices mentioned here that we refer to as augmented reality (AR) in this paper, adopting Azuma’s relatively inclusive definition of AR as “…any system that has the following three characteristics: 1. Combines real and virtual 2. Is interactive in real time 3. Is registered in three dimensions” (Azuma 1997: 356).

There is a considerable body of literature on the field of collaboration and image-guided surgery in the fields of human–computer interaction, computer-supported cooperative work and science and technology studies, much of which is tangent to and resonate well with the perspectives of this article.

Koschmann et al. (2007) approached surgical practices through conversational analyses, studying the role of gestures in producing representations an anatomic regions in an anatomy lesson during a surgery. In a similarly microsociological spirit, Svensson et al. (2007) studied interaction in the operating theatre focusing on the skilled and timely use of mundane objects and artefacts.

Lammer (2002) explored the notion of invasiveness and the shifting interpretations of and boundaries between invasive, minimally invasive and non-invasive surgery when surgical and digital imaging proceedings melt into one another.

Professional vision and the practice of seeing are a recurrent theme in the literature of image-guided interventions. New touchless imaging technologies introduced to the operating theatre introduce new ways of seeing. Mentis and Taylor (2013) and Mentis et al. (2014) studied the constructed and embodied practices of seeing—practices that are not necessarily intuitive in the same way as “ordinary” vision—and the consequences for surgical training. With a greater emphasis on the touchless feature of these imaging systems, O’Hara et al. (2013) and O’Hara et al. (2014) investigate how the visual resources are embedded and made meaningful in the collaborative practices of surgery. Not only do these imaging systems enable clinicians to see inside the body and make otherwise non-visible phenomena visible, but they are also “inherently constitutive of the social practices of surgery itself” (O’Hara et al. 2014, pp. 299–300). A similarly close relationship between practice and the objects for practice is noted by (Koschmann et al. 2011, p. 2): “Understandings of objects are talked and worked into being within concerted action (…). Procedure both determines and is determined by its objects”. We get further support for such an understanding from Alac (2008) and his study of the work with brain scans. Brain scans, Alac says, become visible for the practitioners “not only through visual perception, but also through the involvement of hands” (Alac 2008, p. 484). To understand the work with digital images, one must understand the situated action involved in making sense of them.

In parallel with the development in image-guided interventions, telemedicine is an area of increased interest in health research and practice. There exist different definitions of telemedicine (Fitzpatrick and Ellingsen 2013), of which none are canonical. In this article, telemedicine refers to systems and practices that support diagnostic and therapeutic work independently of where the patient and the health workers are located geographically. Although telemedicine has revolved mostly around information and communication systems including information infrastructure and video conferencing, as well as telecardiology (Nicolini 2007), the above-described developments within traditional, co-located surgery have some particularly interesting features when viewed in the context of telemedicine. Common for many of the developments in co-located, modern surgery is their “digital potential”; the images produced by representational technologies may be transferred in real time to other locations for various purposes, and the robotic technologies that allow working on the patient through cybernetically managed mediators challenge the need for the surgeon and the patient to be co-located. Aanestad (2003) elaborates nicely on how the introduction of cameras mobilises information and makes possible reconfigurations of people, practices and artefacts.

AR is thus gradually being brought into the field of surgery as a potentially powerful resource for safer and more lenient treatment of patients. The combining of real objects (patients and tools) with virtual objects (such as 3D images of the patient and internal organs, and simulated 3D virtual tools operated through robots and cybernetic technologies) enables new forms of interaction and co-location across geographical distances, thus transforming surgery from craftsmanship to sensework. The particular powerful synergy between robotics (precision) and telesurgery (location independence) makes telesurgery a field of growing interest and application, particularly in minimally invasive surgery (see, e.g. Bann et al. 2003; Healey and Benn 2009; Ruurda et al. 2005). Allowing the surgeons to operate remotely (different room or even different country) from both the patient and operating room team, working with a 3D representation of the surgical site, may reduce the degree of trauma involved with patient transportation and increase highly demanded specialised surgeons’ radius of action—in the sense that they may be available to operating theatres at different locations without being restricted by distances and travelling times—to mention a few advantages. And indeed, the use of robotic surgery has increased rapidly and continues to do so: for example, the worldwide installed base of the da Vinci surgical system—widely used for surgical removal of cancerous prostate and uterine cancers among other—has increased from less than 300 in 2004 to 1000 in 2008 and 2300 in 2011 (Abrishami et al. 2014).

The term sensework has previously (Haavik 2014) been introduced as a label for a type of sociotechnical work in safety-critical operations where groups of professionals try to put together pieces of digital sensor data and different sorts of representations to create a coherent picture that gives meaning to familiar and unfamiliar situations. In the case of surgery, one may say that the work is gradually changing from craftsmanship to sensework. Although the craft aspect is still highly pronounced—as we shall also see later when the cases are presented—sensors, representations and visualisations, and interpretation work are becoming increasingly dominating aspects of surgery.

The relationship between technologies—particularly digital technologies—and the organisation of work is intimate. Standardisation of work processes, division of labour, interpretation work and decision-making are all processes that depend on technological resources. And as Healey and Benn (2009) demonstrate, it is necessary to look beyond the surgeon’s interface with the operative site and to more holistically consider the operating room as a system.

In the petroleum industry, the operating philosophy of Integrated Operations (IO) (see, for example, Rosendahl and Hepsø 2013) has sought to develop technologies and organisation in concert to make better use of digital real-time data. In IO, decisions are seen as discrete events that may be managed separately from action (Droivoldsmo et al. 2007; Holst and Nystad 2007), thus seeking support from rationalised models of work and decision-making (March 1994) that dominated in the mid-twentieth century. When it comes to surgery, there is a sparse literature on intraoperative decision-making. However, references to naturalistic decision-making are more pronounced (see, for example, Cristancho et al. 2013; Flin et al. 2007), reflecting the more mature empirical research tradition in the field of surgery than in that of Integrated Operations in the petroleum industry.

The principle for a clear division of labour into discrete tasks of decision-making and action is a fundamental assumption, since AR tools and principles are being implemented in a fashion that aims at supporting work practices assumed to be in accordance with such a division of labour. This is a problematic assumption, and in this article empirical observations from the study of co-localised work in the operating theatre will be used to challenge this assumption. An alternative to decisions as the drivers for action will then be suggested.

Being an operating philosophy which takes stock of AR and uses more and better real-time data, models, and simulations to make collaboration across disciplines and geographical distances possible—which is believed to lead to better decisions and results—IO have spurred considerable interest in the health sector. Several projects have been initiated with the aim of transferring knowledge from the petroleum industry to the health sector (e.g. Fernandes et al. 2014a, b), and there is much optimism with respect to improvements of safety and efficiency of health services based on learnings from the petroleum industry.

The developments of imaging technology in medicine obviously offer great opportunities for telemedicine. At the same time, there is a risk of hubris; just as with the case of Integrated Operations in the petroleum industry (Haavik 2013, 2014), developments in information and communication technology do not necessarily warrant that all aspects of work practices that are essential for operational robustness in co-located settings will be acknowledged and offered a role also in geographically distributed settings. In this article, insights from studies of co-located practices will be used to address fundamental issues of surgical work in particular and of sensework in general. In this paper, it is argued that these are fundamental issues to understand and take into account in the further development of diagnostic and surgical work across geographical distances using telemedicine.

The studies mentioned above provide highly important insight into the microsociology of surgery, the notion and practice of the invasive, minimal invasive and non-invasive, professional vision and the practice of seeing, and touchless technologies. While the present study is tangent to and resonate well with these studies and their findings, its angle and scope represent an aspect of collaboration in AR that have not been subject to much attention in previous research: the relation between technology and organisation of work, or more precise, the relation between the use of digital representations, division of labour and decision-making.

This is reflected in the threefold objective of the article:

• 1. To explore what actors and mechanisms are at work in the landscape of AR-supported sensework in the operating theatre;

• 2. To investigate the relation between decision-making and action in co-located work in the operating theatre;

• 3. To use insights from (1) and (2) to speculate on matters of concern associated with the introduction of AR in future distributed operating theatres.

The article starts by briefly reviewing the sensework perspective and thereafter continues describing the central role of representation and visualisation in scientific practice in general and for sensework in the operating theatre in particular. After this follows an elaboration on the relationship between interpretation work, decision-making and action. Eventually, the consequences of these insights for the diagnostic and surgical telemedicine of the future will be pointed out, especially with respect to the structuration of work processes, decision-making processes and the division of labour.

2 Sensework

Sensework emerged from the need for a language and analytical approach in connection with research on different fields with similar characteristics (Haavik 2014). While many central phenomena in that empirical research domain have been explored and explained through well-established analytical approaches such as organisational sensemaking (Paul and Reddy 2010; Weick 1995), cognitive psychology and problem-solving (Hayes 1989), naturalistic decision-making (Zsambok and Klein 2014), document-oriented studies (Brown and Duguid 1996; Buckland 1997), common information spaces (CIS) (Bossen 2002; Fields 2005; Munkvold and Ellingsen 2007; Rolland et al. 2006) and common ground (Clark and Brennan 1991), there was a need for an approach and a vocabulary that in particular addressed multidisciplinary interpretative work in high-tech environments, where direct access to the phenomena of interest is restricted and the dependence on sensor data and model support is high. It is important to underscore, however, that sensework represents a continuity rather than a break with the many other perspectives and works that address similar environments under the labels of risky work (Owen et al. 2009), acting under uncertainty (Norros 2004), situated action (Suchman 2007), activity systems in a broad sense (Engeström and Middleton 1996) and digitalisation of work (Zuboff 1988) to mention a few. We may identify the same intentions in these approaches to understand how professionals with the aid of various tools work to make sense of and act on the uncertainty and fluidity of sociotechnical systems. However, the environments and systems are developing rapidly, and so are the tools, and sensework represents an effort to catch up with these movements. Development of theory, vocabulary and research framing is necessary to ensure relevance and correspondence with actual systems and practices. This is sought to be done from the shoulders of an established research community looking in the same direction.

And still, some aspects of sensework may actually challenge other perspectives whose intentions may be compatible with sensework, but whose epistemological and ontological assumptions are not.

As a type of work in safety-critical operations where groups of professionals try to put together pieces of information to create a coherent picture that gives meaning to familiar and unfamiliar circumstances, sensework obviously has connotations to sensemaking (Weick 1995). However, sensework differs from sensemaking in particular interest in the material and technological aspects of cooperative work. While Weick (1995) describes sensemaking as “a social process” (p. 39) in which the actors “often invoke imagery associated with symbolic interactionism” (Weick 1995, p. 41) and thus “to understand sensemaking is to pay more attention to sufficient cues for coordination such as generalised other, prototypes, stereotypes, and roles (…)” (Weick 1995, p. 42), sensework involves sensors, models, representations and visualisations that are produced by highly sophisticated apparatuses and that are indispensable for making things work even when not making sense. While Weick’s constructivism is a social constructivism, sensework assumes a sociomaterial or sociotechnical constructivism. Studies of sensework presuppose an acknowledgement of the role of materiality and technology for sensing and thus expand the meaning of cognition from a mental process to a sociotechnical practice [and demonstrating the relation to (Latour 1986; Hutchins 1995a)]. Shared understanding, including shared situation awareness, is another category that is challenged by the sociotechnical constructivism of sensework (see Haavik 2011). The examples of sensemaking and shared situation awareness are mentioned to underscore that in the rich variety of approaches to understand collaborative work in sociotechnical systems, some ideas are indeed overlapping but not all are—and being precise on these nuances are important. Only then do the differences of the perspectives—and of sensework—make a difference.

The present study follows several studies investigating teams of engineers in the land-based drilling organisation of a large Norwegian oil and gas company (Haavik 2013, 2014) and takes on several insights from these studies. From their onshore locations, these teams are responsible for the planning and real-time follow-up of projects aimed at drilling wells which reach down to hydrocarbon reservoirs thousands of metres below the seabed. This work setting is highly distributed in many terms: geographically—the organisation is distributed across offshore and onshore locations; organisationally—the engineers belong to different organisations from the operating company, drilling contractor and third-party service companies; and professionally—the participants bring their specific domain knowledge to the project, such as well construction technology, geology, drilling fluid, reservoir engineering, mud logging, directional drilling and many more.

Sensework refers to formalised and non-formalised work practices that may be traced along axes in different dimensions. This tracing reveals how teams of professionals combine theoretical expertise with operational experience, model-based interpretation with case-based reasoning, and logical problem-solving with pragmatic practices and workarounds in their efforts to carry forward safety-critical operations in high-tech environments. This unfolding of work may be projected onto a three-dimensional landscape, where it, during the course of operations, fluctuates along contingent paths, seldom lending itself to “clean” descriptions such as deploying exclusively theoretical expertise in purely model-based interpretation work. Rather, the practices are situated, diverse and hybrid, adapted to the content and context of the work.

A central insight in these particular studies is the striking asymmetry in the organisations’ acknowledgement of the respective realms of formalised and non-formalised work. While the petroleum engineers largely articulate compliance to governing documentation and prescriptive work processes as the main working practice, the situated, non-formalised practices and strategies acted out in the course of operations are not parts of their narratives. In contrast to formalised work processes, these situated practices are not acknowledged in the organisations’ formal understanding of themselves and their activities.

A central ambition in the petroleum industry is to develop a clearer divide between decision-making and execution, by cultivating more distinct decision-making communities and executing communities, respectively. Decision-making shall be reserved for the onshore organisation, while the offshore organisation shall be responsible solely for execution (Droivoldsmo et al. 2007; Holst and Nystad 2007). There are, however, indications that this division of labour may not always reflect the intertwinement of work as it unfolds in practice. When petroleum engineers onshore and offshore collectively carry the operations forward in challenging situations where multiple interpretations are possible and the “right” decision is not obvious, final decisions may be difficult to identify and separate from other categories of work, such as interpretations and action (Haavik 2014). In fact, analyses of such situations show a blurred border between the categories, if not questioning the justification of the borders.²

This article investigates the practical and theoretical conditions for such a division of labour in settings of sensework by analysing how interpretation and decision-making relate to action in co-localised settings. The comparison is motivated by the hypothesis that the essential conditions for interpretation and decision-making in co-localised settings keep their relevance—or that these essential conditions are what is actually sought to be re-established or imitated—in distributed settings. Hence, the success of new AR-supported arrangements hinges on the adequacy of this assumption.

Engagement with brain tumours that are difficult to reach, identify and remove represents to surgical teams challenges that are—although very different in many respects—comparable with those of petroleum engineers’ engagement with high-pressurised hydrocarbons, equipped with technologies that enhance, represent and visualise the phenomena of interest, and make it suitable for circulation in ways forcefully illustrated by (Latour 1986), and the surgical teams work their way through expected and unexpected terrains and continuously produce sensible, or at least practical and functional, solutions to ill-structured problems.

In this article, we continue investigating the realm of sensework, changing the empirical setting from onshore–offshore collaboration in connection with drilling for oil and gas, to neurosurgical operating theatres where brain tumours and disc herniations are made subjects to elective surgical procedures.

3 Representation in scientific practice

Before we proceed to explore the representational practices of surgical work, we will briefly review the status of representation in scientific practices more generally. The role of representations in scientific and technological practice has been thoroughly elaborated under the headings of science and technology studies (Latour 1986, 1999a; Lynch and Woolgar 1990) and distributed cognition (Hutchins 1995a, b). With the technological development that evidently and ever more profoundly influences sensework in an increasing number and types of domains, the relevance of this phenomenon has not declined (Coopmans et al. 2014).

Selection and mathematisation are powerful techniques in representational practices (Lynch 1988). Through the selection practices of filtering, uniforming, upgrading, defining and modelling, value is added to the “original” objects of interest. Through mathematisation practices, the subjects of interest are given geometric properties, transforming them into objects that may be aggregated and combined with other objects that have identical scale references. Representational practices thus turn—through a trade-off between losing and gaining from one representation to the next—an unruly world into immutable, combinable mobiles (Latour 1986).

The role of these representations—images, diagrams, models and the like—in scientific and technological practices has been portrayed by Latour (1999a) in his analysis of the work of establishing an understanding of the dynamics of the border between the savannah and the forest in Boa Vista in the Amazon. Being impossible to grasp by only looking at it “as it is”, a combination of photographs, soil samples and vegetation samples is transformed into two-dimensional representations and combined in ways that produce new knowledge about the savannah, the forest and the border.

One of the most important insights from studies of representation in scientific practice is that representations are not poorer versions of the “real objects”, but rather the opposite: “The more steps there are in between the objects and those who make judgments about them, the more robust those judgments will be” (Latour 2014, p. 347). Still, we should always keep in mind that this robustness is never free of politics, and ideally this should be accounted for in each case to ensure transparency of both objectives and methodology, as Dumit (2014) demonstrates with examples from the use of brain images in legal proceedings.

4 Method

During a 2-week study at the neurosurgical department in a Norwegian state hospital, three researchers observed sixteen operations, one operations meeting (arranged weekly) and one X-ray meeting (arranged daily).

The focus of the study was on how the surgical teams manage to carry out routine and non-routine operations under familiar and unfamiliar circumstances, and generally under conditions where the objects of interest are well-concealed behind layers of flesh, bone and blood. We searched for answers from a position in the immediate vicinity of the action; every morning two of us found our way to the chairs in the corner of the operating theatre, where we stayed until the last patient had been operated on. From this ringside seat, we had an excellent opportunity to observe what was going on in all parts of the room; the surgeons’ work on the patient, the sterile nurse and the scrub nurse bringing tools back and forth and keeping count of all the equipment, the anaesthetic nurse (and sometimes doctor) monitoring the patient visually and with the aid of sensors, digits and curves on several screens, the technicians and their tools and tasks, and more. Microscopes were used throughout most of the operations, and with cameras mounted on the microscopes and several screens hanging down from the ceiling we could from our position observe in great detail everything that was going on in the operating theatre.

The operating theatre was a new research field for all the researcher. Previous knowledge of the field was limited to review of research literature and videos in addition to some introductionary conversations and interviews with practitioners. Our strategy was to use our outsider status as a leverage and to apply perspectives and methodologies from our own field of expertise to tease out novel insights about work in the operating theatre.

The department is highly modern and represents the state of the art with respect to minimally invasive and image-guiding technologies. Besides the surgical procedures, the operating rooms are also “laboratories” serving the development, testing and clinical implementation of new technology and new treatment modalities (see Fig. 1).

Fig. 1

One of the three operating theatres in the study (not from any of the operations described in this article). (Photo: Frode Nikolaisen, St. Olavs Hospital)

Our observations were documented as written notes. We put great effort into noting everything as thoroughly as possible, and the breaks in between operations were used to work on the notes and discuss our observations with participants from the team we had observed, or from other teams, in the lunch room. There has also been clinician input to the study after the first draft of the paper was written to ensure the correctness of case descriptions and wording.

Audio-visual recording of the operation was considered, but evaluated as suboptimal. The reason for this was partly practical. The process of obtaining permission for audio-visual recordings is time-consuming, and the outcome is not obvious. In the ongoing research project, of which this study represents merely one part, such a process was considered as a risk. Having said that, the researchers involved in the study come from a research tradition where audio-visual recordings are not so usual. We may categorise the present study as affiliated with research strands such as workplace studies and social studies of science, strands where some researchers use audio-visual recordings, other doesn’t. This has not only to do with personal methodical preferences, but with the levels of the action structures one seek to understand. When Heath and Luff (1992) and Svensson et al. (2007) make use of audio-visual recordings when studying collaborative practices and the use of technologies in the London Underground and in the operating theatre, they discover action structures at different and perhaps more demarcated levels than do Latour and Woolgar (1986) and Latour (1999a) when they study laboratory work and fieldwork in the wild without the use of audio-visual recordings. For the objective of the present study, investigating the actors and mechanisms at work in the heterogeneous landscape of AR-supported sensework, hand-written notes were found most appropriate. However, the choice of research methods sometimes involves compromises, and the limitations of the method adopted in the present study are acknowledged. Future studies using audio-visual recordings may supplement this work with valuable observations and understandings of technology in action.

Each operation typically lasted for 1–3 h. Usually, three operations were planned for every day, but rarely more than two were carried out. Either operations lasted longer than planned, emergencies arrived or planned operations were cancelled for other reasons. However, Thursdays were reserved for fast-track disc herniation operations, and these operations usually went as planned without substantial surprises.

The following operations and meetings were observed (Table 1).

Table 1 Operations and meetings observed in the study

The interest in technology and collaboration, and the realm of sensemaking and decision-making shaped both the study and the analysis. Within the framework of these themes, further analysis of the empirical material was done by going through the notes repeatedly and categorising different sections into subthemes. Among the themes that came up early during this process were “representations”, “representational artefacts”, “sterility”, “division of labour”, “visualisation”, “navigation”, “communication”, “ergonomics”, “articulation work”, “action-driven decision-making”, “configuration of tools”, “procedures”, “risk”, “collaboration”, “interpretation”, “mobilisation” and “craftsmanship”. Several rounds of categorisation and re-categorisation resulted in two themes around which the analysis was organised: the role of representations and the relation between interpretation, decisions and action (see Fig. 2). It should be noted that this process of distilling the categories into analytical themes involved close communication with existing, relevant research literature, and hence, to the degree that this study was inspired by Grounded theory it did not presuppose any sort of tabula rasa with respect to theory. Further, as we shall see in Sect. 6, the elaboration of these themes leads to a synthesis and new insights that could not be easily inferred from each of themes from the first round of categorisation; this is referred to as the Janus face of augmented reality.

Fig. 2

Development of analytical themes

Adherence to research ethical standard was ensured by a standard process administered by a department managing the research infrastructure of the hospital. This included the obtaining of permission from clinicians and patients, our commitment to rules for handling sensitive information and guidelines for acceptable conduct during the observations. We did not gather any sensitive data about the patients, and from our observational positions in the operating room we did not in any way interfere with the operating team, apart from when the team members themselves occasionally invited us to ask questions.

5 Sensework in the operating theatre

The distributed organisational configuration of drilling operations is contrasted by a tangible co-localisation in the operating theatre. In the department we studied, the standard operations crew consists of two surgeons performing the actual surgical intervention (one main surgeon and one assistant), two operating nurses (one sterile nurse assisting the surgeons and one non-sterile nurse with a wider, more coordinating and administrative array of tasks) and one anaesthetic nurse monitoring the condition of the patient during the operation. In addition, an anaesthetic doctor in an adjacent room serves all three operating rooms in the department. The doctor may be called upon by the anaesthetic nurse in case any support is needed or special procedures that require the doctor’s involvement are to take place.

Although the majority of these brain, neck and spinal surgical interventions are thoroughly planned in advance, the detailed course of action will to some extent produce its own phenomenology once the intervention starts. The intense atmosphere during such operations, lasting typically for 1–3 h, provides good opportunities to observe the characteristics of sensework unfolding in a type of co-localised setting that is sought to be re-established or imitated in more distributed settings.

Among the many peculiarities of surgical work, this paper emphasises two central observations that emerged as recurring themes through the study, and that may inform not only the understanding of surgical work as such, but also the future design and practices of distributed surgical work or telemedicine.

First, one striking observation was that rather than addressing the patients as such, as they were lying there right in front of them on the operating table, the doctors’ and nurses’ reflections usually referred to different forms of representation of the patient. These representations were produced through different kinds of interaction with the patient, and they were mediated and visualised by technological means. The different representations revealed different aspects of the objects of interest and thus invoked different resources for carrying the work forward.

Second, we also observed the shortfall of explaining surgical work as discrete sequences of interpretation, decision-making and implementation; decisions in many instances turn out to be merely analytical constructs that do not have referents in practical work. When they do have such referents, these often do not determine action, but rather result from action.

The empirical material from this study could be presented in a quantitative fashion, giving support to the emergence of these themes by referring to the occurrence of the instances throughout the material. Although such an approach could surely be valuable for illustrating the nuances of the themes and documenting the recurrence, it would not render justice to the context in which the themes unfolded. Hence, a more qualitative approach was selected, and in the following, two particular illustrative operations are reviewed to account for and illustrate these observations. We organise it so that the first case is presented particularly to illustrate the central role of representations, while the second illustrates the relationship between interpretations, decisions and action. In the following discussion, we show the importance of taking these observations seriously when aiming at using AR to support future distributed telemedicine practices.

5.1 A benign tumour: the role of representations

The following review of a neurosurgical operation demonstrates the central role taken by imaging tools and their representational artefacts in a modern operating theatre. The review is presented with this in mind and will therefore not go into detail on topics that do not shed light on that particular issue.

During a procedure to remove a benign tumour of the pituitary gland, the tumour is enacted by combining a series of representations, with the aid of a series of tools. Through these tools and the practices accompanying them, the ontological status of the tumour as an object is gradually increased, until it reaches a point—established along a scale of pragmatism—where matters of concern in practice turn to states of affairs. This process is presented in the following sections.

The intervention is done through the nose, with limited possibilities for direct visual inspection. In contrast to most of the other operations we observed, and in order for an optimal combination of the position of the patient and the technical configuration in the room, the patient is lying with his head towards the main door, on his back. After finishing positioning the patient and preparing the equipment and the procedure, an X-ray picture is taken. Shortly after, the surgeon (S1) walks over to a PC in the corner of the room, next to where we are sitting, and asks the ultrasound technician (UT1) to accompany him.

S1 informs UT1, constantly referring to the MR image: “He has been operated before, craniotomy. This tumour is a bit different, though”. They discuss the image, pointing and showing. “Here is the visual nerve. And here. This is the component we are going to remove. This [another component] we will leave and hope it will not grow”.

A small figure in the lower right corner of the MR screen proves to be of great help when using the MR image for navigating purposes; when rotating the MR image in three dimensions, a small stickman rotates automatically to unambiguously show the corresponding rotation and orientation of the patient. This feature was actively used and articulated when the surgeons used the MR image for orientation purposes.

UT1:

“We do a frontal operation?”

S1:

“Yes. The main problem for this patient is his vision. And for us it is difficult to separate the normal tissue from the tumour.”

They discuss briefly what kind of indications can be used for this work of separation. Then, they proceed to the issue of navigating to the right spot. S1 again: “We usually do it like this and like this (points to the screen) with pituitary glands…”

The ultrasound technician, who represents the developer and provider of the ultrasound apparatus, participates in this procedure by virtue of his expertise with the tool. Interpreting the kinds of interoperative 3D images produced by the tool requires knowledge of the tool itself and experience from its mode of operation in real usage.³

After the general introduction and performing the WHO safe surgery checklist, the surgeon starts the intervention. He enters through the nose, and X-ray images are taken frequently for the purpose of navigation. The ultrasound technician refers to the screen where the microscope image is projected: “What is that white thing?”

S1:

“I don’t know”

After a while, the surgeon says loudly to everybody in the room (who can see on two different screens what he sees in the microscope): “I wonder if that is the pituitary gland we see there…”. He walks over to the screen in the corner again, where he discusses with the technician: “Should we try and see if we can see anything on the ultrasound?” He starts walking back to the operation table, when he is called back by the technician, who points to the screen: “Be aware of those blood vessels… come here and see”.

The technician prepares the ultrasound probe, and from now on the surgeon and the technician work closely together. The technician manoeuvres the ultrasound apparatus. At this point in time, the surgeon states that everybody may take off their lead coats. Apart from relief since the coats are both heavy and warm to wear, this indicates a milestone in the procedure; the X-ray apparatus has been used for navigating purposes, and that the coats will no longer be necessary implies that they have arrived at the desired location.⁴

Now, the technician starts working with the ultrasound probe again. He and the surgeon discuss the images and try to sort out what is tumour and what is healthy tissue. At this point, a colleague of the technician enters the operating theatre and takes part in the discussion. More pictures are taken. More discussion arises. They can see the tumour, but they note that there is not much manoeuvring space to access it. They walk together back to the MR image displayed on the PC in the corner. The discussion at this point integrates three highly mediated representations that, together with the microscope images, amount to the final representational state of the patient lying on the operating table, a couple of metres away, seemingly on the periphery of the stage.

During the next few minutes, the surgeon demonstrates the craftsmanship of surgery, removing the tumour with basic tools (forceps and scalpels), followed by the production of some final ultrasound images.

At this point, the surgeon announces a timeout. Everybody directs their attention towards the surgeon, who summarises the operation, and thus marks the transition into a new phase of the operation, where the closure of the patient is the only remaining part.

A striking feature throughout the operation is the extent to which the surgeon and the technician have worked towards and on representations of the patient rather than on the patient himself. Now, as the surgical phase of the operation has been brought to an end at 1:55, 2 h after the operation started, the surgeon and the technician spend the last 15 min discussing how these representations can be prepared for the next journey and be even further liberated from the patient, in order to be meaningful to a broader knowledge community. As always in the world of science and practice, making public means publishing in scientific journals:

S1:

“That was fun, wasn’t it?”

UT1:

“Yes. How should we go about publishing this, do you think?”

In the following discussion, issues like how the operating methodology and the results could be framed, and what journals could be most relevant, are central topics. Meanwhile, the technician’s colleague captures and mobilises a selection of the last 2 h’ events by downloading the data from the ultrasound apparatus to a memory stick. The minimalistic features of the little stick, now in the pocket of the technician, illustrate well the increased mobility of the patient’s condition, the surgical intervention and the end result. There is an obvious potential associated with making use of this mobility in real time, across geographical locations, and to involve experts that are not physically present in the operating theatre. As we shall see in the next case, however, care needs to be taken when developing work processes for such collaborative practices.

5.2 A distended abdomen: interpretation, action and decision

This case is presented with the aim of showing the entanglement and indivisibility of interpretations, decisions and actions, and to demonstrate that such rationalistic and formalistic descriptions may be inadequate and may result in a work process design based on misguided assumptions.

The case presents the sensework involved in the course of the early phase of an operation addressing emergent, unexpected circumstances with respect to the patient’s condition. The planned intervention is aimed at removing a tumour in the fifth cervical vertebra. The uncertainty that is about to spread within the team and that guides the course of action in the very early stage of the procedure is already preceded by a more general uncertainty with respect to the operational strategy. The procedure involves the replacement of a vertebra with a piece of bone from the hip. The nurses tell us that they have little experience with this particular procedure. There is also a possibility that they will find a prosthesis made of cement more appropriate, but that will depend on conditions not yet revealed. In addition, the patient’s neck is unusually curved, which may force them to change his position during the operation. On top of that, it is early morning, nine o’clock, and the nurses have not received as much information as they would have wanted.

With these uncertainties as the backdrop to the impending operation, which is announced as possibly lasting anywhere between 4 and 6 h, the anaesthetic nurse notes that she has some problems guiding a tubing down into the patient’s abdomen. Someone asks whether they should use ultrasound to see if they can see anything abnormal.

At this early stage, the surgical team members are still undertaking their individual preparations, and although the note from the anaesthetic nurse is obviously taken note of, it does not seem to cause any significant re-ordering of the ongoing work. A few moments later, however, a second anaesthetic nurse guides a camera down the tubing and notes that everything looks fine down there.

At this stage, the anaesthetic doctor enters the room, obviously aware of the problem. Now it also occurs that the patient’s stomach is quite distended. “An ultrasound technician will soon come to examine him”, the anaesthetic doctor says. “He’s very tense”, he states, patting gently on the patient’s stomach.

A few minutes later the ultrasound technician arrives, bringing with him an ultrasound apparatus. “Does he have any known diseases?”, he asks aloud to the room. “Not that we are aware of”, one of the two operating nurses replies. The technician deploys the ultrasound apparatus and looks at the screen. He comments on some fluid retentions: “Hmm, what could it be? It should be possible to see if it was blood (…) Does he have ascites?”⁵ The sterile nurse walks over to the operation planner and checks the patient’s journal. At this point in time, the surgeon has entered the room and engages in the discussion. The ultrasound technician points to the screen again, addressing the surgeon: “This is the liver… this is the kidney… this is not the gallbladder. There is a possibility that there is some bleeding. Perhaps we should do a full ultrasound scan?

He continues for a while, he identifies a volume of air, and above that a volume of fluids; “Yes, I think this is ascites.” He repeats this four times as he talks through the image on the screen. “Here, you see, here’s the bladder, down here”. The five other team members are standing around him, watching him examining and interpreting. He mumbles that he is still not sure what the liquids are, and whether or not the bladder is empty.

“What we have to know”, says the surgeon, “is whether the inflation is ongoing, or if it is stable”, obviously feeling that they need either to get started with the operation soon, or to cancel it and get the patient thoroughly examined. “If it is ongoing, it can cause us trouble during the operation.”

Approximately 15 min has passed since the ultrasound technician started, and he is still doing these examinations when he suddenly bursts out: “Look here! This is a kidney. The lungs are OK, no ascites!”. The operating nurse asks whether they should replace the catheter with a larger one. No answer is given to that. “He is much better now,” the anaesthetic doctor notes.

At this point, the assistant surgeon enters the room. He strolls towards the patient and percolates lightly on his stomach with a couple of fingers as he passes him. At the same time, the ultrasound technician states: “There is no ongoing bleeding. There are liquids here”.

“But will it be unproblematic to ventilate him?” the assistant surgeon wonders.

Technician: “Yes.”

It is now 10 o’clock, and they are still not sure about the cause of the patient’s distended abdomen. However, by reformulating the problem they have eventually found a way to carry forward the originally planned procedure.

6 Discussion

Through the observations of surgical teams in action, and through interviews with team members, two essential themes emerged that will need to be carefully considered in connection with the design and the intended role of AR in surgical work. These themes concern: (1) the role of representations in surgical work and (2) the structure of work and the division of labour. The two themes will be elaborated on below, triggering a reflection on the non-deterministic nature of AR with respect to which version of reality is actually being augmented.

6.1 The sensation of representation

One thing that struck us as the study progressed was the relatively modest focus that the patients were subjected to. We arrived in the operating theatre with an unspoken assumption that the patients would be the centres around which everything else took place. Instead, we observed that it was the representations of the patients that were the practical points of gravity. First, representational artefacts, such as X-rays and MR, preceded the patients’ presence in the operating theatre. These images had already been integral to nominating the patients for operation in the first place, and they appeared on the screen in the operating theatre before the patients actually arrived. Second, the representational tools to a substantial extent organised the work and the division of labour. The actual progress of operations is hard to plan in advance due to the many uncertainties with respect to the patients’ condition and which operating strategy will be most adequate. However, from the preparation phase and throughout the operations, there are particular procedures associated with the deployment of the representational tools, and—with respect to practices and the division of labour—their deployment hence produces potential reference for future distributed work processes; third, through a range of parallel transformations,⁶ representations temporarily gained a stronger presence than the patients themselves, who were almost completely wrapped up in green paper and as such were only vaguely present throughout the operation. The surgical teams referred to the representations, pointed to the screens, re-presented the patient in many different modes using different representational tools and thus accomplished different versions of the patients that together entered into an augmented and highly workable reality.

Sensework in the operating theatre is thus highly dependent on the manipulations of a range of representational extracts of the patients. Without ordering these in any sort of hierarchy, representations that the surgical team produce and use include visual imagery,⁷ tactile impressions on fingers and light tools,⁸ microscope images, X-ray, CT, MR, and ultrasound, in addition to figures and graphs visualising the numerous measurements made by the anaesthetic nurses and their sensor devices.

What is so sensational about these representations? Are they not merely different—but interrelated—projected references to the object of interest? Yes, that is what they are, and this is exactly what makes them so powerful. A common feature of all these representations is that they are collected by sensory apparatuses. These sensory apparatuses may be human (visual inspections, tactile impressions), they may be non-human (as with the MR, X-ray and ultrasound), or both (as with the microscope). Representations are associated with bodily senses and artificial sensors as well as the way bodily senses and artificial sensor readings make sense. The sensors and senses are highly fine-tuned: the surgeons have steady hands and custom-made tools. Some of them even operate without shoes, introducing additional sensitivity to the operation of the foot paddle-driven hand drill; the surgeons go regularly to the optician to make sure their sight is corrected; the MR, X-ray and ultrasound apparatuses are designed to capture highly specific features of the patient’s body; the apparatuses are calibrated; the resolution of screens that present the images are higher than ever before.

Each of the representations is thus—ideally—produced with the highest possible sensitivity to emphasise particular features of the objects of interest. But more important than the quality of each representation is the difference between them:

The idea of science as a ‘mirror of the world’ is a spurious import of the history of figurative paintings into epistemology. In science, it is more as if the mirror is situated at the very end of long series of transformations between traces, none of which is an exact replica of the former. In other words, scientific imagery is never mimetic. If it were, there would be no gain of information between one step and the next. It is the difference between each step that allows the reference to move on. (Latour 2014, p. 348)

These different representations should not be seen as different perspectives on one fixed, objective state. Rather, they enter into the ontological work involved in the accomplishment, or enactment, of states (Woolgar and Lezaun 2013) that are intervene-able by the surgical team. Thus, instead of focusing on how the different representations may be merged into a congruent image that mirrors any true state of affairs, we should understand the representational work in the course of the operations as a continuation of the diagnostic work, and the production of a pragmatically desirable condition. In the end, the condition and the result of the intervention cannot be seen in isolation from the tools and techniques—and all the ontological work—that entered into this specific accomplishment.

6.2 The decisional nature of actions

According to one of the surgeons in the study, decisions associated with when to stop operating represent some of the most challenging situations in neurosurgery. Such kinds of decisions may be called for when operating a benign tumour that exerts pressure on the visual nerve and causes problems for the patient’s vision. The more of the tumour being removed, the better the result will be with respect to the patient’s vision. On the other hand, the closer the surgeon gets to removing the last parts of the tumour close to the nerve, the higher the risk of hurting the nerve and weakening the vision further is. In many decisions during operations, the connections and sometimes inextricably entwined associations between decisions and actions are remarkable; it is hard to imagine making these decisions without actually being deeply engaged with the actual operation, without seeing and feeling the tumour with the scalpel and without deploying the senses in an ongoing process of interpretation. The decision to stop, for example, may be difficult to locate in time, in words or in protocols. It may be more adequate to speak of the end of an operation as a consequence of the discontinuance of action, rather than an entified decision which dictates that the surgeon and his team stop operating.

Situations and issues that we usually think of as in need of a decision are often resolved without decisions as such. Instead of decision-making, such processes may perhaps more reasonably be referred to as sensework. Consider the case of the distended abdomen, where the surgical team faced the possibility of having to abort an operation before it had started due to an uncertain and possibly dangerous condition. The first symptom of an irregularity was that the anaesthetic nurse had problems guiding a tube down into the abdomen. Then, they notice that the abdomen is distended. This state can be due to (combinations of) many different conditions, of which some are serious and should lead to the abortion of the operation, and others are less serious, for which no special precautions need to be taken. What is characteristic for what follows, a process that is not well described by either decision-making or sensemaking, but could perhaps most adequately be referred to as sensework, is that the decision—the answer to the question ‘should we operate or abort?—is subject to two processes: deferral and displacement. The deferral is a way of buying time and giving the team an opportunity to collect more information that may shed new light on the situation: in the example, they eventually manage to guide the tube into a desirable position; they transport the problem into the “public” for discussion by producing video with a miniature camera connected to the probe; they do something similar with an ultrasound probe from the outside; they examine the written journal on the operation planner; they identify a kidney; and they identify what is not the gall bladder; 15 min later they rule out ascites, and they detect fluids, but rule out an ongoing haemorrhage; in one of the few approaches directly towards the patient, an assistant surgeon entering the room performs a percussion while passing by; a few minutes later they also observe that the abdomen has become less distended. In the course of 25–30 min, they have produced new, heterogeneous knowledge about the patient. But they have not established an unambiguous causal explanation of the condition. What materialises soon is a displacement: the ultrasound technician has just confirmed that there are fluids (which in itself is not a good condition for an operation), and the anaesthetic doctor has just confirmed that the abdomen is getting smaller, not bigger. At this moment, they still do not know what is causing the distended abdomen. It is a difficult situation. But it is a problem that is smoothly displaced—or, as Latour (1999b) would perhaps have termed it, shifted out—by a new perspective, a new question:

• The assistant surgeon: “But will it be unproblematic to ventilate him?”

• The ultrasound technician: “Yes.”

• And the operation starts.

This case illustrates how the decision to carry out or to cancel an operation is addressed through both deferrals and displacements—terms that we do not usually associate with decisions, at least not positively. In the landscape of sensework (Haavik 2014), this process is perhaps mostly associated with pragmatic practices and workarounds in the domain of the non-formalised and the ad hoc,⁹ although we can easily recognise how the process also produces traces along the entire landscape of sensework. How far this deferral and displacement—with its ad hoc division of labour and which is tailored in situ—are from schematic and standardised work processes based on a division of labour between decision and action! A better description of each new sequence of the operation trajectory is that it grows out of a situated process in which the need for a decision is transformed into a need for action and interpretation, or rendered superfluous by their discontinuance.

That decisions may be difficult to locate, and that they may actually be consequences of action rather than drivers for action, has been illustrated and discussed before with empirical reference to offshore petroleum operations (Haavik 2014). The same phenomenon is observed in the operating theatre. With their naturalistic decision-making approach, Cristancho et al. (2013) avoid the simplification implied by the rationalistic approaches. However, the insistence of keeping decisions as the unit of analysis and modelling the process through cycles and steps maintain the rationalistic flavour of the analysis. By orienting the study towards sensework, decisions are not disavowed, but as a point of departure they are given a less pronounced appearance and have to earn their status performatively. Weick (1995) reports similar observations from his studies of sensemaking, and he speaks of decisions not as guiding action, but as retrospective products of sensemaking and action. The nuances may be subtle and subject to non-committal judgment it may look like hair splitting, but when AR tools and practices are being introduced to support the work, this is a highly pertinent distinction. If AR is designed to support work processes that assume actions as guided by decisions and that dictate a division of labour with respect to decisions and execution, those AR tools and practices should certainly be different from AR configurations that are designed to support work practices as those observed in actual surgical work; work practices that tend to unfold in an organic manner and that do not easily lend themselves to division of labour between interpretation, decision-making and action.

6.3 The Janus face of augmented reality

“The more manipulations, the better”, Latour writes, and elaborates further:

The referential quality of a discipline, that is, its ability to reach objects inaccessible otherwise and to transport them into a site where they can be evaluated by peers is entirely dependent on the quality of those chains [cascades of transformations]. The more steps there are in between the objects and those who make judgments about them, the more robust those judgments will be. (Latour 2014, p. 347)

Augmented reality does exactly that, introduces new actors and new transformations into the realm of science and practice, and thus increases the resources and power to make those judgements. That is why the term augmented reality is so apt; the tools and practices of AR not only represent the world, they take part in and strengthen its creation. They do not do that, however, towards one particular state in accordance with some objective reality that is to be accomplished (see Fig. 3). On the other hand, anything does definitely not go—it is the harsh reality that not all operations end in favour of the patient. The uncertainty of what is to be can be seen in the eyes of each member of the surgical team when operations enter into critical phases. It can be recognised in the steady stream of words turning into silence. One of the fascinating aspects of surgery is the uncertainty that lies in the combination of the questions What is where?, What do we recognise it as?, and What can it be turned into? AR tools and practices take on an increasingly important role in answering all these questions.

Fig. 3

Working with representations (from Latour 2014)

The question that is asked here is: what role do they take? We have seen through the presentation of the procedures above that AR has already proven essential in neurosurgery and judged by the strong initiatives on image-guided interventions—such as dedicated centres for research and practice—in Norway; it will be even more so in the future, particularly as robotics and telesurgery becomes increasingly common.

Mol (2002) has demonstrated that there is more than one medical reality and that is obviously also true for medical projections. In telemedicine, augmented reality plays a central role in both enacting and augmenting medical realities. They do not do so autonomously, however, but in concert with the surgical teams. Those teams work in accordance with some agreed arrangements and procedures, or what we may call the politics of operations. How these politics of operations will look in future telemedicine is not obvious, but since telemedicine involves a geographical distribution of actors, they will have to address division of labour and decision-making one way or the other. In doing so, it may be necessary, as underscored by Healey and Benn (2009), to reconsider the role of every actor, and the role of the operating team, expanding on the conventional human–machine interface that tend to focus on the surgeon and his tools. The propagation of robotic proxies may represent a game change in surgery, and the rules of the game should be outlined with respect for the rich, interactive practices that will continue to be essential to decision-making also in telesurgery.

In the petroleum industry’s parallel to telemedicine—Integrated Operations—the politics devise a clear division of labour with respect to decision-making and execution as the basis of the operating regime (Droivoldsmo et al. 2007; Holst and Nystad 2007). Interpretation and decisions should be the responsibility of experts located onshore, while the offshore community should be executors of those decisions. Studies of Integrated Operations (e.g. Haavik 2014) have indicated some problematic aspects of this division of labour, and the present study of surgical practices supports and strengthens this scepticism.

To sum up, AR represents a promising area for future telemedicine, but one needs to consider carefully the arrangement of the processes it is going to support, in order to ensure that the reality to be augmented is the desired one. Tools and practices introduced to a process do not by their nature support the desired outcome. Sometimes, the conditions under which the tools and practices are introduced may actually lead to the opposite. Such is the Janus face of augmented reality. With slightly different connotations, but to all practical ends the same effect, Latour (1991) has labelled this phenomenon antiprogrammes and Turner (1978) has called it anti-tasks. If new AR tools, techniques and practices are to support future telemedicine, it is important that the politics of work, creating invariable framework conditions, are adequate. It can be questioned, and it should surely be investigated further, whether the division of labour with respect to interpretation, decisions and execution is adequate in that respect.

7 Conclusion

“Keep your coats on!” The request came from a surgeon at the beginning of one of the observed operations, right after the first X-ray image had been taken. Not only was it a highly practical piece of information, as taking these heavy coats on and off during the procedure is time-consuming and causes activities that challenge the sterile regime in the operating theatre. Metaphorically, the statement is also strongly laden with the following message: There is always yet a representation to be made.

In modern neurosurgical work, representations take on a central role in operations, in what can be described as an augmented reality setting and practice. A prerequisite for modifying the state of the patients’ condition is to transform them into different types of representational artefacts through the use of microscopy, X-ray, CT, MR, ultrasound and others. These representational artefacts not only afford interpretation and orientation, they are also what are most explicitly addressed in the operations, while the patients are wrapped up in paper and can almost be considered as media from which the representations are produced. In state-of-the-art operating theatres, all these representations are digital and thus highly mobile. As such, they are candidates for travelling wide and fast and for supporting operations where the operating teams are distributed geographically.

In rationalistic models of work, actions are accomplished as a response to preceding processes of interpretation and decision-making. This is a dominating view and an envisioned organising principle of IO in the petroleum industry, where the operating regime is characterised by multidisciplinary collaboration across geographical distances, enabled by shared real-time data, models and simulations and resting on a principle that decisions are to be made by onshore experts, while the offshore community should execute those decisions. The present study, however, illustrates problematic aspects of such a division of labour. While representations are what is being worked on and through, both in subsea drilling operations and in the operating theatre, these representations enter into a context where interpretation, decision-making and action are inseparable aspects of practice. This challenges the foundational, underlying idea of the division of labour in IO, and it puts forward the same challenges for any future telemedicine that intends to adopt similar models of work as a basis for designing work processes. This notwithstanding, there are many examples of successfully standardising—formalising and institutionalising—certain tasks or types of work. One example is the WHO safe surgery checklist, whose positive effect is largely unquestioned. In aviation formalised practices also play a crucial role for flight safety. The most striking feature of sensework is exactly this: it is never either formalised or non-formalised. Sensework tends to follow intricate pathways across landscapes of formalised and non-formalised practices (Fig. 4). To learn more about these pathways is an important task for future research. How do they vary within the same type of work? How do they vary between different types of work? Are they predictable, and can they be manipulated into better designed work processes?

Fig. 4

Landscape of sensework (Haavik 2014)

Representations are never apolitical. The politics of representations are their strength—they amplify some aspects and suppress others. However, the question of which aspects to amplify and which to suppress is not obvious. Therefore, any object and situation may be represented in many different ways, and instead of speaking of augmented reality, one could speak of augmented realities. Consequently, being informed about the politics and practice of representation in each particular case is crucial for any judgement of the reality that they claim to represent.

In the operating theatre, this informing is an integral part of the co-localised work. In future surgical telemedicine, where actors may be distributed geographically, there are therefore two points that must be considered very carefully. First, one should be aware that in practice, interpretation, decision and practice are inseparable aspects of work, and choosing to organise according to a division of labour with respect to these aspects may therefore be inappropriate. Second, if one still chooses such an organisation, AR tools and practices should be implemented in such a way that those who are making decisions are at all times well informed about the politics and production of the representational artefacts they act upon, and to all practical ends this implies that decision-makers must participate in the whole course of the operation and not merely be called upon to make decisions at certain points in time.

Sensework is not a ready-made framework for analysing sociotechnical work in a predefined manner. Rather, it represents an interim methodology for exploring types of work that become ever more common in many sectors, not only the health sector. Developments in sensor and representational technologies, as well as information infrastructures, make new techniques and work processes possible to an increasingly affordable price. These developments introduce changes that transcend the mere technical aspects of work; it may both enable and require new working and organising principles for individuals, teams and organisations. While much of the existing literature is well developed for many separate aspects such as visualisation and representations, professional vision, collective sensemaking and decision-making, the developments portrayed above beg for investigations that recombine and add to this literature in order to catch up theoretically with the essence of these new conditions for collaborative work. Sensework contributes to that by adding to and developing research agendas, analytical perspectives and vocabularies of approaches that in many respects share the agenda of the present article. Future research may build on this in different ways. One way is to focus on the empirical domain and investigate further the concrete findings in this article on representations and division of labour in the operating theatre, through the lenses of either sensework or other suitable frameworks. Another possibility would be to focus on the perspective and undertake new sensework studies on different empirical fields.