The Cost of Improved Overview: An Analysis of the Use of Electronic Whiteboards in Emergency Departments
Forming and maintaining an overview of an information space is key to competent action in many situations and often supported by overview displays. We investigate the cost of the improved overview associated with the introduction of electronic whiteboards in four emergency departments (EDs). In such a dynamic environment the work that goes into keeping the whiteboard current is, we contend, an indicator of the cost of maintaining an overview. On the basis of log data for the period 2012–2014 we find that the ED clinicians make an average of 1.91 whiteboard changes per minute to keep the whiteboard current. Performing these changes takes an estimated 6647 h a year in each ED. While the whiteboard is well-like and has improved the clinicians’ overview, our cost-of-overview estimation shows that it consumes substantial staff resources. This reflects the value the clinicians assign to having an overview but also reveals the amount of resources removed from other activities to maintain this overview.
KeywordsCost of use Overview formation Electronic whiteboard Healthcare
Professionals in engineering, healthcare, management, and other dynamic domains are confronted with large amounts of information in their daily work and must be able to navigate it competently. Relatedly, common non-work activities involve monitoring continuously evolving social media for updates of interest and searching vast information spaces for items about specified topics. The sheer amount of information has made it challenging for users to maintain the overview necessary to navigate the information and has made overview displays an important part of many systems [1, 2]. One indication of the importance of overview displays is Shneiderman’s  visual information-seeking mantra: “Overview first, zoom and filter, then details-on-demand”. Before users know where to zoom in and what to filter away, they first need an overview of the information space. However, an overview does not come for free. It is especially demanding to maintain an overview when the information space changes dynamically because the overview decays quickly and therefore has to be continuously updated to stay current. That is, while an initial overview may be necessary, it is not sufficient. This study investigates the amount of work involved in maintaining an overview of the dynamically evolving state of an emergency department (ED).
EDs are the common entry point to hospitals for most patients with acute problems. In the ED the patients are assessed (triaged) to determine the severity of their problem . Patients with high-severity problems are attended straight away to arrive at an initial diagnose and then transferred to the appropriate inpatient department for further treatment. Patients with low-severity problems may wait for hours before they are seen by a clinician. This patient flow means that most patients stay in the ED for a brief period only. But the patient volume is large, and every now and then it exceeds the available ED resources [5, 6]. Initial information about the patients’ condition will tend to be rudimentary and additional information must be acquired quickly, reliably, and often under time pressure. Any oversight about a patient’s condition may be detrimental to her health but this pressure toward detailed patient examinations must be balanced against the need to see a number of patients quickly. These work conditions create an environment in which it is important to have, and continuously maintain, an overview of the current state of the ED: Which patients are in more urgent need of attention? Which clinicians are responsible for which patients? What are the health problems of my current patients? What treatment should be administered to these patients? What is the status of this treatment? Are new patients about to arrive? And so forth.
It is important to note that the whiteboard has been successful in supporting the ED clinicians in forming and maintaining an overview of their work. In interviews conducted as part of our activities relating to the whiteboard a physician at ED3, for example, expressed that “It gives a great overview. I cannot imagine that we could do without it.” More formally, a survey at ED1 and ED2 showed that the clinicians experienced an improvement in their overview of their work when the electronic whiteboard replaced the former dry-erase whiteboards . In addition, before/after measurements at ED4 showed that the nurses’ mental workload decreased at the beginning of their shifts when they form an overview of the ED . That is, the work that goes into changing the whiteboard to keep it current has improved the clinicians’ overview of their work and is, we contend, an indicator of the cost of the improved overview.
2 The Notion of Overview
Hornbæk and Hertzum  emphasize the distinction between overviews (displays in a user interface) and overviewing (a user’s awareness of an aspect of an information space). While the whiteboard is an example of an overview display, it appears risky to assume that overviewing will ensue simply from attending to the whiteboard. Yet, Spence  and others take the term overview to imply that the user becomes aware of information in a rapid or even pre-attentive manner, that is without cognitive effort. This suggests that overviewing requires little more than periodically glancing at the whiteboard. Other researchers describe the process of acquiring an overview as more active, thereby suggesting that the user makes sense of the information space through active involvement with the overview display. For example, Bossen and Jensen [13, p. 257] tentatively define the process of achieving an overview as “how health care professionals arrive at a sufficiently informed, accountable and coherent understanding of a situation, so that they are capable of acting consciously and with confidence.” This definition states that to provide competent patient treatment the ED clinicians must form and maintain an overview based on which treatment decisions can be made. That is, overviewing is a central clinical activity. In addition, the definition suggests that users need to engage with the displayed information to make sense of it. Glancing at the whiteboard may be key to this engagement but it is not sufficient.
In order for a display to support its users in forming and maintaining an overview it must contain accurate information. While previous research on overviewing  – and related notions such as sensemaking  and situation awareness  – accentuates that in dynamic environments the users’ understanding of the situation must evolve to stay current, it speaks less about the work that goes into keeping overview displays and other external representations current. In their investigation of the cost structure of sensemaking, Russell et al.  found that the main cost was associated with finding and encoding the relevant information. This activity amounted to over 75% of the total time even though it appears that the information space (a set of documents) was known and static. We contend that in a dynamic environment the continuous work of finding the relevant information and encoding it on an overview display is, by far, the biggest cost that comes with using the display to maintain an overview.
The four EDs in Region Zealand introduced the same electronic whiteboard in December 2009 (ED1), January 2010 (ED2), January 2011 (ED3), and May 2011 (ED4). The EDs were part of medium-sized hospitals that collectively served a population of approximately 820000 citizens. Some of the whiteboard data, such as lab-test results, were automatically updated when new data became available, but most of the whiteboard data were entered and updated manually. This study is exclusively about the data that were entered and updated manually. Prior to conducting the study we obtained approval from the healthcare region.
3.1 The Log Data
All changes of the whiteboard content were automatically logged. For the purpose of this study the whiteboard vendor, Imatis, produced a version of the logs from which all patient names, clinician names, and other information that might identify persons had been removed. These anonymized log data covered the three-year period 2012–2014. However, we had to discard the periods January 2013–January 2014 (ED1) and November 2013–January 2014 (ED2–ED4) from the analysis because they contained long intervals of no data. After also removing 741 outliers (defined as ED visits longer than seven days, i.e. more than 50 times the median length of stay), the dataset comprised 380611 ED visits. The data for these visits consisted of more than 10 million log entries, each documenting a manually performed change of the whiteboard content. A log entry contained a timestamp, an event type, any values associated with the event, and a system-generated identifier of the visit to which the event pertained. For example, the event type ‘TRIAGEChanged’ along with the event value ‘5’ showed that a clinician had changed the patient’s triage level to 5 (indicating a life-threatening condition).
3.2 Data Analysis
We used the timestamp and the visit identifier of the log entries to determine the number of change events per minute and per patient. We also calculated the month-by-month evolution in the number of change events and their distribution onto day (08–16), evening (16–00), and night (00–08) shifts. These calculations served to assess whether the cost of the improved overview was evenly distributed across time.
To convert the number of change events into time spent we needed an estimate of the average duration of a change event. On the basis of an experiment with 18 clinicians from ED2, Rasmussen and Hertzum  found that it took the clinicians an average of 26.2 s to make a change on the whiteboard, including the time to log on to the whiteboard (by briefly holding a personal token onto a reader). Most whiteboard interactions consist of logons to make one or two changes. The clinicians in the experiment were experienced users of the whiteboard and they made fairly simple changes. To avoid overestimating the duration of a change event we set it to 20 s.
Before the introduction of the electronic whiteboard the EDs used dry-erase whiteboards. To provide an, admittedly rough, estimate of the added time involved in using the electronic rather than the dry-erase whiteboard we estimated that it took equally long to make a change on the electronic and dry-erase whiteboards and that the electronic whiteboard was changed 82% more often than the dry-erase whiteboard. Our basis for estimating that it took equally long to make changes on the two whiteboards was that, depending on the kind of change, Rasmussen and Hertzum  found either no difference in the time required to change the whiteboards or slightly longer times for the electronic whiteboard. Our basis for estimating that the electronic whiteboard was updated 82% more often than the dry-erase whiteboard was that Hertzum and Simonsen  arrived at this relationship between the number of changes on the two whiteboards in ED3. It should however be noted that the relationship was determined on the basis of few data, all of which from day shifts. That is, our estimate of the added time involved in using the electronic rather than the dry-erase whiteboard should be treated as merely suggestive.
As a preamble to the analysis we note that all four EDs saw 30–40 thousand patients a year. That is, they were about equally busy.
4.1 Number of Whiteboard Changes
Number of changes to the content of the whiteboard
Number of change events
ED visits (i.e., patients)
Changes per patient
Changes per minute
Increase in whiteboard changes divided onto work shifts
Changes per minute
Increase (Jan. 2012 to Dec. 2014)
4.2 Cost-of-Overview Estimation
Cost-of-overview estimation for 12 months of whiteboard use
Number of change events
Hours spent (estimated)
Added hours (estimated)
5 Concluding Discussion
The clinicians change the content of the whiteboard more than twice a minute during day and evening shifts and more than once every two minutes during night shifts. In the dynamic ED environment these changes are necessary to keep the whiteboard current. Importantly, the use of the whiteboard is merely recommended, not mandated, by hospital management; the ED clinicians spend time updating the whiteboard because they find that the improved overview justifies the time spent. We estimate that the time spent amounts to 6647 h a year in each of the EDs or the equivalent of about 1.7 extra full-time clinicians compared to the previous use of dry-erase whiteboards.
The cost-of-overview estimate shows the large value the clinicians assign to having an overview of the state of the ED. It is also evident that this overview is a collaborative accomplishment: Multiple clinicians must contribute updates to keep the whiteboard current and, in return, the whiteboard supports multiple clinicians in forming and maintaining an overview. This collaborative aspect of overview has previously been investigated by, for example, Bossen and Jensen , who note that novice and experienced clinicians form an overview in quite different ways but still contribute to the collaborative accomplishment of an overview. Relatedly, several studies show that the value of whiteboards is, in part, that they create a place where people meet face to face and information is exchanged [7, 18, 19]. Meeting at the whiteboard is a valued occasion for consulting experienced colleagues, for obtaining details not on the whiteboard as well as for glancing at the whiteboard. In addition, the cost-of-overview estimate shows the potential of automating the whiteboard updates by deriving them from data in the electronic patient record. Such automation requires high-quality data because bypassing manual data entry also means bypassing a clinical judgement of whether the data make sense. However, automatic updates appear to be the only way to reduce the cost of overview while at the same time increasing the number of whiteboard changes in order to improve the overview further.
Two limitations should be remembered in interpreting the results of this study. First, the dataset is from one work domain and one country. While the four EDs provide evidence that the results are not peculiar to one hospital, we acknowledge that the results may be specific to healthcare and to Denmark. Second, the conversion of the number of change events to hours spent is no better than the estimate of the duration of the events and that of the ratio of electronic to dry-erase whiteboard changes. The latter estimate, in particular, is based on few empirical data. Much more research is needed to quantify the cost of overview. This study proposes that in dynamic environments the number of changes needed to keep an overview display current is indicative of this cost and shows that it may be substantial.
This study is part of the Clinical Communication project, which is a research and development collaboration between Roskilde University, University of Copenhagen, Imatis, and Region Zealand. The interview quote at the end of Sect. 1 is from an interview conducted in collaboration with Jesper Simonsen from Roskilde University. Special thanks are due to Rasmus Rasmussen at Imatis for making the anonymized version of the log data.
- 2.Card, S.K., Mackinlay, J.D., Shneiderman, B. (eds.): Readings in Information Visualization: Using Vision to Think. Morgan Kaufmann, San Francisco (1999)Google Scholar
- 3.Shneiderman, B.: The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings of the 1996 IEEE Conference on Visual Languages, pp. 336–343. IEEE Press, Los Alamitos, CA (1996)Google Scholar
- 8.Rasmussen, R., Fleron, B., Hertzum, M., Simonsen, J.: Balancing tradition and transcendence in the implementation of emergency-department electronic whiteboards. In: Molka-Danielsen, J., Nicolaisen, H.W., Persson, J.S. (eds.) Selected Papers of the Information Systems Research Seminar in Scandinavia 2010, pp. 73–87. Tapir Academic Press, Trondheim (2010)Google Scholar
- 12.Spence, R.: Information Visualization: Design for Interaction, 2nd edn. Prentice Hall, Harlow (2007)Google Scholar
- 13.Bossen, C., Jensen, L.G.: How physicians ‘achieve overview’: a case-based study in a hospital ward. In: Proceedings of the CSCW2014 Conference on Computer Supported Cooperative Work & Social Computing, pp. 257–268. ACM Press, New York (2014)Google Scholar
- 16.Russell, D.M., Stefik, M.J., Pirolli, P., Card, S.K.: The cost structure of sensemaking. In: Proceedings of the INTERCHI1993 Conference on Human Factors in Computing Systems, pp. 269–276. ACM Press, New York (1993)Google Scholar
- 17.Rasmussen, R., Hertzum, M.: Consider the details: a study of the reading distance and revision time of electronic over dry-erase whiteboards. In: Proceedings of the Danish HCI Research Symposium, DHRS2012, pp. 24–27. University of Southern Denmark, Sønderborg, DK (2012)Google Scholar
- 18.Scupelli, P., Xiao, Y., Fussell, S.R., Kiesler, S., Gross, M.D.: Supporting coordination in surgical suites: physical aspects of common information spaces. In: Proceedings of the CHI 2010 Conference on Human Factors in Computing Systems, pp. 1777–1786. ACM Press, New York (2010)Google Scholar