Learning medicine through collaboration and action: collaborative, experiential, networked learning environments

Abstract

The SUMMIT Lab and William LeRoy Heinrichs, at Stanford University, were honored to be the 2002 awardees of the Satava Award for Virtual Reality in Medicine. Since the award, the group has followed two main threads of research, which we describe below. The first, “building a high-performance, network-aware, collaborative learning environment” has investigated the framework and components needed when students in multiple locations collaborate using computation-intensive simulations and large image datasets. The second thread, “online, interactive human physiology for medical education and training”, has focused on the application of interactive physiology models embedded in 3D visualizations of virtual patients in naturalistic medical environments. These environments support immersive, experiential learning where students act as medical providers and manage authentic medical events and crises. These research efforts, and our conclusions, are presented in the chapter below.

1 Building a high-performance, network aware, collaborative learning environment

1.1 Introduction

The HAVnet project (Haptic, Audio, Visual Network for Education), funded by the National Library of Medicine, supported our research and development of a framework for an advanced network infrastructure for health education and medical research. We have shown (Fig. 1) that such a network infrastructure requires a Middleware system that monitors and reports network conditions to network-aware Applications that can self-scale and self-optimize based on network “weather reports.” The core system and applications have been developed within the context of two medical testbeds, a clinical anatomy testbed and a clinical skills testbed. Each testbed focuses on applications that challenge networks in unique ways. Beginning with local testbeds, we have extended our testbeds to national and international scope, and have evaluated them for educational, technical and enterprise impact. In this section of the chapter, we provide insight into the new directions that will be required for networked access, through powerful interfaces, to complex anatomy and clinical data supporting medical research and education. A complete report is available at http://havnet.stanford.edu/resources/report.html with a summary report at http://havnet.stanford.edu/.

Fig. 1

Framework for a networked, collaborative learning environment

1.2 Infrastructure

Development of a networked, collaborative environment depends on the availability of a robust, high performance computing and communication infrastructure, such as the following:

1.2.1 Remote computation and storage

The applications and infrastructure developed in this project are based on an architecture in which large databases like the visible human data set, and the core applications that work with them, reside on remote servers. Learners, on client workstations, interact with the server applications to provide a collaborative working environment that can encompass multiple sites.

1.2.2 Multicast

The client applications permit multiple views of the same data set, and collaborative interactions led from any workstation. The leader, a role that is easily passed between students, can control the views displayed at all workstations and can position a pointer visible to all participants. The ability to do this depends on the Multicast protocol. This is a key infrastructure component that is not uniformly implemented across the global network. Experimental alternatives to multicast, such as tunneling, exist and are in use. A robust multicast capability, or a viable option, will be essential in developing large-scale collaborative learning environments.

1.2.3 Internet2

In all of our work we have assumed the availability of Internet2 level service. Shortly after commencing the project we upgraded our connectivity from 100 Mb/s to 1 Gb/s. This involved a solution to the “last mile problem,” upgrading the connectivity from the SUMMIT laboratory to the university’s gateway, and thence to the CENIC backbone.

1.2.4 Network traffic data

A networked, collaborative environment that adapts to the available connectivity, requires frequent monitoring of network traffic. Network traffic conditions change by time of day and day of week. They also change as the network architecture evolves within the institution, regionally, and on the national and international backbone. In Fall 2004, we monitored network traffic over many weeks with 100 MB/s connectivity, and again in 2006 with 1 GB/s connectivity (Fig. 2). Detailed traffic pattern graphs are available in the HAVnet Final Report. (See Appendix 1: Experiments to characterize networks based on RSV requirements http://havnet.stanford.edu/resources/report_appendices.html).

Fig. 2

Monitoring network traffic parameters in local, national and international testbeds. The figure shows the network layout, the average network traffic parameters of throughput, delay, packet loss and jitter, and the variation of one of these parameters, delay, over the course of a single day

1.3 Middleware

A middleware environment is needed for collaborative learning environments. We describe the suite of middleware we developed to support self-scaling and self-optimizing applications for the end-to-end performance experience of the learner. (See relevant reports in Appendix 2: Middleware. http://havnet.stanford.edu/resources/report_appendices.html).

1.3.1 InformationChannels and the registration server

The InformationChannels framework supports the formation of complex networks of communicating application components (clients and servers) with minimal direct learner intervention. The framework accomplishes this by focusing on the channels of information exchanged between application components.

Applications announce their ability to provide or consume channels of information by periodically sending channel announcements on a multicast address. A channel is identified by the provider’s name, the channel’s name, and the channel’s application type. A channel announcement specifies the time interval used for resending the announcement. The announcement contains the network parameters (IP address, port number etc.) required to establish the connection.

A registration server (possibly with multiple instances for redundancy) listens to its multicast address and maintains a list of all active channels. The registration server responds to queries about currently active channels. Application components can use this capability to discover the availability of remote services. An automatically-generated web based query page also allows learners to discover the existence of channels.

1.3.2 A collaboration framework

A collaborative application, such as the Remote Stereo Viewer (RSV), supports the formation of a group of learners through the following process (Fig. 3) (Dev et al. 2006). The RSV server, with the RSV server application and the RSV-formatted image sets, announces its ability to provide access to an anatomy image set. The Registration server receives this announcement. A learner, through the web interface, queries for active RSV channels (Anderson et al. 1994). The learner selects the link for one of the advertised image sets. The RSV client on the learner’s computer (Host A) is automatically launched and connected to the RSV server for the image set. The learner can now interact with the anatomy image set (Ellaway 2005). The learner indicates a desire to collaborate with other learners and the RSV client on Host A begins announcing an active collaboration channel (William et al. 2006). A second learner (on Host B) finds the collaboration channel through the web interface and selects the link. This automatically launches the RSV client on the second learner’s computer, which now connects to the image set server as well as to the multicast address used for communication with the first client. The actions of the first learner are now seen by the second learner. Continuing this one step further, either client can query the registration server and discover the existence of other services, such as, a label set server for the image set being viewed.

Fig. 3

Illustration of the process of discovering available applications and setting up collaborations. For details, see the section “A collaboration framework”

1.3.3 WeatherStations

The ability of our applications to adapt to network traffic conditions depends, in part, on the availability of data on current network conditions. Network performance is monitored by WeatherStations, developed as part of the HAVnet project, that make use of the InformationChannels collaboration framework. To monitor end-to-end performance across the national testbeds used in the HAVnet project we have operated weather stations at Stanford, CENIC—Sunnyvale, CA, WISCNet–Milwaukee, UW—La Crosse, the Michigan Center for Biological Information, and the University of Michigan Medical School.

Each weather station announces a channel indicating its ability to conduct various network measurements. By querying the registration server each weather station discovers the existence of peer stations and schedules tests. The tests consist of

• Ping loss and roundtrip time

• TCP throughput

• UDP maximum send rate with less than 1% loss

• Multicast group setup time and loss rate

Weather stations are periodically polled for the results, which are placed into the performance database as well as being used to update performance graphs.

1.3.4 MultiServ: multicast lease server

This server provides a mechanism for “leasing” multicast addresses from an available pool. This allows RSV to obtain a multicast address to use with collaborating peers and allows WeatherStations to obtain multicast addresses for use in conducting network tests. The server utilizes the InformationChannels facility to create the lease mechanism. When an application is given an address to use the application begins to announce an “MLease” channel where the name of the channel is the multicast address. As long as the channel continues to be reasserted by the application the multicast lease server marks the address as being unavailable.

1.3.5 LogServ: logging performance data

This server advertises a channel that applications can use to relay performance data, which this server then logs into the performance database. Applications can log their performance by communicating with the LogServ through a client side API. The client queues log messages, sends them as single UDP packets to the LogServ and continues to resend them until acknowledged by the LogServ. If the application quits before all log messages are acknowledged the remaining queue of messages is written to a local file. The next time the application is run it will attempt to resend these messages.

Both RSV and the WeatherStations query for the presence of a log server and, if found, use it to log all performance data.

1.3.6 COR and CORQ: correlating performance data

This server correlates the weather station measured metrics and application performance data. Since specific performance data are unique to each application the COR server creates correlation tables unique to each application. The general principle is to aggregate application performance information based upon the prevailing weather conditions at the time the application was running. As a secondary correlation, application data is also aggregated by the time of day and day of week. The correlation mechanism allows individual computers to be treated separately or for the performance data from a group of machines (e.g., a teaching lab) to be combined.

The correlation query server, CORQ, allows applications to ask for their aggregate performance experience for the weather conditions currently prevailing between a specific client and server locations. Clients can ask for the performance experience from a specific computer or for the combined experience of all machines within their local zone.

1.3.7 Scalable transport protocols

Each application will have unique ways in which it can adapt its behavior to changing network performance conditions as a result of design choices. RSV is designed to support unpredictable navigation through large image sets by transporting images on demand. This implies a transport protocol that attempts to send image packets at as high of a rate as possible while balancing this against the goal of zero-packet loss since lost packets incur a round-trip time penalty to recover.

1.3.8 Integrating middleware into a network-aware scalable application

The middleware components described are integrated into an adaptive system where the application’s features are selected based on network weather conditions (Fig. 4). For example, a learner selects an anatomy image resource, such as “Lung and Pleura” on the course’s iAnatomy web page. The click on the data link automatically launches the RSV client program on the learner’s computer, and links to the corresponding program on the server. The lung images begin displaying on the learner’s screen.

Fig. 4

Illustration of the process underlying network-aware scalability. a The user selects an image resource on the iAnatomy web page. b The Remote Stereo Viewer client is automatically launched on the user’s computer. c In the background, the RSV client queries the Log Server and the Performance Database to determine the viewing and transmission parameters for optimal viewing by the user

In the background, the application sends its network performance data to the LogServ program, and queries (CorQ) the performance database about any changes it needs to make to its transmission settings. The current network weather (bandwidth, delay, packet loss, jitter, multicast setup) is checked against the information stored in the LogServ database.

If CorQ suggests that current network weather requires changes to the application settings, the scalable application selects some of the many possibilities in a predetermined priority order. The possible changes we have tested include a change in image resolution, frame transmission rate, and in the rate of interleaving of prefetch data according to the application’s speed in decoding compressed image data.

Experiments conducted by us suggest that responsiveness of the application is one of the most important characteristics sought by the user. Other scalable features, such as image resolution, can be modified as long as they remain adequate for the learning task.

1.4 Applications

Applications were selected and customized for the HAVnet project to support collaborative and distance learning. Selected applications were enhanced significantly so as to support our research in creating network-aware, scalable applications. They are described below.

1.4.1 Remote stereo viewer

Description:

The RSV is a client/server application that provides immediate access to large sets of high-resolution stereo images. The image sets are organized as multi-dimensional grids. Each image set has at least one dimension, consisting of a series of images forming a 360° revolution of viewpoint around an anatomical subject. Consecutive images around this rotation axis are used to construct a stereo view of the object. Horizontal mouse motion moves the image in this dimension through the image set. Additional dimensions, mapped to vertical mouse motion, may show the structure at various levels of dissection or magnification. The application assumes that the learner will view only a “small” selection of the images and consequently does not warrant downloading the entire image set. RSV 2.0 uses the InformationChannels framework and includes the ability to form collaborative groups of learners. The content includes both hand and knee image sets, labeled for use in anatomical study (Dev et al. 2006). (See also Appendix 3: Remote Stereo Viewer at http://havnet.stanford.edu/resources/report_appendices.html).

Network characteristics:

Image sets are stored on the server as JPEG images and transported to the client on demand using a UDP layer protocol. The transport rate is configured to be equal to the JPEG decompression rate of the client, allowing the client to interleave receipt and decompression of data. Between Stanford and University of Wisconsin La Crosse, transport produces bursts of between 30 and 40 MB/s.

End-to-end performance support:

We further developed this application for use with our testbed networks. RSV 3.0 incorporates the collaboration and end-to-end performance support of the core infrastructure. It reports actual image transport performance information to the WeatherHistory mechanism. The client also queries current weather conditions and uses this information to optimize its behavior accordingly.

All clients in a collaboration group can have independent connections to the image server and submit separate image requests. RSV also incorporates the ability to multicast an image to all collaborating clients, significantly improving the application’s scalability.

Scalable transport in RSV:

Remote Stereo Viewer is designed to support unpredictable navigation through large image sets. Typically, images are sent in high bandwidth bursts where the burst duration is a fraction of the total round-trip time. Since this means that the transport link would be idle for significant periods of time, the client pre-fetches images in the neighborhood of the last request to utilize the available capacity of the transport link as long as this does not negatively impact transport efficiency. The client can also adjust the image resolution requested to better balance use of the transport link. The performance data logged by RSV consists of its transport success experience (packet loss percent, percent of images transported with zero loss) at different send rates.

1.4.2 Exploratory haptics trainer: SPRING

SPRING is a software platform for developing surgical simulation applications, initially developed by Dr Kevin Montgomery of the National Biocomputation Center in 1999. The set of networked, cooperating applications in SPRING offers an interactive 3D world of tissue and tool models. The models use standard data formats and support both rigid and deformable models with customized physical tissue parameters. We have enhanced the software of SPRING while making it more accessible to potential learners and developers. Our major activities have included updating the code base and development environments for use with modern software engineering tools, improved algorithms, and resolution of many software problems. The software is networked, cross-platform, and open source, with a web site, availability on SourceForge, and extensive documentation. SPRING has been presented at workshops and conferences as a development platform on Open Source Surgical Simulation. Partly as a result of these efforts, SPRING is now in use as a research tool and simulator development platform at multiple university sites around the world. We intend that improved availability and support will allow computer science groups and medical schools to design and develop simulation applications for medical education and training (see Appendix 3: Exploratory Haptics Trainer, at http://havnet.stanford.edu/resources/report_appendices.html).

A series of haptic experiments have been hosted on SPRING, including comparison between actual and perceived stiffness of virtual membranes, and perceptual experiments on the range of haptic feedback that people can detect. We have also studied the role of network parameters on haptically-enabled task-oriented activities performed by subjects with a variety of medical experience and training.

1.4.3 The Collab Room

The Collaboration Room opened in 2005 has provided the HAVnet team with a greatly improved ability to participate in and host video collaborative networked events and research (Fig. 5). It is a 400-sq.ft. experimental space to study the impact of new collaboration technologies and new teaching methods on medical education of the future. This space is used for planning and staging interactive high bandwidth educational events, virtual reality team training sessions, and pilot studies for simulator validation. The room is equipped with Access Grid and other high-resolution videoconferencing system, capable of connecting teams simultaneously from multiple locations worldwide. Rich media data streams, such as large projected stereo images from remote simulators are part of the learning experience. The infrastructure supports easy switching to other collaboration tools, such as DVTS, standard H.323 videoconferencing, desktop-sharing and videoconferencing from many vendors (Microsoft, Marratech, Skype, and others) or other web-based communication tools such as Adobe Connect. (See Appendix 3: Collab Room at http://havnet.stanford.edu/resources/report_appendices.html).

Fig. 5

The Collab Room is outfitted with simulation equipment, ceiling mounted computer projectors, and multiple video cameras, with the ability for flexible re-organization of the space for a variety of learning activities

Around the room are mobile carts with state-of-the-art surgical simulators where residents and fellows evaluate modules on clipping, dissecting, grasping and other basic surgical skills. Key to the room’s design, therefore, is flexibility so that the room can be set up and torn down for the variety of activities required for their educational research.

1.4.4 Remote tactile sensor

The dermatologist, conducting an examination of a remote patient, needs both high quality video and high quality haptics for diagnostic purposes. We have designed and constructed a system for remote diagnosis of dermatological disorders employing both visual examination and palpation. It employs a haptic master–slave robot system (Fig. 6). This sub-project turned out to be more challenging than was originally envisaged, and the full system is still being tested. However, we did make significant progress in many different, and relevant, areas (see Appendix 3: Remote Tactile Sensor, http://havnet.stanford.edu/resources/report_appendices.html).

Fig. 6

Connectivity diagram for the Remote Tactile Sensor

We developed and tested a tactile probe suitable for the detection of skin texture, and the evaluation of skin profile. The unit is compact and light in weight suitable for mounting on a slave robotic arm. The sensing is done by an array of piezoelectric sensors formed on a piece of PZT film. As part of the design of this sensor we did extensive work on mathematical modeling of tactile sensors interacting with compliant substrates, such as skin for the purpose of evaluating texture. The tactile probe has been tested on human subjects and has been successful at discriminating between the textures of skin on the back of the arm and on the front of the arm and hand. We developed separate means of sensing of skin profile: that is the palpation of lumps in order to evaluate their geometry, location and mechanical properties. This requires a combination of the tactile probe and the native haptics of the slave manipulator. We have successfully operated the tactile probe mounted on a Sensable Technologies Omni slave robot and programmed to maintain constant pressure on the skin, in order to sense the skin profile.

The network parameter of particular relevance to the Remote Tactile Sensor is latency. Closing a control loop around a delay of the order of that generated by speed of light and switching delays (10’s or 100’s of milliseconds) results in dynamic instability. Control algorithms, such as the wave variable algorithm, cause a softening of the haptic sensation. We are investigating calibration methods based on the average latency for each specific session. We are also investigating a model-driven approach to collecting haptic data where the learner then interacts with that model, rather than the actual patient.

1.5 Clinical anatomy testbed

We designed and implemented local, national, and global testbeds to link learners at distributed sites through Internet2, and provided access to remote media resources through richly interactive interfaces. We used these testbeds to demonstrate, investigate and evaluate technical and pedagogic requirements for collaborative learning among students at distributed sites. Lessons learnt in each use of a testbed were used to improve the configuration and process for subsequent testbed experiments.

1.5.1 Local testbed

The local testbed was set up at Stanford University, between the Anatomy division’s classrooms and our laboratory. Faculty in the laboratory guided students in the classrooms through a series of stereo views of anatomy. The actual teaching sessions were preceded by numerous test sessions that were needed to set up the technology.

The primary application used in the testbed was the RSV. Using RSV, students accessed stereo images of the dissected hand and the knee, to get a three-dimensional view of dissected anatomy. The viewpoint could be moved interactively, allowing rotation of the anatomy and dissection through multiple layers. RSV supports collaboration by allowing students to form groups to observe others’ interactions with the images.

The local testbed was critical in identifying the technical infrastructure essential for remote collaboration in learning. Setting up the collaborative group, simultaneous viewing of an image with its cursor, and transfer of leadership, all required use of a key feature of the network—Multicast. Our local testbed showed that, even within Stanford, the multicast protocol was not implemented uniformly over the network, leading to failure of multicast between some pairs of learners. A second requirement for collaborative learning was the use of video collaboration. The teacher found it essential to see the tacit body language cues that were indicative of failure to understand or follow the lesson. A third requirement was good audio, with no echo. This was achieved either through the use of headsets or through echo-canceling microphones. A fourth requirement was the configuration of identical equipment and versions of software at each site, in order to reduce the chances of system incompatibility for complex applications. A major outcome of the experiments with the local testbed was the specification of a new experimental learning space, the Collab Room, as a flexibly configurable space, suitable for testing many types of distributed learning (Table 1).

Table 1 Comparisons of technical results of two networked teaching events

1.5.2 National testbed

For our national testbed, we linked faculty at Stanford University with students at University of Michigan, Ann Arbor, using Internet2 at 1 GB/s. Both accessed image resources located at Wisconsin and at Stanford. The goal of this testbed was to determine the issues that would arise as we scaled up the testbed (Fig. 7).

Fig. 7

In the national testbed, students at the University of Michigan, Ann Arbor, and faculty at Stanford University, accessed anatomy images from University of Wisconsin, La Crosse

Creating a stable network infrastructure that would support our application, RSV, proved even more difficult in the national testbed than in the local testbed. Each site worked with its institutional network provider to obtain adequate bandwidth. However, obtaining multicast functionality required testing of every point-to-point connection, with the involvement of the regional networks, CENIC, WiscNet and MERIT, as well as the network engineers at each institution. Intervention of Internet2 network engineering leadership was required for multicast problems to be addressed. This heroic effort made it clear that the network infrastructure for collaboration is inadequate, and that, until these services are available at the network level, collaboration may have to be a function of each application.

1.5.3 International testbed

For our international testbed, we partnered with ORION, Ontario’s high performance network group, and the Northern Ontario School of Medicine (NOSM), a new medical school, with an inherently distributed student body. Classes are held simultaneously at their two campuses, Sudbury and Thunder Bay, 500 miles apart.

Faculty at NOSM accessed the Bassett collection of stereo images of dissection remotely, to prepare image sets that were uploaded to the RSV server at Stanford. The images were labeled in RSV and then used in interactive small group learning by both faculty and students. Since NOSM provides only prosected dissection specimens, and no actual dissection experience, the three-dimensional stereo images proved attractive to the students. NOSM continues to use this collection for teaching even though the testbed experiment is complete.

In the evaluation, all students agreed or strongly agreed that the use of the images in RSV was worthwhile. Eighty-eight percent agreed or strongly agreed that 3D virtual reality and the Bassett collection should be incorporated into their curriculum resources. The students had some excellent practical suggestions on how the technology and the material could be incorporated into their curriculum.

As an extension to our testbed teaching experiments, we undertook numerous demonstrations to institutions around the world. Demonstrations to most parts of the world were over Internet connections of 2 MB/s or less, and with significant delay. In these situations, slide presentations using videoconferencing applications were the most effective. In some demonstrations, Internet2 quality bandwidth was available. The remote site could access our resources, such as RSV images, interactively, though only in mono-viewing, not stereo. However, true multi-site collaboration was not available. Screen-sharing applications, such as vnc, supported collaboration but interaction was slow. Some sites had stereo-viewing capability, enhancing the viewing experience. Others, such as the KISTI and MEDRIC sites in Korea, had high-definition videoconferencing capability, thus providing an extremely strong sense of presence between our laboratory and theirs. However, only those sites, such as NOSM and a few others, who had installed the RSV client and had multicast capability, experienced the collaborative interaction that our application and content was able to provide.

1.6 Clinical skills testbed

The clinical skills testbed encompassed many areas, most including a component of haptics perception. (See Appendix 5: Clinical Skills Testbed, http://havnet.stanford.edu/resources/report_appendices.html).

1.6.1 Skills training with simulators

Procedure simulators support practice of surgical tasks, from camera control, to grasping and clipping, to conducting a procedure such as excision of an ectopic pregnancy. To understand the technical and pedagogic issues of training with simulators, we conducted training sessions with ob-gyn and other residents. Most simulators are still stand-alone, with no collaboration, say, between the learner and a remote teacher. We have experimented with networked surgical simulators, based on Australia’s CSIRO-developed software and on Stanford’s open source SPRING software. The potential for networked training with surgical simulators is significant but still unrealized.

1.6.2 Video-supported learning events

Video telecast of real-time surgery is a common distance training method. To create a greater sense of presence and immediacy, we experimented with high-resolution two-way video (DVTS) and stereo video from the laparoscope camera. A laparoscopic appendectomy was conducted in an operating room at Stanford University, and was observed by residents in Sydney Australia. The project was a technical success. Since then, we have implemented and participated in numerous DVTS-supported distributed learning events, as part of conference sessions (multiple sessions for the Asia-Pacific Advanced Network and for Internet2) and as infrastructure for courses that demand high quality video and images. The DVTS infrastructure is free, reliable, and has high quality transmission of video. It does need knowledgeable technical support, both for network issues and for audio-visual quality, as well as high bandwidth, 30 MB/s per one-way video stream.

1.6.3 Performance score in simulation tasks

In collaboration with the Society for Laparoendoscopic Surgeons, we developed a collaborative, formal study for acquiring benchmark data for criterion-based training of basic technical surgical skills with five widely-used training simulators. Seventeen laparoscopic surgeons spent three half-days learning to use the simulators. We collected data from which Performance Scores (on 4th-attempt data) could be rigorously calculated. Scores were a linear weighted sum of the results from metrics embedded in the simulator, and were of the form $b0 + b1 \cdot X1 + b2 \cdot X2 + \cdots + bk \cdot Xk.$ The coefficients, b0, b1, … bk, were adjusted for each task, using a linear regression process, to generate a performance score for each task that improved from a value of 50 on the first attempt and to an asymptote of 100. Fourth attempt data, using this score, was determined to be an accurate predictor of a subject’s performance. This statistical method will be a valuable tool for evaluating performance scores for subsequent studies of networked simulators (Fig. 8).

Fig. 8

Graphs of practice attempts by individual subjects (surgeons). Analysis indicates that, for most subjects, the performance in the fourth attempt is an accurate predictor of later performance on the simulator

1.6.4 Experiments in haptic perception

We performed a series of experiments to measure haptic perception using both real and simulated forces. We first created a durometer containing latex sheets of various thicknesses. Learners probed these using laparoscopic tools, but without visual feedback. Subjects were able to feel differences in elasticity and could correctly rank the sheets in order of stiffness.

Virtual membranes with haptic output were modeled using the SPRING platform, comparing objects with four different constant stiffnesses to an unknown that matched one of the other four. The task was to pick the closest match to the center object by probing and sensing force differences. Network latency was also introduced, adding a time delay from 0 to 100 ms between probing and the force feedback. This apparently simple task proved to be difficult for most subjects.

We tested several profiles of force feedback as a function of penetration depth. Although learners judged the profile that increases force as the square of the depth (quadratic) to be most realistic, subjects performed at the same level with both linear and quadratic force profiles.

In an apparently simpler task, comparing two virtual surfaces using a low-friction, finger held haptic interface, subjects usually picked the difference in the force levels reliably when one force was twice the magnitude of the other. A significant number of errors were made with smaller force differences.

When both membranes resisted with similar force, but a network delay greater than 50 ms was introduced in sensing force on one membrane, subjects usually interpreted the two forces as being different. This is compatible with our earlier studies that indicated 100 ms as the maximum loop delay before a position-following task became unstable. (See Appendix 5: Experiments in Haptic Perception, http://havnet.stanford.edu/resources/report_appendices.html).

1.7 Evaluation

(See Appendix 6: Evaluation, http://havnet.stanford.edu/resources/report_appendices.html).

1.7.1 Impact on technology

Infrastructure and middleware:

The work performed under this project in transport protocols, performance monitoring and end-to-end performance has had several impacts on the research community.

Within the Internet2 community, the network requirements of our applications influenced the development of an API design for a new transport protocol in the Bulk Transport working group of Internet2. This transport protocol will have better performance characteristics than TCP by attempting to differentiate congestive loss from non-congestive loss by monitoring changes in connection latency.

Our extensive use of multicast has demonstrated that monitoring multicast performance should become a standard part of network monitoring. The project’s use of multicast also resulted in multicast being made an operational priority within the Wiscnet network. The unique performance needs of the applications developed by this project have been used in planning and soliciting for a new state network backbone.

User response to network performance degradation:

We conducted a study of student perception of RSV images under degraded network conditions and under degraded image quality. The degradations included reduction of the available bandwidth for the transport of the images through the network links and two different latencies between server and client.

The analysis of the data provides some important insight into the impact of network performance measures on learner perception. For example, medical students rated with scores around 4 (considered very good) the scenario in which the RSV application worked with a network connection set at 5 MB/s and with an image set at 50% resolution. This is a striking finding considering that in the usual non-degraded scenario this application works with a bandwidth above 70 MB/s and 100% image resolution. This opens the opportunity to extend the use of RSV on IEEE 802.11 Wireless LAN environments, where bandwidth is scarce. We also investigated the impact of responsiveness versus image quality on the overall learner perception as we degraded network bandwidth and latency. We found that learners could tolerate and use images of slightly degraded quality (50% resolution) as long as the system was perceived as highly responsive, with little or no perceived latency as new images were selected.

Open source surgical simulation software:

SPRING: In December 2005, we released the SPRING package as a robust, publicly usable suite of software applications, with freely available development code, on SourceForge.net. We have developed and maintained a website for SPRING learners that describes surgical simulation, applications, and the use of SPRING for developing practical simulation. Documentation on the architecture and details of SPRING and its deployment are available, as well as links to the SourceForge project. We have hosted a 3-day workshop, and presented at conferences, to build awareness of this powerful software.

SPRING as an open source software has opened a technology platform to students, research groups, and developers around the world. In addition to raising awareness of surgical simulation, SPRING’s availability has lowered the barrier to incorporating 3D modeling, deformable surface physics, and haptic interaction into projects around the world. We have initiated and facilitated growth of a community of learners, developers, and researchers in open source surgical simulation through this project.

1.7.2 Impact on learning

The use of advanced technologies in medical and surgical education has a significant impact on learning. The impact is most easily recognized in terms of access. For example, in many schools and many parts of the world, dissection is no longer a part of the medical school curriculum, so the opportunity to do a “virtual” dissection becomes essential for these students. Richly interactive, distributed learning technologies, such as the RSV application, give learners access to complex datasets that use powerful learning resources, 7 days a week, from any location—on campus, across town, or from another city!

Surgical education, at the residency level, is being transformed by the introduction of simulators—both “stand alone” part-task trainers and networked simulations which give trainees the opportunity to work “one on one” with a master surgeon—manipulating virtual tissue and organs of the same simulated patient—even when trainee and surgeon are in different geographic locations. These new learning technologies also make it possible for surgeons to watch a live surgical demonstration in 3D stereo, as if they had a front row seat in the operating theatre, regardless of their actual location.

The addition of force feedback, or haptics, to surgical simulators makes it possible for trainees to “feel” what the instructor feels, receive “hand on hand” guidance and respond to individualized instruction from a master surgeon in a distributed learning environment.

Evaluation of these technologies shows considerable acceptance in spite of technical difficulties. After a surgical learning event (SimTech, 2004) between Stanford University and surgical residents at Canberra, Australia, 100% of the residents rated the learning value of the stereo images as acceptable or better, and 97% felt that the performance of the networked application, over Internet2 and its global equivalent, was acceptable or better. Medical students at Northern Ontario School of Medicine assessed the RSV application, the images and the labeling capability as useful or better. Many suggested that their use in small group learning and teaching between students would be very useful.

1.7.3 Impact on enterprise

Through the HAVnet project, SUMMIT has impacted the local enterprise, at Stanford University, as well as institutions and organizations globally.

Learning and Knowledge Center:

A newly funded building program, scheduled for opening in early 2010, will incorporate state-of-the-art learning technologies in classroom configurations that take advantage of networked facilities. SUMMIT’s leadership, through HAVnet and other projects, has contributed the vision for learning spaces that enable networked visualization of media-rich anatomy, of simulations with standardized and virtual patients, surgical simulation for individual skills development, and virtual worlds for team training.

Center for Immersive and Simulation-based Learning:

The strong focus on simulation at SUMMIT, the VA, Childrens Hospital, and in the surgery department, has led to the formation of the new Center for Immersive and Simulation-based Learning. The center’s mission is to improve patient safety, patient care, education, and research through innovations in immersive and simulation-based learning techniques and tools.

Institutions outside Stanford:

As part of the clinical anatomy testbed activities, The University of Michigan’s medical school has developed a new learning space for collaborative and visualization-based learning. Network performance has been a key requirement for this space, including the requirement for multicast. This room is now in routine use at Michigan.

Similarly, the Northern Ontario School of Medicine has pressed for 10 GB/s connectivity to both their campuses, with a requirement that this support simultaneous teaching at both facilities. The stereo images of anatomy will be the backbone of the visualization-based teaching component at NOSM.

Collaborative learning spaces:

The Collab Room at SUMMIT was based on an innovative collaborative learning facility at Stanford in Wallenberg Hall. We customized the concept to create a room suitable for medical teaching and learning. Since the space opened in 2005, SUMMIT has hosted about 100 groups each year, interested in the space and the applications it houses. Sites at Stanford and elsewhere have modified the design and implemented their own collaborative learning spaces.

1.8 Future directions

We see several directions for our future research. The first is the technical infrastructure and the user interfaces to support the increasingly complex data and algorithms that will be accessed and used by future healthcare workers. Collaborative learning, access to remote experts, and transparent access to computing, storage and collaboration resources will be essential to future research. Our past and current work provides an excellent foundation for future research in this direction.

New technologies that promise to be relevant to medical education continue to be developed. The consequence is a continually increasing demand on Internet performance. When we started the current project we had 100 MB/s internet connectivity. That was expanded to 1 GB/s for the purposes of this project. Several universities now have 10 GB/s connectivity, and for purposes of planning only a few years ahead we need to think in terms of 100 GB/s. The implications in terms of the ways in which we will run interactive applications, and the ways in which software and hardware infrastructure will be affected remain to be explored. There will certainly be issues like the multicast protocol of the second generation internet that has not been consistently implemented in the way that was originally envisaged, and consequently remains a major limitation on interactive networked applications.

New visualization systems, high-resolution display walls, and high-definition video will become essential. Wave-front displays are becoming commercially available and offer a very viable alternative to stereo video. There are issues to be explored as to how one interacts with a wave-front model, and what are the implications for display of the very complex data sets characteristic of anatomical and physiological medical models.

2 Online, interactive human physiology for medical education and training

2.1 Introduction

Anatomy and physiology are fundamental building blocks in the study of biology and the health sciences of nursing, medicine, and dentistry. Yesteryear’s educational methods of cadaveric dissections, and animal experiments in physiology laboratories, have, or are being replaced in many training institutions. These changes are prompted by factors of cost and maintenance, animal rights, and time allocation in the curriculum, in competition with expanding knowledge. One of many consequences of the changed emphasis in these basic and pre-clinical disciplines is the paucity of graduate students and teachers to provide instruction in these “gross” topics—structural biology and genomics are the current focus. Another is the loss of opportunities for clinical skills development of dissection, a defining experience in the professional development of medical professionals, and of monitoring pharmaco-therapeutic interventions in anesthetized animals. These trends, and how SUMMIT’s work during the past 6 years is contributing new options, learning technologies and online educational tools are the topics that follow.

2.2 Online models of human physiology

2.2.1 The online information smorgasbord

A “Google search” for human physiology (http://www.google.com/search?client=firefox-a&rls=org.mozilla%3Aen-US%3Aofficial&channel=s&hl=en&q=human+physiology&btnG=Google+Search) yields over 7 million websites where one can find books, animations, videos, secondary websites, etc. Some of these are offered by colleges and universities around the world, others by companies, and still others by individuals. Multiple levels of complexity are offered for different learners; biology for high schoolers’ and pre-professional study, nursing, and medical students. Some information is free, and other arrangements require registration, subscription, or online purchase. Advertisements for information available on CD/DVD formats abound.

2.2.2 Standardized patients

Simulation-based learning environments in medicine usually require some graphical personalization of one or more patients. The classic example of the simulated or Virtual Patient is the “Standardized Patient,” represented physically by a trained human actor who feigns a clinical condition that is probed and resolved by a trainee in an OSCE (Anderson et al. 1994; Ellaway 2005; http://www.virtualpatients.net/). The simulations of physiology in such simulations are “by-report” in the clinical scenario. This decade’s long practice in medical education is the model for the computer-based technologies that follow.

Virtual representations of patients are familiar in healthcare education, in medical records, and in pharmaceutical development domains. These representations range from anatomical structures to scripted verbal communication, static descriptive data, or real-time processes of biomedical reactions (William et al. 2006). An example of an interactive Cyberpatient (http://www.netmedicine.com/cyberpt/cyberptframe.htm) presents a “patient report” webpage prompting the user to select an appropriate action by clicking on one of several graphical button’s that execute the decision selected; a new screen advises if the action was either incorrect or correct; if correct, the user is informed of the correct result, reported on a new screen, perhaps with an ECG tracing for indicating the physiological consequence, or presenting a new query. Students interact with descriptions, video images, videos, and displays of radiological films that they select.

2.2.3 Virtual patients

In another type of virtual patient, functional models are continuously interactive. Users are able at any time to manipulate data to demonstrate structural features, cognitive or social attributes, or surgical manipulations (http://www.acssurgery.com/abstracts/acs/acs0104.htm), however few of those available have physiological systems.

Another learning technology is the computer-based or online “virtual patient” that provides static vignettes of selected clinical conditions represented by graphical interfaces, textual, and auditory feedback (Garfield et al. 1989). An early entry into advanced learning systems for interactive simulation of patient (ISP) cases was introduced to meet the key pedagogical goals wherein students activate and construct the cases. The innovative ISP-VR system features a video-based illness history-taking function using keyboard input, highly interactive physical examination procedures, extensive laboratory tests and detailed user feedback (Bergin and Fors 2003).

This type of online physiology application involves clinical scenario simulations in which the virtual patient is represented as a video clip. Users are able to query the virtual patients who respond audibly with pre-recorded answers based upon recognition of keywords. The user may be provided graphical evidence of a condition, or physiological sounds, e.g., breathe sounds. This style of virtual patient has an embedded database of laboratory tests that can be selected; the results retrieved are consistent with the diagnosis in individual clinical scenarios. These laboratory values are pre-selected and static, and not responsive to therapeutic interventions. Nevertheless, development of therapy plans is required for completion of the lesson. Such virtual patients are well suited for the learning of diagnostic reasoning and medical management (Bergin and Fors 2003).

2.2.4 Libraries of virtual patients

However two recently aggregated, large libraries of virtual patients are recently available, one (http://www.aamc.org/mededportal) from the MedEdPORTAL of the AAMC (American Association of Medical Colleges), and the other from the European-based Web-SP project (http://websp.lime.ki.se/about). For the former peer-reviewed virtual patient cases, of which over one-hundred have been submitted,

“… learners can virtually experience a patient interview, perform many aspects of the physical exam, and even make diagnostic and therapeutic decisions.”

In the latter, case simulations features include

“… interactive history taking, complete physical examination (inspection, auscultation, palpation, percussion, vitals, neurological exams etc.), complete lab section including chemical labs, X-ray, MRI, ultrasound, CT, physiology lab, pharmacology lab, pathology lab etc.), diagnosis and differentials, therapy, interactive session feedback, integrated references and online database sources. All functions are freely chosen by the user and are available at any time, except the detailed feedback, which only is displayed when the user has submitted a diagnosis and therapy proposal”.

Other medical institutions have also embarked on the path of developing virtual patients, including Harvard Medical School (http://research.bidmc.harvard.edu/VPTutorials/), New York University (http://www.tinkering.net/vp/), the University of Edinburgh (http://labyrinth.mvm.ed.ac.uk/), and Tufts University School of Medicine (http://tusk.tufts.edu/view/url/H1185C/471802/490012/).

Such efforts and collections indicate the need and acceptance of using virtual patients in early clinical education. However most groups indicate the labor and resource-intensive effort needed to develop media-rich programs. Several offer authoring systems to facilitate the creation of virtual patients.

2.2.5 Dynamic virtual patients

In contrast to the above type of cases, Entelos, Inc., a company with expertise in bio-physiology has modeled virtual patients with mathematical systems labeled PhysioLab disease systems operating intrinsically (http://www.entelos.com/virtualPatients.php). After modeling human health,

“… these researchers draw from a large volume of published data about normal physiology and disease data, ensuring that PhysioLab disease systems are of a high enough quality to maintain the stability and dynamics of a homeostatic system. The model has been validated by running simulations and comparing virtual experimental results to known experimental results on the molecular, cellular, tissue, and whole patient levels.” “In simple terms, a representative healthy human is “given” a disease, creating a virtual patient. Entelos can create many virtual patients that represent known and hypothesized causes for the disease and, using bio-simulation experiments, test therapies to understand a patient’s likely response to treatment. Virtual patients respond differently to treatment depending on their lifestyle and why they have the disease just as subpopulations of actual patients would in the clinic” (http://www.entelos.com/virtualPatients.php). These virtual patients have no graphical representations.

We have amplified another category of virtual patients that affords a dynamic patho-physiology in silico model operating intrinsically, simulating appropriate representations of a victim/avatar’s gross physiology reflected in the vital signs (Heinrichs et al. 2008a,2008b). These patients could be considered the “gross physiological” counterpart to the “metabolic physiological” virtual patients described by Entelos, Inc.

These virtual patients include responses to therapeutic interventions for virtual pathological conditions. Thus, a dynamic virtual patient’s system can be initiated by displaying the initial vital signs, which upon initiation in a clinical scenario at time zero, the models begin operating continuously, anticipating that a user will accurately diagnose and manage a patient’s clinical condition, preventing patient demise within time-constraints. If accurate assessment and timely interventions are introduced, the deteriorating patho-physiological condition is reversed, and the patient’s vital signs return to normal. In other words, the clinical condition deteriorates during the interval of diagnosing the problem or if appropriate interventions are not applied in a timely manner, and conversely, if appropriate and timely remedial measures are applied, the impending demise is averted. Such simulations provide immediate feedback for the application of knowledge of clinical management. As such, they extend the cognitive domain into the practice arena where immediate feedback is obtained, options are available, and performance can be measured. Such simulations are similar to those provided by physical patient simulators, serving for learning crisis management on a case-by-case basis. A few online simulations with different degrees of complexity focus on trauma patients in virtual Emergency Departments, as well as patients in other “critical care” areas of a virtual hospital (Heinrichs et al. 2008).

2.3 SUMMIT’s first virtual emergency department

Our initial Virtual ED (VED) developed with Atmosphere, an Adobe Systems, Inc (San Jose, CA, USA) beta software product, offered six virtual patients whose physiological model were written in Java script. This simulation platform was very simple in concept, intended primarily for demonstrating computer modeling, but we extended that purpose by introducing dynamic virtual patients for learning (Fig. 9). Our online virtual cases were modeled after clinical trauma scenarios developed for the patient simulator in the Clinical Simulation Center at the Karolinska Institutet by collaborators in Stockholm. Four physiological parameters, including heart rate, blood pressure, respiratory rate, and oxygen saturation were programmed into the Atmosphere system using Java script. Teams of students/interns selected avatars that represented their decisions and actions in the VED. Virtual patients responded immediately to team members’ clinical interventions; if they were timely and appropriate, virtual patients survived. Changes in patients’ vital signs were displayed on a monitor in the VED (Youngblood et al. 2008). Users interact with the VED I by selecting diagnostic and therapeutic action from a menu by clicking on their choice. (Fig. 10).

Fig. 9

A virtual ED scene with a patient on a gurney surrounded by six avatars hearing the ‘patient report’. The vital signs are reported in the monitor screen

Fig. 10

The menu bar by which users can select diagnostic and therapeutic actions that control the vital signs displayed. The choices are purposefully disordered to avoid guiding users

2.3.1 Impact on learning

Formative research among 31 senior medical students and interns recruited from Stanford and the University of California San Francisco, demonstrated in a pre-test and post-test design with four training cases, that cognitive learning was similar in online virtual patients as with those same cases programmed into a patient simulator. A significant difference between these two types of simulations is the lack of psychomotor interactions of the online users; using the mouse and keyboard for executing one’s actions produces slightly less immersion during practice sessions, compared with the physical patient simulators.

2.4 Beginning Anew with OLIVE—VED II

The very favorable results of this 2003–2004 project utilizing “bare-bones” physiology provided the academic impetus for SUMMIT’s continuing studies of online virtual cases. The practical impetus came after Forterra Systems, Inc., a video game company in San Mateo, near by Stanford University, identified SUMMIT’s work with virtual medical worlds, and connected with us, providing opportunity to extend our physiology models via a successful, TATRC-sponsored SBIR application (SBIR FY04.3 A04-182 Phases II & III Contract W81XWH-05-C-0040). The title of the SBIR was Medical Simulation Training for First Response to Chemical, Biological, Radiological, and Nuclear Events. The Forterra game platform, OnLine, Interactive, Virtual Environment (OLIVE), is a sophisticated system that provides the needed security needed now for addressing medical issues without interruption of onlookers, sometimes with disruptive behaviors. This type of anonymous individual is called a “griefer”. As simulations become more robust and applicable to deeper levels of medical simulation. OLIVE has been the platform preferred by the military, and an increasing number of corporations such as IBM, Cisco, and BP that are developing virtual worlds for communication and training.

In the VED II system (Fig. 11), each virtual victim has unique gender, age, severity of injury, and rate of hemorrhage modifiers that produce unique models that operate simultaneously. For example, the model specifies that all victims will begin having seizures if and when the systolic blood pressure reaches 200 mmHg. The role player activates an animation of an avatar convulsing, and continues that action until hearing a caregiver’s report to the team that the drug diazepam has been administered; the animation is then stopped by the role player, allowing the victim avatar to lie still, or to respond to other scripted actions (http://www.youtube.com/watch?v=RwQlHNlpVcE&NR=1).

Fig. 11

A virtual Emergency Department scene constructed by artists at Forterra Systems, Inc., based upon photographs of the Stanford University Medical Center’s Emergency Department. A virtual patient lies in the bed while a virtual caregiver reads the status report, views the patient’s appearance, and observes the vital signs from the monitor. The controls in the lower left allow the user to make queries about the patient’s clinical condition, introduce remedial actions, administer fluid resuscitation measures, and select appropriate medications

2.5 A new challenge: “Families” of virtual patients for MOS

From the earlier genre of MMORPG’s (Massively, Multiple, Online, Role-playing Games) originally designed as social environments with the capacity for hundreds or even thousands of players simultaneously, has emerged the generic acronym MOS, acronym for Multi-player Online Simulation. Among the many video games under this banner, a sub-set is labeled Serious Games as non-entertainment video game venues, and a smaller segment is dubbed Serious Virtual Worlds. In all of these virtual worlds, each avatar is the in-world fictional representative of a real person who plays an assigned team role, or in personal “play,” whatever role they choose. Play-acting is akin to dramatic expression, and each virtual victim requires a role-player to provide the accompanying emotive behaviors, audible sounds, and some actions. In this latest group of emerging learning environments, requiring the representations of many patient victims simultaneously, we offer a new example, a “family” of virtual patients that has shared a common experience, but with unique injuries for each. Table 2 presents examples for five such victims.

Table 2 Five patient profiles for in-hospital response (Dirty Bomb Scenario II)

For simulations of mass casualty events, as may occur after exposure either to bomb explosions, to nerve toxins or other catastrophes such as earthquakes, airplane or train crashes, etc., multiple virtual patients are required to provide authentic practice. In these emergency situations, clinical conditions deteriorate rapidly, necessitating efficient and accurate “life-saving” diagnosis and treatment in a virtual emergency department. To meet the need for training healthcare providers for management of such events, unthinkable before 9/11, we devised an effective method of developing patho-physiological models. We offer the method of organization that leads to programming in an expert-friendly, rule-based approach for creating virtual patient cases with temporal evolution of selected clinical conditions. We are now able to report on our first two models, asphyxial shock and traumatic, hemorrhagic, hypovolemic shock. These models have in formative research, provided realistic, clinically relevant responses to user interventions that are unanimously satisfactory to experienced clinicians.

2.5.1 Models for CBRNE training

The Forterra/SUMMIT collaboration developed two potential training approaches; a pre-hospital, and an in-hospital simulation. The former focused on training first responders to a mass casualty. The trainees would be firemen, police, and emergency medical technicians. We decided to select for further development the in-hospital simulation that would address the training needs of physicians, nurses, para-professionals, and hospital administrators. And we chose the chemical, sarin, a nerve toxin agent as the causal agent in one of the CBRNE mass disaster incidents. The other simulation was for a radioactive bomb blast.

The patho-physiological model for sarin toxicity was driven by the blood oxygenation deficit described in an “asphyxia” model cascade; decreased SaO₂ results in increased respiratory rates, falling blood pressure from hypoxia, with tachycardia as the consequence. This sarin model accommodated one null and six exposure level doses of a nerve toxin by two routes, inhalation and/or dermal. It incorporated response data of SaO₂, respiratory rate, blood pressure, and pulse rate. These clinical indicators (vital signs) responded appropriately to the different doses in a time-limited manner if treated with O₂, atropine, 2-PAM, and/or benzodiazepine. For realism and creating interesting scenarios, the “asphyxia” model also afforded modifications for co-morbidities such as asthma, obesity, heart disease, diabetes, and heavy cigarette smoking, as well as for age (adults) and gender (Table 3).

Table 3 Rules that comprise the asphyxial model

The SUMMIT trauma model was more complicated. It incorporated four summary scales used routinely by clinicians during trauma assessments; a severity scale, a score for different organs, a scale that includes the standard Glasgow Coma Score for head injuries, and the standard ACS hypovolemia stages due to hemorrhage. The physiological cascade is hypovolemia from hemorrhage, prompting hypotension, tachycardia, hypoxia, and tachypnea (Table 4).

Table 4 An example of the rule for the relationship between hemorrhage and vital signs states

2.5.2 HFSM-based modeling

Numerous methods are available to implement dynamic physiology models. Models have been developed based on detailed mathematical equations linking vital signs to detailed internal physiologic models. The metabolic models by Entelos (http://www.entelos.com/virtualPatients.php) are an example of the richness and complexity of such models. However, these complex models cannot be run in real-time for user interaction in a virtual world. We have chosen to implement fairly simple dynamic models, as described above (Sects. B.5, B.5.1). We use a computational method known as Hierarchical Finite State Machines where we represent a series of physiological states, and allow the patient to transition between these states based on the severity of their injuries and on the interventions taken by the caregivers.

One of the challenges of enabling multiple instances of virtual patients to operate simultaneously and in real-time in the simulation, is reducing the complexity of the simulations. We have successfully used the HFSM method to operate the patho-physiology models of two types, one for asphyxial shock, and the other, more complex, of hemorrhagic hypovolemic shock. For each type of shock, ten virtual patient victims are delivered into the VED, where they receive resuscitative care by three or four physicians and six or seven experienced RN’s. Unlike real-world Incident Command practice sessions in which patho-physiology cannot for ethical reasons be implemented in the role-players, the vital signs of injured victims in virtual worlds deteriorate continuously and at selected rates unless timely and appropriate remediation is introduced. Although virtual victims can be studied individually for practicing pre-surgical management of trauma victims, the affordance of multiple unique victims functioning simultaneously allows training in patient triage, critical care management, team communication, resource management, etc.

2.6 Results of formative research

Ten physicians with an average of 4 years of post-training experience, and 12 nurses with an average of 9.5 years of ED experience at Stanford University Medical Center, and San Mateo County Hospital provided the data that follows. An exit questionnaire constructed as a Likert Scale was the test instrument. The study had IRB approval, and the participants were compensated with stipends. The majority never plays videogames, and a few individuals play only occasionally. All individuals were given an orientation to the game interface, and opportunity to practice using it. Two-thirds had prior Code Triage training, and one-half had prior CBRNE training. One group practiced with both trauma and sarin scenarios, and the other group played the sarin scenarios twice.

Among these trainees, 68 percent felt that they were immersed in the virtual world much or all of the time. Only three individuals found it difficult to communicate with colleagues in-world. Prior to the training, only four trainees were confident about managing mass casualty incidents of these types, but after training, nineteen felt either “confident” or “very confident”: thirteen attributed this change to practicing in the VED. Not surprisingly, 21 of the trainees reported that the scenarios were useful for improving healthcare team skills training, the primary objective for creating them. But a surprising response was that eighteen trainees believed that the cases were also instructive in learning about clinical skills management of such incidents.

We interpret that training healthcare teams in virtual worlds with dynamic virtual patients is an effective method of training, at least for uncommonly occurring incidents. Comments of trainees indicated their expectation that persons with videogames experience would be even more likely to find this method of training to be attractive.

2.7 Developing virtual patients for various acute clinical scenarios

Virtual patients are designed empirically with unique histories (or back-stories, in gaming vocabulary), each with gender and age, different BMI’s, disease profiles, vital signs, and drug therapy lists (Dev et al. 2007a,2007b). Selecting among these patient “features” from a database of real-patients allows an instructor to design hundreds of virtual individuals by choosing among four to six variables, based upon learning goals. For example, virtual obese patients express population-based risk factors for diabetes mellitus and hypertension; add a genetic risk factor, and the probability increases significantly that the physiology model will randomly exhibit those complications. For external actions such as exposure to an environmental chemical, the immediate clinical condition reflects a “dose” and route of administration of a toxic agent, or proximity to a blast, all in relative, but arbitrary terms because of the absence of experimental data from humans. Changes in vital signs are approximations based upon clinical experience and “expert knowledge” except when experimental data are available to guide development.

2.7.1 Developing patho-physiological models for sub-acute clinical conditions

In work-in-progress, patho-physiological models are being developed for sub-acute conditions, such as sepsis, that evolve over days to weeks, for diseases such as SARS, pandemic flu, cholera and plague. For human pregnancy, that evolves over months, avatars can be assigned the physiological attributes of the different weeks of gestation. Adding the “weeks of gestation” variable to the physiology model allows for the creation of dozens of unique virtual pregnancy avatars with different physiques, basal metabolic indicators, and hematocrits. These attributes are associated with different plasma volumes, red-cell mass, blood volumes, and cardiac stroke volume and output values, all calculated from published databases. Pregnancies that terminate abnormally in the first trimester are subject to additional maternal complications such as hemorrhage that can be chronicled with the physiologic changes appropriate for early pregnancy. Those that proceed into the second and third trimesters require both maternal and fetal physiological systems. These physiologic models are in development.

3 Conclusion

Since the Satava award in 2002, our research group has developed two threads of research, the software framework and components needed when students in multiple locations collaborate using computation-intensive simulations and large image datasets, and online learning environments that support immersive experiential learning in multi-person online virtual worlds. These collaborative, immersive learning environments are able to support the learners of the future such as medical students or community workers coming together online from scattered rural locations, student groups learning medical protocols in authentic immersive settings, professionals practicing new procedures without leaving their site of work, and teams learning to rehearse and work together before disasters or catastrophes strike. Our experiments and evaluations show that many of these environments have significant technical and logistic improvements needed, and yet, even in this nascent state, they have been proven to be useful and effective learning environments.