Background

An Extranet is a private network that uses the Internet to securely share information or operations with selected partners; it is accessible from any web browser. The literature describes the use and potential benefits of local area Extranets in hospitals,[1] physician offices,[2, 3] and small managed health care organizations [4]. A current U.S. National Emergency Medical Extranet (NEME) is being constructed to support emergency medicine and public health [5]. Building on the Centre for Disease Control's (CDC) national communication infrastructure, the US has begun to develop their Extranet to manage local public health emergencies, and plans to expand it for communicable disease surveillance and control activities [6].

A systematic review of the use of information technology in the event of bioterrorism evaluated web-based communication systems that link public health officials with clinicians and the public [7]. Although such systems have not been tested in crisis situations, the review indicates that they can securely manage disease-reporting needs of local and state officials. The systems evaluated most were those communicating abnormal findings in electronic medical records between institutions and clinicians. Three communication systems had the capacity to support rapid reporting and dissemination of information related to naturally occurring and bioterrorist-related infectious disease, although they required the use of electronic medical records.

A provincial Severe Acute Respiratory Syndrome (SARS) emergency was declared in Ontario on March 26, 2003. Two days later, a SARS Steering Committee was convened in Hamilton, Ontario, linking Public Health and Community Services (PHCS) with other partners (Table 1). Hamilton is located about 70 kilometers west of Toronto, which was the centre of the outbreak. Information management was critical during the crisis. Communication from the Ontario Provincial Operations Centre proceeded through email and fax to hospitals, long-term care facilities, health units, and the Ontario Medical Association. Simultaneously, information came from the World Health Organization (WHO), Health Canada, other health units and health care partners. The multi-source nature of the information made it difficult to manage and synthesize the most current, accurate information in a timely fashion. In addition, information impacting health units and Emergency Medical Services (EMS) might have reached only hospital staff. Directives changed often, necessitating version control measures to ensure that everyone was working from the most recent documents. This situation was difficult to manage with emails as updates were often sent daily and document file sizes were too large to send. A decision was made to develop a single, limited-access web-based system to manage constantly changing local and provincial SARS control information.

Table 1 Respondents by Agency or Institution (n = 34)
Full size table

In response to demands from the steering committee for up-to-date information, the City of Hamilton's Public Health and Community Services (PHCS) and Information Technology (IT) Division developed and launched an Extranet on April 3, 2003. The Extranet was developed to support the SARS steering committee in disseminating to partners comprehensive information that was timely, current, accurate and relevant. The Extranet also supported the implementation of local strategies using a secure accessible and managed centralized access point. SARS steering committee members (N = 53) received a common ID and password to access the site. The site did not require users to connect through a Virtual Private Network (VPN), nor was there any validation of IP addresses to validate users. The Extranet, however, was viewed as a private network using the Internet to securely share information with partners. The security aspects made it an Extranet.

The secure site provided document distribution, version control of documents such as clinical guidelines and directives, and control of links to information for schools and workplaces. An email link was provided to access the health unit's web master. Because of the speed at which this site had to be developed, there was insufficient time for developing other interactive communication tools or tracking IP addresses. Ideally, participatory design principles, [8, 9] would have been used to design the site, but there was no time to actively involve end users or track the costs to set up and maintain the site. Set-up and maintenance of the site was estimated to require one programmer (100 hours @ $40/hr), one Information Architect and Business/Technical Liaison (95 hours @ $50/hr), and one communicable disease expert (10 hours @ $75/hr) over the twelve weeks the Extranet was in use (total = $9,500).

The objectives of this research were to examine a) the use and perceptions of a local SARS Extranet, and b) its potential to support future public health information dissemination and communication with the local community.

Methods

The local university ethics review board approved the study. Participants included all 53 SARS Steering Committee members. The authors designed and tested a survey containing 13 forced-choice and 8 open-ended questions, which focused on perceptions on the Extranet and on other SARS information and communication sources. Both web-based and paper-based surveys were used to ease completion and simplify data collection as well as to increase the response rate.

User testing was conducted with the online version of the survey to identify problems with use, ensure survey comprehension, calculate average completion time (10–15 minutes), and identify if dynamic or technical functionality posed challenges. A three-member observation team tested the web survey with four PHCS employees who were familiar with the SARS Extranet and comfortable using the web. Instructions were given to each participant based on a think-aloud protocol, [10–14]. Participants were asked to complete the web survey, while observers took notes. Participants were instructed to think out loud, make statements, or ask questions as they made their way through the survey. They were informed that observers would not respond, but that observers would benefit from hearing their comments and watching their actions. Observers noted problems spoken and/or observed, which were later discussed in a debriefing session after each test. They participated in a final debriefing session to identify problems that occurred with more than one participant; this process provided direction for survey improvements. Several minor survey modifications were made prior to the launch. The online survey form was dynamic and, depending on how a respondent replied to one question, determined the next series of questions. Thus, changes identified through the think-aloud technique related to branching of questions (If yes, go here; If no, go there). No other significant issues were identified. The paper-based survey was also modified slightly to match the web version.

On July 3rd 2003, all steering committee members were sent an information email along with a letter of introduction, an invitation to participate, and a direct link to the online survey. One week later, a reminder email was sent to non-respondents; two weeks later, the paper-based survey and self-addressed stamped envelope was mailed. Quantitative data were entered into SPSS for analysis. Two researchers independently coded the open-ended questions. Consensus was reached with the entire research team whenever differences in coding existed.

Results

Of the 53 SARS Steering Committee members, four were ineligible as follows: one person moved, one was a research team member, and two were not active on the committee. Of the 49 eligible participants, three refused to complete the survey, three of them were disqualified because their responses were inconsistent, and nine of them did not respond. Respondents were representative of the steering committee, which included members from various agencies (Table 1). The response rate was 69.4% (n = 34).

Hamilton SARS Extranet compared with other SARS communication strategies

Thirty-four respondents rated the usefulness (1-lowest to 5-highest) of communication strategies used during the outbreak (Table 2). Email, teleconferences, and face-to-face meetings were rated the highest. Among SARS web-based resources, the Ontario Ministry of Health and Long-Term Care (MOHLTC) (mean = 4.1), the Center for Disease Control (CDC) (mean = 4.0), and the World Health Organization (WHO) (mean = 4.0) sites were the most valued. The Hamilton SARS Extranet site was rated fifth among the web resources (mean = 3.4).

Table 2 Usefulness of Various Communication Strategies during the SARS outbreak to respondents (N = 34) (1= not at all useful to 5 = extremely useful)
Full size table

Use and perceived utility of the Hamilton SARS extranet

The site had 3478 hits, and 30 respondents (88.2%) visited the Extranet (Table 3). The home page received the average number of 37.4 hits per day, while clinical guidelines and directives (10.3) and SARS Steering Committee minutes (7.3) received the next-highest average number of hits per day. Twenty-eight of them rated the quality of the Extranet and reported frequency of use. Even though the Extranet was rated lower than some other SARS web sites, when participants were asked specifically about the Hamilton Extranet, they rated it high overall (Mean = 4.0; SD = 0.8) (1 – lowest to 5 – highest). Criteria included relevance (Mean = 4.4; SD = 0.7), comprehensiveness (Mean = 4.0; SD = 0.8) accuracy (Mean = 4.3; SD = 0.6), and timeliness (Mean = 3.9; SD = 0.9). Most respondents (63.3%; n = 19) who used the Extranet visited the site at least once every few days. There was a difference in visiting habit between those who rated the Extranet most favourably (mean value ≥ 4 out of five points) and others. Those rating it most favourably visited the site more often (not statistically significant using two-sided Fisher's exact test, P > 0.50) than the others.

Table 3 Analysis of Hits to Pages on Site
Full size table

Thirty participants were asked a forced-choice question about whether they visited the site and why they visited. Most commonly, users visited to obtain SARS Steering Committee minutes (63.3%), access "Hamilton medical advisories" (53.3%), and obtain the SARS Steering Committee contact list (53.3%) (Table 4). These results corroborate the number of actual hits per page, Table 3. Other reasons for visiting included getting access to provincial directives and links to other websites. Although the difference was only marginally significant (P = 0.063), those who rated the website most favorably cited more reasons for using it than did others.

Table 4 Extranet Resources Used by Number of Extranet Users (N = 30)
Full size table

Eighteen participants revealed that they shared information from the Extranet with others outside of the committee. The information they shared most often included provincial directives and local information such as Hamilton medical advisories and Steering Committee minutes (Table 5). The average number of hits per page corroborates this finding, as these resources were used the most (Table 4). Eight (22.8%) respondents reported sharing the website address, username and password with middle management and staff.

Table 5 Type of Information Shared by those who Shared Information with People Outside the SARS Steering Committee (N = 18)
Full size table

Open-ended questions were included to gain a better understanding of the barriers, strengths, limitations, and other features of a local Extranet. Most respondents indicated that they experienced no personal barriers to accessing the Extranet. However, a few respondents indicated that time to access the site and duplication of information sent through email and the Extranet were barriers. A concern was raised related to Extranet access: "Some folks had access/password who should not have access, and were following information not yet implemented or disseminated internally." One person indicated that access was expanded to others as the outbreak was prolonged.

Regarding the strengths of the system, many respondents indicated that they appreciated the ability to access current, timely and comprehensive information in one location. One respondent noted " [It was] very useful for staff to have info in one place, in early days of SARS where no other provincial web sites [were] set up to access information." Another participant noted that the Extranet was able to "dispel rumours [and] contained a broad scope of information in one locale."

Respondents identified no weaknesses of the Extranet; however, a few commented on the manpower and time needed to keep the site current as well as the reliance on IT staff to upload information in a timely fashion. A few also noted the redundancy of the site; for example, "Once the provincial site [was] up [there was] no need for the Extranet." One participant highlighted a problem with Internet access: " [The] web network went down on one occasion during [the] crisis and delayed [the] info getting out."

Respondents mentioned that using an electronic system for communication during an emergency was a good learning experience. Some noted areas for improvement: They wanted technical support to be available (help-desk) for participants who might have difficulties accessing the secure web site. They also identified a need for a system that can be activated immediately in future public health emergencies.

Potential future use of a local Extranet

Thirty respondents identified features they would like to see on a future Extranet so as to enhance communication with the public health department. Of these, 19 (63.3%) wanted to use the Extranet as a mechanism to send private email to public health experts; 19 (63.3%) wanted to use it to share a database for surveillance of infectious diseases; 15 (50%) wanted to use it as a bulletin board communication system; 9 (30.0%) wanted online certificate courses on public health matters; and 1 (3.3%) wanted to use it as a mechanism for physicians to send immunization data. From the open-ended questions, eleven respondents mentioned that the Extranet could be used for communicable disease communication, surveillance and/or reporting, and two respondents believed that the Extranet could support preparedness for future public health emergencies.

Discussion

Our results showed that participants preferred email, face-to-face meetings, or phone communication to any Internet communication strategy. This is not surprising, considering these strategies are more interactive than visiting a typical web site. Respondents indicated that they wanted future Extranets to support private email to experts as well as have a bulletin board for feature use, validating our conjecture that they would prefer more interactivity. Two-way communication features may be important features to consider in the creation of future Extranet solutions so as to enhance interactivity. Further research is needed to measure the perceived value and impact of interactive technologies. In addition to having synchronous communication modes, such as a discussion board or email link, web-conferencing and real-time online communication using text chat may provide more effective interactive web-based solutions.

We can speculate on the various reasons why email was a preferred method of communication compared with web sites. An administrator likely views the use of email as part of his or her daily routine activity, but does not view searching web sites as a routine. Information is automatically 'pushed' to users in email, whereas users need to actively seek information from an Internet site. Furthermore, a web site is typically less interactive than email, with which users can easily reply to senders. Although the Hamilton Extranet had an email link, it was not used. The Extranet was likely not perceived by users as a two-way communication tool but, rather, a place to find information.

The Hamilton Extranet was rated low relative to other SARS web sites. Perhaps this was because the newly developed Hamilton Extranet had not had a chance to establish its credibility compared with long-established sites such as those from the CDC, WHO and the provincial MoHLTC. Despite this finding, the majority rated the Hamilton Extranet high overall, considering its own merits. Possible reasons for this positive opinion may be extrapolated from the findings: Given the most common reasons for visiting the site, the Extranet was perceived as useful to support local information needs such as meeting minutes, and Hamilton directives. In addition, until the provincial SARS site was launched, the Hamilton Extranet was the only source of provincial information. Once the MOHLTC website was launched, redundancy with Extranet information was inevitable; our findings supported this view. Also, provincial MoHLTC staff may not have been interested in specific local issues.

Despite the Extranet's shortcomings, the findings generally support the view that access to both local and provincial information is needed; however, better planning, coordination and alignment of local and provincial web communication strategies are required. It is essential to clarify without delay where the responsibility for communication begins and ends among local, provincial and federal public health agencies during emergencies.

The development of a local Extranet must be viewed as an evolutionary process. SARS Extranet users were moving away from email and fax communication to rely more on the Internet. The Extranet was built quickly to meet immediate local needs during the SARS emergency without taking time to fully engage end-users in design. Governance structures are needed to set guidelines about the practice of sharing passwords. Health units are thus urged to work with their communities before the next emergency to create a secure password-protected Internet communication system. Through such partnerships, local communities will feel more ownership and familiarity with the communication infrastructure.

Two areas of concern for a future Extranet are the security of the site and privacy of information. Many users shared with others both the information and the user IDs and passwords. Had there been sensitive information on the Extranet, a risk of improper access or release of information would have existed. These issues can be dealt with by more stringent security policies and technological improvements to the Extranet.

Looking beyond SARS, respondents were asked to respond to potential future Extranet applications for public health: Communication on communicable disease was the most common response. Not surprisingly, this response reflects the needs of the SARS Steering Committee, which was responsible for infection control during the SARS emergency. An Extranet also has the potential to include activities such as surveillance and public health education. In addition, an Extranet for use during non-emergency situations may create a useful infrastructure to support local communication in future emergencies.

This study has some important limitations. The survey was conducted three months after the initial SARS outbreak, so participants may not have accurately recalled their use of the site. This uncertainty is further complicated by the wide variety of SARS sites. The lack of statistical significance may be a result of the small size of the respondents. Finally, because the Extranet was developed quickly, it was not possible to plan for a rigorous evaluation. Qualitative evaluation methods would have been useful to answer questions about why participants rated the Hamilton site lower than other sites. Given more time for planning, set-up, and design of the Extranet, long-and short-term objectives would have been developed from which to frame a program evaluation. A more accurate system for tracking individual use would also have been incorporated. Better assessment of the human and financial costs of running the Extranet would have been a useful measure against the perceived value of the Extranet.

Conclusion

In conclusion, our study showed that interactive communication strategies – email, face-to-face meetings and teleconferences – were preferred over static web sites during the SARS outbreak. This finding has important implications for the future development and evaluation of web-based communication solutions. It is likely that more interactivity in web-based communication solutions would enhance their use and value. Although the CDC, WHO, and federal and provincial sources of SARS health information on the Internet were rated higher than that on the Hamilton Extranet, a local area Extranet appears to have an important role in supporting local information sharing and communication in an emergency situation. There is a need, however, for anticipatory planning, coordination and development of an Extranet communication system to ensure we are prepared for the next public health emergency. In addition, the findings indicate that users envisioned an expanded role of a local Extranet to support public health communication and education beyond the management of emergencies. Lessons learned from this study provide a foundation on which to build for future emergencies.