A Community of Concern: The Massachusetts Coalition for the Prevention of Medical Errors

One day in January 1997, John Noble, an internist from Boston City Hospital who I knew from somewhere—perhaps residency days—walked into my office and said, “We should form a state coalition for the prevention of medical errors.” His idea was to bring to the table the key players in health who tended not to talk much with one another—regulators and the regulated, academics and practitioners, etc.

With the commissioner, we called a meeting of leaders of the Department of Public Health (DPH), MHA, MMS, MassPro, the federal peer review organization, and several hospitals. We stated that our hope was to drive improvement by sharing information and to restore the public trust by increasing public awareness of what we were doing to prevent errors. The Coalition would make information available to health professionals and healthcare institutions for quality improvement programs. It would be a vehicle for taking action to improve care. Everyone was enthusiastic.
By May a number of additional organizations had signed up, including the state licensing boards, nurses organizations, the Harvard Controlled Risk Insurance Company (CRICO), the American Association of Retired Persons (AARP), state and federal agencies, and professional associations, as well as several hospitals and clinical researchers.
We agreed on a mission statement that our goal was to develop and implement a statewide initiative to improve patient safety and minimize medical errors. The specific goals were: • To establish and implement best practices to minimize medical error. • To increase awareness of error prevention strategies through public and professional education. • To identify areas of mutual interest and minimize duplication of regulatory and The Joint Commission requirements so that efforts are focused on initiatives that can best improve patient care.
The energy at the first meeting was palpable. Virtually everyone in Massachusetts had been touched by the Betsy Lehman story. Most of the participants, however, knew little else about patient safety. Many were not aware of the Medical Practice Study or of the recent Annenberg Conference. But they were eager to learn and anxious to be at the table. From the beginning, a critical element in moving ahead was the strong support of the DPH and the MHA, who provided staff and office space.
The focus of this and the other early meetings was on framing the problem properly and understanding the perspectives of the different stakeholders (providers, regulators, public, and media). We acknowledged the tension existing between providers and the agencies that regulate them. This was the first time that healthcare providers and government agencies in Massachusetts had ever sat down together to talk openly about medical errors and what they could do together to prevent them.
We believed that the strength of the Coalition would come from participation of representatives from all stakeholders. Thus, a concerted effort was made to ensure that the membership reflected all segments of the healthcare industry, regional interests, providers, payers and regulators, as well as all types of practitioners. We enlisted membership from state and federal agencies with responsibility for licensure and oversight; professional associations representing hospitals, physicians, nurses, nursing executives, and long-term-care institutions; individual healthcare providers; malpractice insurance carriers; accrediting bodies; clinical researchers; and consumer organizations.
Four people provided the leadership that made it happen. John Noble was an academic internist at Boston University who brought a practicing physician's concern about safety, as well as the perspective of the chief regulator, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO), where he was a Board member and later chairman. Leslie Kirle was an enthusiastic administrator and public health advocate who represented MHA's strong commitment and support. Connie Crowley-Ganser, a registered nurse and the quality VP at Children's Hospital, brought the perspectives both of nursing and hospitals. Nancy Ridley was an experienced bureaucrat with the DPH who felt a strong obligation to make healthcare safe. The energy of these four people, their skills, and their cooperation as a team were crucial to the early success of the Coalition. As noted, members had never all worked together on anything as sensitive as medical errors. These four leaders embodied something quite revolutionary: not just collaboration, but enthusiastic commitment to a cause by a diverse group of key stakeholders.
The Coalition was officially launched on July 31, 1998. The mission statement, structure, and process had been developed, and 21 organizations had confirmed their commitment to the Coalition's mission and goals. The Massachusetts Hospital Association took the lead in providing the initial seed money and resources to launch the Coalition and move it forward. DPH and MMS also provided support. The full list of participating organizations is in Appendix 8.1.
We held a press briefing to educate selected print media about the Coalition and its mission. This resulted in several positive stories in key newspapers and journals. It was an important first step in engaging the public in a meaningful dialogue about errors and strategies for prevention.
In a presentation at Annenberg II a few months later, John Noble reflected on the Coalition and its initial success. He noted that the motivating force for buy-in for all members was the shared goal of making the healthcare system as safe as possible for patients and care providers.
From the beginning, the Coalition worked unceasingly to promote communication between key parties. In addition to being a forum for the "heavies" (professional societies and regulators), it also provided a setting that encouraged input from clinicians and consumers as individuals and through their organizations' representatives. The message was clear: patient safety is everyone's responsibility. This emphasis on inclusivity helped enlist broad support for practice and systems changes while building trust and credibility.
John Noble and Connie Crowley-Ganser co-chaired the meetings of the Coalition for the first several years, and they and Leslie Kirle "made things happen." The founding members were the steering committee until a governing Board elected by the members was established in 2002.

Medication Consensus Group
Early on, the Coalition formed a Medication Consensus Group to focus on preventing medication errors. This built on an earlier MHA project that showed poor use of safe medications recommended by the Institute for Safe Medication Practices (ISMP). The group included nurses, physicians, pharmacists, and administrators representing 20 hospitals of different sizes from around the state. They developed short-term (immediate implementation) recommendations, such as unit dosing and removal of concentrated KCl from nursing units, and long-term recommendations such as bar coding, computerized prescriber order entry systems, and electronic medication administration records.
By mid-1999, we had finalized Best Practice Recommendations to Reduce Medication Errors. After they were formally endorsed by the Coalition, I made a presentation about it to the MHA Board, assuming their support would be pro forma since the hospital association had been a leader of the medication safety effort even before the Coalition was founded.
To my surprise, some members of the Board, CEOs of hospitals, were dubious. They were concerned about "telling doctors how to practice." Fortunately, others spoke out in its defense, and the Board approved it. The DPH also gave its stamp of approval.
The best practice recommendations were then sent to doctors and hospitals. Again, we got helpful media coverage. In 2001, a survey of Massachusetts hospitals showed that 70% had fully implemented some of the Best Practice Recommendations and 90% had partially implemented some. The recommendations soon spread widely. Within a year they were being used by hospitals in Michigan and Wisconsin in addition to Massachusetts. They were cited in the IOM report.
Looking back 20 years later, it is gratifying to note that while only a few of these practices were in common use at the time, they are now all standard practice. The Coalition was not the only group pushing for standards-ISMP had long advocated most of them-but it was one of the first outside the pharmacy world to do so.
The Medication Consensus Group also developed Safety First Alerts. The first three, Look Alike/Sound Alike Drugs and Packages, Transcription and Administration of Medications, and Automation, were published in 2000. The Group also collaborated with the Institute for Family-Centered Care to develop and publish a brochure for patients, Your Role in Safe Medication Use. This was distributed to physicians to be given to patients in their offices. It was well received and is still available 20 years later!

Leadership Forum
In July 1999, the Coalition and the Massachusetts Medical Society sponsored another innovation: the first Leadership Forum. We brought experts together with a diverse group of stakeholders to talk for the first time about barriers and solutions to medical errors.
After welcoming remarks by the leaders of the MMS, MHA, and DPH, John Noble gave the history of the development of the Coalition, and I gave the keynote address on creating a culture of safety. Marty Hatlie spoke about the NPSF and Eleanor Vogt presented the new video, Beyond Blame. John Nance of ABC News then moderated a star-studded panel of local experts to discuss barriers to talking about errors.
The forum was a great success and became an annual event thereafter, focusing on specific issues, such as reducing restraints and seclusion, communicating unanticipated outcomes and medical errors, and improving safety and quality of ICU care.

Regulatory Consensus Group
A Regulatory Consensus Group was created to align regulatory environments to facilitate adoption of best practices. We had the right people at the table to work on this issue and compare current regulatory requirements. Could we consolidate forms to simplify reporting?
A workshop was held with stakeholders, followed by development of a detailed comparison of requirements levied by the Board of Registration in Medicine (BRM), DPH, the Department of Mental Health (DMH), the Medical Examiner, JCAHO, the Center for Medicare and Medicaid Services (CMS), and the Food and Drug Administration (FDA) for reporting of incidents, time frame, nature of the incident, investigation, corrective measures, definitions, and codes. The group brought awareness in an objective and concrete way to the magnitude of overlapping regulatory requirements and served as the seed for the later work on accountability that the coalition spearheaded.

Restraint Consensus Group
The Restraint and Seclusion Policy and Practice Consensus Group addressed these issues in various venues: psychiatric care, children's care, emergency rooms, and long-term-care facilities. It brought together leaders, staff, administrators, physicians, and nurses from across Massachusetts to review practices, share experiences and techniques that worked, and brainstorm new ideas to minimizing the use of restraints.
The group's report, Best Practice Recommendations To Improve Patient Safety Related to Restraint & Seclusion Use, advocated the appropriate use of restraints and seclusion by ensuring staff are welltrained, implementation of a comprehensive clinical assessment before restraints are applied, development of guidelines for the need for restraint or seclusion use, routine monitoring of the safe use of restraints and seclusion, and education of patients and their responsible parties about the organization's restraint and seclusion reduction efforts. The Coalition promoted adoption of these recommendations by developing an improvement workbook and through educational programs and leadership initiatives showcasing restraint-free practices.
Through all the early years, Leslie Kirle of MHA was the person that made the initiatives work. She was the "go to" person who recruited members for consensus groups, organized and convened meetings, and generally made it all work. It was a heady time, with a multitude of projects and intense interest by all members. Many people had good ideas. Leslie turned them into action. But Leslie could only do it part time. She had other duties at MHA. The Board recognized the need for a full-time director. It took more than a year to find the right person, but in the end they succeeded brilliantly.
In 2001, Paula Griswold was appointed as the first executive director. She quickly took over management of the many Coalition activities. In addition to updates on activities of the participants, she built a strong educational component into the monthly Coalition meetings. Coalition members and local or national experts share research and new programs in patient safety. This educational aspect of the monthly meetings has been very helpful to the members and a key reason they attend the meetings regularly.
Also in 2001, Senator Moore succeeded in getting the state legislature to create the Betsy Lehman Center. However, it would be several Paula Griswold. (All rights reserved) more years before it would be funded and able to carry out its mission of motivating and implementing statewide safety programs. By 2001, coalitions had developed in Michigan, Pittsburgh, and Delmarva (Delaware, Maryland, and Virginia).

DPH Project
But were we actually making patient care safer? By 2002 the Coalition, MHA, and DPH recognized the need for a more aggressive approach to action and research to implement safe practices and to improve reporting. In collaboration with the DPH, we applied for and received a substantial grant from the Agency for Health Research and Quality (AHRQ).
The grant had four aims: to improve the DPH reporting system, to evaluate hospital leadership perceptions of public reporting, to measure perceptions and experience of hospitalized patients concerning adverse events and disclosure, and to develop and implement two safe practices. Eric Schneider, Joel Weisman, and Arnie Epstein of HMS and HSPH and Jack Fowler of the Center for Survey Research led the research team for the first three aims.
The purpose of Aim 1 was to evaluate and improve the DPH mandatory hospital reporting system (MARS), possibly by adopting the National Quality Forum (NQF) list of serious reportable events. After evaluating the content and characteristics of a representative sample of 800 incident reports made by 72 hospitals during 1999-2004, the researchers concluded that if Massachusetts had adopted the NQF standard and accompanying list of reportable events, up to 83% of incidents would not have been reported.
A new system was developed, along with a data abstraction tool to capture key elements from medical records. Use of Internet technology was analyzed to identify where there might be a good fit for the medication error-web-based reporting system, but no solution was identified. In 2003 the project was expanded to include a comparison of MARS to the patient care assessment (PCA) reports required by the Massachusetts Board of Registration in Medicine. However, they were unable to reach the goal of creating a single combined report form.

Surveys
To evaluate leadership perceptions of public reporting, CEOs and COOs were surveyed in six states: two with mandatory reporting and public disclosure (CO and MA), two with mandatory reporting and no public disclosure (FL and PA), and two with no mandatory reporting (GA and TX). The results from the sample of 203 hospitals were surprisingly similar: 69% believed that public disclosure discourages internal (within the hospital) reporting, 79% believed it encourages filing of lawsuits, and only 28% felt it improved patient safety [1]. Although all three groups were strongly opposed to public disclosure, hospitals in states where it was required were less concerned about increased lawsuits, suggesting that familiarity bred less contempt.
Patient perceptions and experience with adverse events in the hospital were evaluated by a patient survey of 2582 randomly selected patients from 16 hospitals in Massachusetts. The results were shocking: 25% of patients reported "negative" events that study physicians identified as adverse events (AEs). Three-fourths of these were significant or serious; 31% were preventable. These rates were 2-3 times those reported by previous record review studies [2].
The study then compared the yield of patient reports to standard record review by examining their medical records using the Medical Practice Study method of nurse screening and two physician independent reviews (Chap. 1). Patients reported AE in 29% of cases, and record review found an additional 11% of patients with AE. But the striking finding was that three-fourths of patient-reported AE were not discovered by record review [3].
Further analysis showed that disclosure of the AE to the patient by the medical team only occurred 40% of the time. Disclosure was more likely if additional treatment was needed and less likely if the AEs were preventable (an error). Patients were twice as likely to rate the quality of care high when there was disclosure [4]. High patient participation in their care was associated with fewer AE (49%) and higher likelihood that patients would rate the quality of their care good or excellent [5].

Implementing Best Practices
The fourth part of AHRQ grant provided support to the Coalition and MHA to develop two voluntary best practice initiatives and get them adopted statewide.
We began by widely soliciting input, calling on clinicians to tell us what "kept you awake at night." To select the practices, we used four criteria: importance, existence of elements of a practice, effectiveness, and feasibility of implementation. From ten widely suggested topics, we chose two: reconciling medications and communicating critical test results. It fell to me and the Coalition staff to make them happen, so I chaired planning groups for each. We made sure to have several change experts from IHI on each committee.

The Reconciling Medications Project
Reconciling medications is the process of making sure that "every hospitalized patient receives all the medications they were taken prior to admission unless they are specifically discontinued by their caregivers and ensuring that they are ordered in the correct dose, route, and frequency." [6] It is a problem because the information may be difficult to obtain, and the responsibility for doing it is unclear. Often, it just isn't done. Reconciling is a classic system problem.
Gina Rogers staffed this project and did a terrific job. We convened a consensus group of physicians, nurses, and pharmacists to develop the best practice, as well as core measurements and an implementation toolkit of suggested strategies for the QI teams to use.
The best practice for reconciling included four steps: (1) establish who (doctor, nurse, pharmacist) has primary responsibility, (2) obtain an accurate preadmission medication list, (3) write accurate admission orders, and (4) reconcile all variances. In addition, the best practice called for providing continuing support and maintenance by adopting clear policies and procedures, adopting a standard form, and providing ongoing education and monitoring.

The Reconciling Medications Project
To implement the practice, we used the IHI Breakthrough Series Collaborative approach in which hospitals use the PDSA (Plan-Do-Study-Act) Model for Improvement. Frank Federico joined us from IHI to help.
The MHA urged all hospital CEOs to participate. Of the 50 hospitals in the state, three-fourths did. Baseline risk assessment in 20 hospitals revealed that 59% of medications were unreconciled. The need for the project was clear.
We brought participating teams together four times in 2003 and 2004 for 2-day learning sessions, coaching on the PDSA method, and to report on progress and share successful strategies. The teams tested implementation strategies and monitored their progress with common measures. They filed monthly reports and communicated with each other over a listserv guided by expert faculty.
Adoption of the safe practice proved challenging. At the conclusion of the collaborative, 64% of hospitals had succeeded in establishing a workable system for reconciliation of medications, but only 20% had succeeded in getting it used hospital-wide at all locations.
A survey of hospital teams after the collaborative ended showed that the factors associated with successful implementation were those common to all collaboratives: strong leadership support, engagement of key stakeholders, use of small tests of change, use of data and examples of errors to motivate change, measuring whether changes are leading to improvement, and attendance at the collaborative learning sessions.
The key lesson specific to reconciliation was that there must be a clear assignment of responsibility-everyone must know whose job reconciliation is, and if it is the admitting nurse, there must be a clear backup assignment of a physician or pharmacist to correct the variances.
The second and even more challenging practice was communicating critical test results.

Communicating Critical Test Results
Communicating critical test results is the process of ensuring that test results are immediately and reliably communicated to the responsible physician. For critically ill patients or those with life-threatening conditions, getting the results of blood tests, imaging, EKGs, or biopsies in a timely fashion can make the difference between life and death.
One might assume that would occur without fail. Unfortunately, that is not the case-even in the best of hospitals. Wide variations exist in the definitions of critical test results and how they are communicated to the responsible clinician [7][8][9]. Each hospital had its own system; there were no uniform standards at the state or national level. Even within a single institution, laboratories, radiology, and cardiology often differed in their practices, and significant delays occur frequently. All shocking, when you think about it. Lives are in the balance-and lives can be lost when treatment is delayed.
The Communicating Critical Test Results collaborative took on these problems. As with the reconciling medications project, a Consensus Group was convened that included the full array of stakeholders: doctors, nurses, pharmacists, and administrators, plus representatives from blood testing laboratories, radiology, cardiology, and pathology.
The process of developing the safe practice recommendations took months, but under the competent leadership of the project director, Doris Hanna, and medication safety expert David Bates, the Consensus Group succeeded in developing a set of Safe Practice Recommendations (Appendix 8.2) and a "starter set" of critical test results.
Definition and timing. The essential first point is that each institution must reach a consensus about which tests are critical. Criteria must be agreed on and applied uniformly in all venues, laboratory, radiology, cardiology, etc., and in all practice areas: inpatient, outpatient, and ED. The typical hospital list is too long, definitions are not clear, and there is no agreed-upon standard for what qualifies as a critical test result (CTR). We recommended a color-code system that is useful and easy to understand: Red values indicate the patient is in imminent danger of death, significant morbidity, or serious adverse consequences unless treatment is initiated immediately (e.g., a blood potassium level of 6 or greater). Results must be reported as soon as possible, at most within an hour.
Orange values are abnormalities that warrant rapid, but not immediate, attention by the responsible clinician (e.g., BUN over 100). The report should be delivered to the responsible party within 6-8 hours.
Yellow values are abnormalities that are not urgent but require diagnosis and treatment in a timely and reliable manner (e.g., a biopsy showing cancer). Maximum time: 3 days.
To whom should the report be given? This must be the person who can take appropriate action. In many institutions the report was delivered to the nurse on the unit, who then had to find the doctor to take action. It was agreed that practice must stop.
Who does the report go to if the ordering provider is not available? This proved to be a difficult problem. If the responsible physician is not in the hospital, finding them quickly can be difficult. If it is a resident, they may no longer be on call. Hospitals must implement call systems that link every patient with a responsible provider at all times.
How should results be reported? For red values, person-to-person verbal communication by phone to the responsible physician was deemed essential. For orange and yellow values, indirect delivery by e-mail or through an intermediary such as a nurse or ward clerk is acceptable. The provider must acknowledge to the sender receipt of the result within the defined time frame (6-8 hours or 3 days), and the system must verify that it happens.
The full set of recommendations can be found in Appendix 8.2, which also includes the implementation context for each recommendation. For example, the recommendation for red results includes the explicit steps of whom to call in addition to the responsible physician; what to do if no response after 15, 30, and 45 minutes; and the failsafe plan at 1 hour.
To facilitate adoption, the consensus group developed a starter set of specific thresholds for red, orange, and yellow values for all laboratory, cardiology, and radiology tests. Hospitals were encouraged to modify these values to make them their own. The starter set is available at http://www.macoalition.org/initiatives.shtml.
As with reconciling medications, the recommendations and starter set of critical test results were disseminated in a statewide collaborative of Massachusetts hospitals where we assisted hospital teams in implementing the system, testing changes, and sharing successful strategies.
A year after the conclusion of the two collaboratives, we commissioned the Center for Survey Research to survey the participating institutions to determine the extent to which they implemented the new practices. How successful were we in actually changing practice?
Of 66 acute care hospitals in Massachusetts, 58 (88%) participated in 1 collaborative and 32 participated in both. For reconciling medications, 50% had some implementation, and 20% had fully implemented them. For communicating critical test results, 65% had some implementation, and 20% fully implemented [10].
These rates were comparable to IHI success rates for collaboratives. It would be another several years before Peter Pronovost demonstrated the power of a more intensive implementation strategy to yield a much higher rate of implementation (75%) of a safe practice for insertion of central lines [11,12]. (See Chap. 6.) The major barriers to success were resistance to change, complexity, and competing priorities for staff time. Few teams met as frequently as required, and hospitals sometimes didn't send full teams to collaborative meetings. Despite a lot of advance preparation, getting leaders involved and learning change methods proved difficult for most.
We published the recommendations from each of the initiatives and the results of our experiences in three papers in The Joint Commission Journal on Quality and Patient Safety in 2005 and 2006 [6,13,14] and an overview paper in Quality and Safety in Health Care in 2006 [10].
Despite the mixed results, these initiatives had considerable impact statewide and on individual participants. They demonstrated that important systems change was possible and that the Coalition was a major force making that happen. A nurse manager at a hospital that implemented both practices commented that the work she did was the most rewarding work she had done during her 25-year career.
But the greatest impact of these initiatives was nationwide when The Joint Commission (whose journal published our results) made them two of their National Patient Safety Goals, signaling to all hospitals not only that these were important, but that hospitals were expected to implement them. By 2006, The Joint Commission reported that 90% of hospitals had improved reporting of CTR and 100% had developed a process for reconciliation.

Impact of the Coalition
The coalition was a powerful force for change in Massachusetts. One reason is clear: from the beginning, the key players-the Department of Public Health, the Massachusetts Hospital Association, and the Massachusetts Medical Society-were enthusiastic participants and provided both leadership and material support. This was several years before the legendary Institute of Medicine report, To Err Is Human. The stimulus was much closer to home: the tragic death of Betsy Lehman, which had a powerful impact on our community at every level.
We had another advantage: Several of the national leaders in quality improvement and error prevention were in Boston, and by 1997, thanks to IHI, there was already a cadre of people in the hospitals with first-hand experience in improvement. They welcomed the support of their work and the opportunity to learn from others. The Coalition got the conversation about patient safety into the C-suites and board rooms. Patient safety became a priority.
The support of the Massachusetts Hospital Association was particularly critical. The president, Ron Hollander, strongly supported the coalition and sincerely wanted the effort to succeed, as did his successor, Andy Dreyfus. MHA provided space, staff support, and day-today leadership by Leslie Kirle. Likewise, strong support from the Commissioner of Public Health, Howard Koh, and leadership by Nancy Ridley, the DPH representative to the coalition, were critical to getting the coalition going.
Since the coalition came into being, a number of other states and regions have created coalitions: Arkansas, California, Colorado, Florida, Georgia, Iowa, Maryland, Minnesota, Pennsylvania, Tennessee, Texas, Utah, Virginia, and Wisconsin [15].
Under Paula Griswold's leadership, the Coalition expanded its efforts. She made the monthly meetings a "must-do" for members who want to stay current with developments in patient safety. A number of statewide educational programs have been held. The Coalition continued to convene collaboratives, including ones on long-term care and ambulatory care. Keeping up is also facilitated by links to relevant research and notices that the Coalition distributes each month about virtually everything happening in patient safety worldwide. The Coalition website, http://www.macoalition.org, lists initiatives and educational programs.
The Coalition accomplished an incredible amount in the first few years, both in agenda setting and in activities that brought together key stakeholders to produce meaningful deliverables. It was a major force in beginning to change the mindset of its members away from punishment of individuals to changing systems, the paradigm shift that drives patient safety. Its initiatives produced tangible results, driving home human factors lessons about the effectiveness of systems change. It truly changed the conversation and understanding. Now, more than 20 years after its founding, the Massachusetts Coalition for the Prevention of Medical Errors continues to be a major influence for patient safety in the Commonwealth of Massachusetts. • Describe explicit steps in notification system; when reporters should initiate and follow up on notifying the ordering provider. • Use direct person-to-person call to provider, not secretary or other intermediary. (A backup call to a nurse may also be advisable.) • Develop a fail-safe plan for communicating CTR when ordering or covering provider cannot be contacted within the time frame. • Ensure 100% acknowledgment for every test result on the list, i.e., that the sender has received confirmation from the responsible recipient that they have received the report. Caller must know that a responsible party has the information-for all three priorities.
6. Establish a shared policy for uniform communication of all types of test results to all recipients: • Use the same policy regarding definitions and time windows across all domains. • Encourage and foster shared accountability and teamwork across and between clinical disciplines. • Decide what information should be included in the report.
7. How to design reliability into the system: • Use forcing functions at point of ordering to identify the ordering provider. • Use forcing functions at point of ordering to include a minimum of information to support the interpretation of results. • Create tracking systems to assure timely and reliable communication.
8. How to support and maintain systems: • Partner with patients in the communication of test results.
• Provide orientation and ongoing education on procedures for communicating CTR to all health providers. • Provide ongoing monitoring of the effectiveness of the systems-weekly failure rates, response times, etc. Adapted from Ref [13], Table 3 Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/ licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.