Combining mistake-proofing and Jidoka to achieve world class quality in clinical chemistry

Abstract

Introduction

There have been many wonderful examples of successful automation. Unfortunately, there are also too many examples of failures like the Denver, Colorado airport where automated baggage handling added nearly $1B dollars to the airport cost, delayed the airport opening by more than a year, and was eventually abandoned [1]. For automation to be a truly successful undertaking, it must succeed on many levels, including: (a) a financial success, (b) a performance success, (c) and a quality success. In reality these attributes are strongly related, and paying attention to a few key attributes of automation can make all of the difference in the outcome.

The data in Fig. 1 was derived from the International Motor Vehicle Study conducted by Harvard and MIT in the early 1990s. It shows the number of assembly defects per hundred vehicles as a function of the level of automation, and is one of the largest sets of data ever accumulated on this topic [2, 3]. Each point represents the average for hundreds of thousands of vehicles. The line represents the least squares linear fit to the data. Although quality is improving with automation, the line is surprisingly horizontal and the correlation is exceptionally poor (r ² = 0.014). The figure shows that automation can contribute to quality, but automation by itself does not assure good quality. There are many other important factors essential for success.

Fig. 1

Assembly defects per 100 vehicles versus the percentage of assembly steps that have been automated. The data is derived from the International Motor Vehicle Study [2, 3]. In the legend, Eur is for Europe, NIC is newly industrialized countries, US is the United States, NA stands for North America, and J is for Japan. Where a “slash” is used the left side is the management country, and the right side indicates the plant location. For Example, J/NA is a Japanese managed plant in North America

The first inclination may be to dismiss the automotive data, since Clinical Chemistry is such a different technology. However, Siloaho’s research of Clinical Chemistry laboratories in Finland reached a similar conclusion. She observed that technology changes, which were usually equated with more automation or quality system implementations, had a positive but rather modest effect on analytical quality [4]! In some cases, and for some analytes, technology changes had deteriorating effects on quality. Note how closely these observations match the results measured in the automotive world. Automation can make a difference in quality, but it is not the striking change that is generally expected.

Why automation fails to deliver world class quality

The relatively weak quality improvement achieved through automation shown in Fig. 1 is inconsistent with widely held beliefs, and undermines one of the key justifications for implementing automation. Understanding the nature of non-conformances on automated equipment is an essential part of clarifying the differences between the actual and perceived quality as the level of automation increases.

Tavormina and Buckley developed a method of showering small parts with ultrasonic and microwave energy for inspection. By sensing the reflected waves, their systems inspected every feature on small parts at rates up to 600 parts per minute. This equipment has been used primarily with automated equipment to detect and remove defective product from the production stream [5]. Figure 2 is typical inspection data generated using their equipment that compares product dimensions to the control limits.

Fig. 2

Simultaneous measurement of multiple dimensions on small parts processed on automated equipment. Note that most parts are well within the control limits with random excursions that dramatically exceed specifications. This figure was provided to the author in correspondence from S. Buckley, at CogniSense in San Jose, CA, on May 11, 1992

Unlike the distribution that would be expected based on the traditional variation paradigm, Shawn Buckley discovered that the dimensions on most parts produced on automated equipment were very close to nominal value, with some shifts and drifts in the mean as illustrated by the wavy horizontal line in Fig. 2. However, on virtually every automated process they found that the random defects were typically 0.1–0.5%, illustrated by the random excursions that exceed control limits. Taking a traditional random sample, virtually every product would be within tolerance. About once in every 200–1,000 samples, a defective product will be discovered, but normally this observation will be discarded as an outlier. As a result, using traditional inspection methods the quality appears to be outstanding when the true defect rate is still in excess of 0.1%. For comparison, this non-conformance rate is 20–100 times worse than world class quality leaders are achieving with mistake-proofing.

The rare random events observed in automated processes mirror the experience in manual operations. These random events have been traced to mistakes, which have been cited as a common cause of nonconformities in clinical laboratories. Plebani and Carraro [6] confirmed 189 laboratory mistakes among a total of 40,490 analyses, or a 0.47% error rate. Goldschmidt and Lent [7] studied faults or near-accidents (FONA) at the hospitals of Tilburg in the Netherlands. In this study, 93–97% of the mistakes were traced to human error. Lapworth and Teal [8] cited two studies where clinical laboratory mistake rates were in the range of 0.3–2.3%, although their own study identified an average mistake rate of only 0.05%. Boone [9] observed overall mistake rates of roughly 100 per 100,000 (0.1%) in a hospital clinical laboratory. Similarly, in a study of turnaround times for urgent clinical tests, Pellar et al. [10] found that mistakes were a leading source of delays. Notice that the mistake rates of 0.1–0.5% for automated equipment identified by Tavormina and Buckley is in the same range as the error rates cited in clinical chemistry laboratories (0.05–2.3%).

The link between complexity and quality

The system-level defect rates are directly proportional to the complexity of system-level tasks represented by the time it takes to execute the process. This relationship is shown for three manufacturers and 28 products in Fig. 3 [2]. The solid lines are the least-squares fit to the data, and have virtually an identical slope. It should be noted that the production for each product involved both manual and automated processes in this study. All manufacturers used a mix of manual and automated operations to make each product. The consistent correlations demonstrate that the link between defects and complexity is more robust than the relationship between defects and the level of automation, and points to some of the most important changes that can be made to improve quality.

Fig. 3

Defects per unit versus the system-level complexity determined by the total manual assembly time (TAT) minus a constant (t ₀) times the total number of assembly operations (TOP) [2]. Note that the scales are logarithmic

The data presented in Fig. 3 reveals two distinctly different ways to improve quality: (a) make the task easier, or (b) improve the quality control. When a product or process is simplified, the complexity decreases. For example, if the disk drive manufacturer could change the Complexity Factor for a process or design from a value of 1,000 to 500 s, the expected change in defects per unit would decrease from approximately 0.6 to 0.25, a significant improvement. The complexity of a task is never fixed. It can always be simplified, and often the complexity can be cut in half with relatively minor changes. Also note the differences in the vertical positions of the curves for the three companies. The disk drive manufacturer would also reduce defects if their quality control improved to more closely match the performance of the electronic equipment manufacturer, who is outperforming Motorola.

Making the system level task easier

Workers are anxious to convert to new methods of doing work only when it clearly helps them with their task. Surprisingly, automation may sometimes make the task more difficult. I encountered a classic example while consulting for a hospital. The number one national patient safety goal for improvement identified by JCAHO, the organization responsible for accreditation of healthcare facilities in the United States, is patient identification [11]. This has been the highest priority patient safety goal since JCAHO began to specify goals in 2003. To prevent medications being administered to the wrong patients, the Electronic Medication Administration Record (eMAR) systems are being incorporated into medical carts. The eMAR systems replace manually written records of medication administration with barcode scans. Nurses pull up prescriptions on a computer, scan the patient’s armband barcode, and scan the medication barcode before administering medications to create the electronic record of the medication, and make an automated check of the medication against the correct patient and prescription.

Although automating the record keeping solves some problems, the nurses must now move the med cart to the bedside, which is more difficult than carrying medications into a patient’s room. Furthermore, if they are interrupted by an urgent problem, the med cart cannot be left unlocked and unattended, delaying urgent response as they move and secure the cart. In some cases, the password entry and barcode scanning can be more difficult and time consuming than a manual written record. As a result, the automated electronic comparison of medication barcodes and armbands imposes significantly more difficult tasks on nurses who are already stressed. One of the results of this difficulty is that the correct use of the eMAR system is often circumvented by scanning barcodes in patient’s records rather than the one on the wristband at the bedside. One of the critical tests of effective automation is, therefore, whether or not it makes the system level activities easier for the user and not just the task that the equipment is designed to automate.

Jidoka: automation with a human touch

To understand the best method of automation we need to review Jidoka, a Japanese term for their approach to automation; the best translation of Jidoka is “automation with a human touch.” Jidoka has the following essential attributes [12]:

• The work of the equipment and operators is distinguished.

• The equipment and operators work independently.

• The setup, loading, and unloading of equipment is mistake-proofed.

Work distinction

Generally, the optimum work for operators is to setup, load, and unload equipment. The work of equipment is to repeatedly perform an operation with consistency. In many cases, these roles are confused. Why are operators best at setting up, loading, and unloading equipment? These are generally the most difficult tasks. The variety of tasks and conditions involved in these operations is often what makes them very difficult to fully automate. On an economic basis it requires roughly 100,000 repetitions of an operation each year to justify the lowest levels of automation. Sophisticated automation requires 1,000,000–5,000,000 operations per year [13]. Thus, it is more effective for the operator to setup and load equipment like a clinical chemistry analyzer, while the analyzer is better at performing the analysis.

Independent operation

The machine should be able to do its work while the worker performs his or her work. In too many cases, the worker stands and watches the machine work, or the worker is busy while the machine is idle. Independence is improved if the work “flows” through the system rather than a random movement to a variety of equipment. In order to work independently, the machine must have new capabilities. This requires the ability to detect when a non-conforming condition exists and to shutdown, reject bad product, or be self-correcting.

A laboratory centrifuge illustrates one type of machine that may or may not be able to work independently. If a centrifuge is imbalanced, standard models will normally shut themselves down. An alarm or warning would help by alerting the operator of the imbalanced condition, so that the problem may be corrected immediately rather than waiting until the operator believes that the process has been completed [14]. Better yet, a self-balancing centrifuge could complete its operations independent of additional operator action.

Often extremely simple sensors make it possible for the machine to work independently. Many products for the clinical chemistry environment already have this capability, but all should. When the machines can operate independently, the operators are free to perform their work. Independence makes it possible for laboratory technicians to operate more pieces of equipment.

Mistake-proof setup

The biggest quality problems in the operation of automated equipment are human errors [12] in setup, loading, and unloading. Of these, setup errors are the most serious, because such errors can cause quality problems for the greatest number of products. As a result, to achieve the highest levels of quality the setup, loading, and unloading must be mistake-proofed. Move instructions out of the manuals, and make them part of the machine. If the machine must be setup differently for various functions, pointers on dial gauges showing the correct setting for each specific configuration can be useful. A magnetic strip that shows the correct settings for the specific setup makes it obvious whether the setting is correct or incorrect.

Correct specimen identification begins with labeling when the specimen is collected. The Clinical Chemistry Laboratory may be blamed for errors made before they receive the specimen. Thus, it is in the best interest of the laboratory to correct these quality problems. Some hospitals preprint labels for patients. When the phlebotomist collects the specimen, they go to the nursing station to get labels for the specimen. They can inadvertently select the labels for the wrong patient, causing errors in the analysis that are potentially serious. A better approach is to use bedside label printing. When the patient’s armband is scanned, the patient identification is compared to the database of requested laboratory tests. The correct number of labels with the specific test requested for each label can then be printed on demand, virtually assuring that the right quantity and type of specimens are collected and the specimens are not labeled for the wrong patient.

Rather than a self-balancing centrifuge, an even better approach would be to mistake-proof the setup so that the centrifuge can only be loaded in a balanced way. Often analyzers are set up to perform different functions. Errors occur when a specimen is put in the wrong analyzer. Symbols could be printed on the labels matching the test to be performed, and providing a visual check on the correct analyzer to be used [14]. It would be better still if the analyzer shut down, and provided a warning if an incorrect specimen were loaded. Even better would be a unique specimen shape (perhaps a feature on a cap), that only allows it to be loaded in the correct analyzer for the sample. Naturally, the selection of the cap or specimen container would also have to be mistake-proofed. Not only should mistake-proofing assure that the product can only be loaded on the correct equipment, it should also prevent setup in the wrong orientation.

A key part of mistake-proofing is to make the state of a product or process obvious. Because the label obscures the specimen, it is sometimes impossible to determine whether a specimen has or has not been centrifuged. Redesigning the label so that it cannot cover the entire specimen, or putting a strip on the outside that changes when centrifuged, would avoid wasted time and effort and unnecessary exposure to potential health hazards [14].

Specimens may be processed twice, or a critical process may be skipped. The most common problem in this case is that the input staging area is not clearly distinguished from the output, or that the direction of staging changes from one piece of equipment to another. To prevent such serious errors throughout the laboratory there needs to be a “common” loading and unloading paradigm. For example one way to address this would be to always place items to be processed on the left-hand side of the process. When removed from the equipment after correct processing, they could always be moved to the right, just as we read from left to right.

The mistake-proofing process

The process of mistake-proofing differs significantly from traditional quality control methods. For mistake-proofing we use Toyota’s problem-solving wheel. There are six simple steps as shown in Fig. 4. When teams begin the mistake-proofing process, there are more mistakes than they can address. The first step is to prioritize and select the problems that will be mistake-proofed. Three different factors are important in identifying the highest priority problems: (a) how frequently the problem occurs, (b) the impact on the process flow, and (c) the impact on the company and customer. When the problem is selected, it is then analyzed to identify the root cause or causes, and to determine if the problem is the result of unnecessary task complexity, a mistake, or variation, which each require different control methods. If the problem is unnecessary complexity, the product or process is simplified; if the problem is due to variation, we apply the traditional methods of variation control; and if the problem is a mistake we use mistake-proofing.

Fig. 4

Toyota’s six-step problem solving pattern organized in a “wheel”

When the cause of a problem has been traced to a mistake, additional analysis facilitates the development of solutions. Mistake-proofing is a skill that grows with exercise. Novices have difficulty developing solutions because mistake-proofing is not common practice and examples are limited. A catalog of mistake-principles and examples has been assembled [15]. By classifying the mistake, teams can quickly find a summary of solution principles, and examples that are relevant to solving their specific problem. The next step is to use these principles and ideas to generate solutions. We should never accept a single proposed solution. Regardless of how simple or complex the problem, multiple solutions should be generated. The purpose of preparing multiple solutions is to compare their relative strengths and weakness. This comparison facilitates the selection of the best solution, and identifies common weaknesses that are overcome with new solutions that would not have been considered. Once the concept is selected, it is implemented and an evaluation is made of the effectiveness of the solution. If the solution is effective and will help in other work areas, the solution is standardized throughout the organization.

A mistake-proofing example

The process of developing mistake-proofing solutions can be illustrated with the centrifuge problem previously described, where the unit shuts down unexpectedly if it is not in balance. Because the unexpected shutdown of a centrifuge can cause delay in processing stat specimens, this problem was selected for correction. Variation occurs in the distribution in fill level for each specimen vial, but such problems are generally not the cause of imbalance. Imbalance is caused by putting an empty vial in the centrifuge, putting the vials in an unbalanced pattern, or failure to remove a vial from a previous loading. The initial analysis indicates that each causal factor is a mistake in this case. In the second part of the analysis, the mistakes associated with shutdown due to imbalance can be classified as follows:

• Omitted part (not enough specimens are loaded).

• Added part (too many vials are loaded or a processed vial is not removed).

• Wrong location (a vial is placed in the wrong pattern in the centrifuge).

• Wrong part (an empty vial or vial of the wrong size is selected).

• Omitted information (no warning of the shutdown provided).

• Inadequate warning (a warning is given, but is not sufficient to alert operators).

As the preceding list indicates, there are often many errors associated with tasks, even simple ones. To fully mistake-proof a process, the major mistakes all need to be addressed. In the short term, the fastest and easiest approach may be to address the problem of omitted information. A few of the thirteen principles that prevent omitted information include:

• 1. Make information non-removable,

• 2. Add sensors to detect things that are difficult to observe,

• 3. Add signals to make the information obvious, and

• 4. Create and use a checklist to identify and gather needed information,

Of these principles, only numbers 2 and 3 are useful in solving this problem. Finding relevant examples that illustrate principles 2 and 3 concludes the analysis.

Next a variety of solutions are generated. Three concepts for sensing the imbalance are presented in Table 1, labeled concept A, B, and C. For concept A, a vibration sensor is attached to the side or back of the centrifuge. When the vibration is excessive, the alarm light turns on. Adjustments in the sensor must be made to assure that the alarm does not trigger when the centrifuge would not shut down and does trigger when it will shutdown. On the second device, a current sensor detects the current of the motor draw. Shutdown turns off the current flow prematurely, which is detected to provide an alarm. Since the shutdown would typically occur during spin up, the sensor uses a timer to ensure that the power is on until the full speed is achieved. The third concept involves wiring directly in the centrifuge shutdown switch to actuate an alarm if the shutdown switch is activated. This could void the warranty, and would impact the downtime for implementation. Concepts D, E, and F illustrate alternatives for alarms: lights, audible alarms, or a combination. Concept G is the self-balancing centrifuge, which eliminates the need for warnings.

Table 1 Comparison of alternative concepts for providing a warning of centrifuge imbalance

For each attribute listed on the left, the concepts are scored on a scale of 1–5, 5 being the best performance, and 1 being the poorest performance. The best concepts have the highest total score. It is important to note that none of the concepts are ideal. Each has limitations. Overall, the current sensor scores the highest. Because of low cost, speed of implementation, and avoidance of rearranging the laboratory, the current sensor concept could be more attractive than a self-balancing centrifuge. Naturally, some of the sensor attributes do not apply to the warning devices (these are left blank in the table). Workers are most likely to appreciate a simple warning light in this environment, based on the scoring of concepts D, E, and F. It is important to note that the light should be located as close to eye level as possible, but high enough that it can be seen from every direction.

This concept comparison is based on the Pugh method, which has proven highly effective in selecting superior concepts [16]. Note that this comparison identifies key weaknesses of each concept. A vibration sensor that does not require adjustment would make this concept more attractive. As an alternative, if connection to the internal centrifuge switch could be easily made without voiding the warranty, this concept would be significantly more attractive. By this comparison, opportunities for improving existing concepts are identified, as well as the opportunity for developing new concepts.

Once the concept is selected, it is implemented and evaluated. If the concept is effective, it should be standardized. For example, if the clinical chemistry laboratory in a hospital that is part of a chain of hospitals finds a successful solution, the implementation should be communicated to all hospitals in the chain, with the management expectation that the changes are to be implemented in each facility where the solution applies.

The effectiveness of Toyota’s quality control methods has been independently confirmed in many settings [17]. Stark Manufacturing, Inc. reduced nonconformities from 0.08% to less than 0.0005% over a 4-year period with mistake-proofing [18]. In addition to virtually eliminating nonconformities, their experience suggests that mistake-proofing has roughly doubled the productivity of the operations where it is applied. One US supplier for Toyota had produced 60,000 parts before they observed their first defect (0.0017%), an incredibly low defect rate for the manufacturing start-up of a new product [19]. Given the gradual reduction in nonconformities achieved over decades of quality control improvement, the dramatic impact of mistake-proofing on quality control is singularly remarkable, and must be a part of every effective quality control effort.

Conclusions

Whether or not automation is implemented, wherever possible every product and process should be designed as though it would be automated. This requires simplification of the parts, handling, and processes. This is one of the most important quality contributions that can be made, and is beneficial even if subsequent evaluation does not justify automation. Effective automation should always reduce the difficulty of the global task. When automation is properly implemented with a “human touch”, it helps the workers perform their tasks faster and more easily with fewer errors. Rather than opposing such automation, operators will embrace it once they realize that it will help them.

Mistake-proofing does take effort, and a significant number of mistake-proofing devices are essential for controlling mistakes. As a consequence, there is no instant path to world class quality. Mistake-proofing is not the only quality tool that is required. However, when mistake-proofing can be used to control processes, it is the most effective and lowest cost alternative of all the quality methods.