Keywords

1 Introduction

A wide range of industries utilize procedures in some form or fashion. Whether flipping switches in a chemical processing plant, or assembling the parts for a military weapon, procedures provide a level of assurance that the task will be performed consistently, correctly, and in a predetermined sequence [1]. Procedures are typically a set of written instructions that guide the worker through a series of steps. Each step is intended to be followed in a predefined order. In this fashion, procedures augment the workers capacity to do the job reliably and safely. Procedures are particularly useful in situations when the job is complex, and the consequences associated with making an error can be catastrophic.

It is only within recent history that procedures were accepted as a standard way of doing business. Before procedures came into use, the quality of work was almost entirely dependent upon the “skill-of-the-craft.” This principle was based on the idea that written instructions of any kind were neither called for or needed. Rather than having a procedure in hand, workers relied on their skills and know-how of the rules to perform the job [2]. To “know the ropes” could be described as an early example of “skill of the craft.” Its origins are believed to be a nautical expression dating back to the late 18th century. To “know the ropes” meant that sailors working on early sailing ships were qualified and ready to perform the duties assigned to them. For instance, knowing which rope raised and lowered which sail, and learning to correctly tie knots [3].

Similarly, throughout the 19th century, apprenticeships were generally the means one obtained skill-of-the-craft. For instance, the steamboat pilots in the 1800’s learned how to do the job while working under the supervision of a more senior pilot. To earn their license, apprentice pilots were taught to read the river. This entailed having the mental sharpness to know what lays ahead, whether it happens to be snags, rocks, sandbars, or geographical landmarks. Most importantly, to successfully navigate the Mississippi river, the pilot had to know the depth of the water at different places as well as the nature of the currents. Ultimately, the long hours of training in the wheelhouse would give the pilot a sense of “feel” of the boat, for all seasons of the year, and all foreseeable weather conditions [4]. Mark Twain was one of those steamboat apprentices that succeeded in reading the river, and for his accomplishments, he earned a steamboat pilot license in 1859 [5].

Then beginning in the early 20th century, the basic knowledge required to interact with newer technologies became increasingly more complex. From that period, Ford Model T owners could have a manual containing diagrams for servicing their new cars. For example, one diagram provided the owner with a lubrication schedule that points out where and at what mileage interval grease should be applied [6].

Then in the latter half of the 20th century, the demand for production output particularly in the energy sector, began to expand. As a result, plants started scaling up in size and capacity to meet demand. According to Perrow [7], growth in output coincides with increasing levels of system complexity. In turn, the abrupt rise in complexity makes it harder to understand the actual risk associated with operating one of these plants. Thus, despite our best efforts at controlling risk, we are left with a limited understanding of the underlying processes. In the absence of having a comprehensive state of knowledge in terms of how a plant operates, it is difficult to recognize and anticipate what can go wrong. The cause and effect relationships between systems, subsystems, and related components are too numerous to contemplate. As a result, the risk analyst has an incomplete picture of what it takes to predict accidents and finding ways to mitigate their severity.

But, if complexity exists, it must still be dealt with in a constructive way. One approach involves building layers of safety protection against the myriad number of events that could happen. The layers can be described as defense in depth measures that increase the chances that plant mishaps both anticipated and unanticipated, can be ameliorated or stopped altogether [8]. Today, procedures significantly contribute to the defense in depth philosophy. Figure 1 [8] illustrates how defense in depth measures might be implemented to mitigate the effects of a fire.

Fig. 1.
figure 1

(Adapted from: [8].)

Example of defense in depth measures that are in place to protect assets against the threat of a fire.

Full size image

Assuming a fire begins to propagate, the fire sprinkler provides the first layer of defense in depth. Next, the structures were built with fire proofing materials, constituting the second layer of defense in depth. Finally, a procedural control is followed to routinely pick up trash and materials that could contribute to the combustible load if a fire took place. Thus, the procedure for controlling combustibles becomes the third layer of defense in depth. Overall, the three layers act in concert to avoid or minimize potential losses if the fire happened to occur.

As increasingly more complex industries progressed into the 21st century, procedural controls were endorsed as a way of assuring that large processing facilities operate in a safe and stable configuration. Regrettably, the 1986 nuclear accident at Chernoby is a testament to the importance of having procedures, and what happens when procedures are not followed. At Chernobyl, activities were underway to test the operability of safety systems during loss of offsite power. To conduct the test, the supervisors decided to violate their own technical specifications for controlling reactivity and power. As the test continued, the reactor systems began to merge toward conditions that the personnel at the plant did not understand. Essentially, by choosing to ignore the instructions that would keep the plant in a safe state, a catastrophe ensued [9].

2 Computer Based Procedures: Why and for What Purpose?

It is fair to assume that procedures are here to stay. Procedures, particularly for complex high-risk systems, provide an added measure of protection and formality to the operations. While paper-based procedures remain a traditional mainstay of many industries, there is a growing interest in replacing paper with Computer-Based Procedures. If paper is an acceptable way of managing an industrial process, why transition to paperless? Some reasons are presented in Table 1. In Table 1, one primary motivator for going paperless is the often-held belief that it is an easier way to maintain documents. Essentially, if procedures are maintained from cradle to grave in a data base, product quality goes up and the administrative burden goes down. Additionally, a collection of documents held in a data base is searchable, and it is relatively simple to link related material [10]. At the shop floor level, Computer-Based Procedures minimize opportunities to pick up and use an outdated version. Plus, with Computer-Based Procedures, the order for completing the steps is controlled by the software. As such, it is more difficult to mistakenly miss a step in the process [11]. Furthermore, Computer-Based Procedures are usually set up to facilitate data collection. Rather than write values down on paper, they’re directly entered on the screen. Once entered, the values can be checked for reasonableness and accuracy as well [13].

Table 1. Reasons for transitioning to Computer-Based Procedures. (Adapted from: [10, 11], and [13])
Full size table

Thus, there are advantages to having Computer-Based Procedures. The same advantages are simply not possible for paper-based procedures. And it is apparent that Computer-Based Procedures could minimize human errors associated with missed steps, picking up the wrong version, and entering incorrect data. But it’s less clear knowing how and to what extent overall human performance and decision making may be impacted by Computer-Based Procedures relative to paper [12]. Results gleaned from prior usability evaluations tend to suggest these concerns are warranted. In fact, there were indications that Computer-Based Procedures had an adverse impact on human performance and decision making. Specifically, these indications suggested that the path from paper to paperless environments was not as simple as it seemed. Principally, the operators reported that certain features were difficult and confusing to use [13, 14].

Over time, after observing operators and asking them questions about the problems they were having, their experiences were found to align with a few main issues. These issues are discussed in the next section.

3 Computer-Based Procedures: Issues

The issues presented here are not intended to be exhaustive, as there are different ways of looking at the problems people are having with Computer-Based Procedures. For each problem, numerous solutions are perhaps available, depending on the context of the work and goals expected to be accomplished using Computer-Based Procedures. Rather, the issues described here can be thought of as a snapshot of lessons learned that the author feels should be taken into consideration before the choice is made to transition from paper to paperless. By raising a level of awareness, it is hoped that the same mistakes and missteps that happened in the past will not be repeated in the future.

3.1 Integrate Computer Based Procedures with Current Operations

The choice to move from paper to paperless presents a brand-new way of doing business. Therefore, careful consideration should be given to the degree of impact Computer-Based Procedures may have on the organization, and the people responsible for doing the work. Referring to Moray’s [15] characterization of a highly reliable organization, possible impacts could occur at the workstation, where the Computer-Based Procedures would be pulled up on the screen and acted upon. At this level, issues could arise as to whether the screen that is intended to display the procedures interferes with established work patterns. For instance, it would be of interest to know if, while reading and responding to the procedural instructions, the operators could still access the equipment, tools, and parts they need to perform the task.

Within the local environment, it would also be useful to know if the Computer-Based Procedures can be seen under current lighting conditions. Additionally, with the introduction of Computer-Based Procedures, operators may have to reconsider how they might communicate and collaborate amongst themselves, and at the same time, continue to maintain a strong sense of situational awareness in the workplace.

Finally, because a Computer-Based Procedures system is intended to replace the former paper-based system, impacts should also be assessed at the point where procedures are developed. Essentially, the basic rules for assuring the technically accuracy apply, whether the procedure happens to be presented in hardcopy or online [11]. Additional aspects, dealing with the infrastructure for producing and releasing final procedures to the shop floor, should also be reconsidered. For example, resources should be allocated to revise the local format and style guides for writing procedure. What may have worked well for paper no longer applies to Computer-Based Procedures. Other changes include the mechanisms by which procedures should be updated when new information becomes available. Above all, the processes for validation and quality assurance may also be different. As the steps taken to validate Computer-Based Procedures will likely be unique relative to paper procedures.

3.2 Affordances of Paper Are Naturally Embedded in the Flow of Work

Despite the benefits likely to be realized by Computer-Based Procedures, giving up paper also has a downside. The downside of giving up the affordances that paper provides us in daily life. Building upon the Theory of Affordances introduced by James J. Gibson, Sellen and Harper [10] describe some affordances regarding paper. When someone skims through a pile of documents, the physical paper allows for spatial flexibility. Only a short glance is necessary to know, “….where things are.” [10] (p. 102). Whereas, compared to computer-based presentations, the operator is often limited to what can be viewed through the screen. Another closely related affordance with paper is the ease that users can scribble and jot notes directly on the page. It appears that the combined aid of the eye and the hand makes it easier to grasp the full meaning of the content. When completing the steps of a procedure, it is not unusual to mark up the pages for attention getting purposes, such as place keeping. As for most Computer-Based Procedures, a pen input feature is seldom available.

In short, according to Sellen and Harper [10], there are just certain things that people can do with paper that cannot be accommodated, at least not easily, with Computer-Based Procedures. Given the range of affordances that paper offers; it is important to carefully balance technological solutions with the inherent value paper naturally gives us.

3.3 Automatic Branching Doesn’t Make Step to Step Navigation Any Easier

There are instances where computer-based procedure systems were developed without due consideration for the tools the operator would need to navigate through the entire range of steps [13, 14]. Nielsen [16] claims it should be just as easy to navigate the Web as it would to leaf across pages in a book. However, considering the problems operators have had navigating computerized procedures, it is expected they would find the task of leafing through a book less daunting. It appears that navigational capabilities are either added as an afterthought or overlooked entirely. This lack of emphasis on navigation may also be attributed to the perception that Computer-Based Procedures are a medium for presentation. True, the technology is intended as a device for displaying procedural content. But just being able to read content is of little value if the operators do not know if they are at the correct place in the procedure. Additionally, if they discover they are in the wrong place, there is no roadmap that shows them how to get to the correct place in the procedure.

Problems with navigation were particularly prevalent when operators encountered the automatic branching feature. The succinct definition for automatic branching is: “The procedure branches automatically to the appropriate instructions based on a choice made by the technician at the specified point in the procedure.” [17] (p. 68). Expanding on this definition, automatic branching occurs when the operator is asked to read a step and, based on the information contained in the step, enter a value in an assigned field. (Typically, the value entails recording a measurement.) When the value is entered, the system advances to a next step. The action is automatic, without operator input. As a result, the operators are often left wondering why they were directed to that specific place in the procedure. Since the operators were unaware of what happened, they would occasionally believe themselves at fault for making a mistake.

The following example describes how automatic branching features are problematic for the operator. To best understand automatic branching, it is essential to first appreciate how the mechanics of branching are intended to work in a paper-based procedure. To illustrate, a paper-based procedure used to check the car’s engine oil is presented in Fig. 2. In Step No. 1, the operator is instructed to read the oil level with the dipstick. This is determined by comparing the level of the oil against two marks, ranging from FULL to LOW, that are referenced on the dip stick (See Fig. 3.) In Step No. 2, the operator checks the box for the oil level that was just read. Three choices are available (Oil level > “LOW” AND ≤ “FULL”; Oil level ≤ “LOW”; and Oil level > “FULL”). In Step No. 3, the operator picks one of three columns, depending on the oil level that was previously checked off.

Fig. 2.
figure 2

Checking the engine oil: A typical format for paper-based procedures.

Full size image
Fig. 3.
figure 3

Oil dipstick showing the “O.K.” acceptable range between “Low level” and “Full level.”

Full size image

Next, a flow chart of a computer-based procedure for changing the engine oil is provided in Fig. 4. Again, same as for the paper version, the operator is directed by the procedure to read the oil level. Once the level is identified, the operator presses a button that represents the reading. Here, the choices are the same as for the paper-based procedure (Oil level > “LOW” AND ≤ “FULL”; Oil level ≤ “LOW”; and Oil level > “FULL”). If the oil level is unacceptable, the flow chart follows a pathway that is like that for the paper-based procedure. That is, instructing the operator to either add or drain oil and read the new level.

Fig. 4.
figure 4

Checking the engine oil: Example of automatic branching.

Full size image

If the oil level is acceptable, the operator presses the button for > “LOW” AND ≤ “FULL”; . After the button is pressed, automatic branching prompts the operators to check the air filter. The system does not provide feedback that explains to the operators how or why they arrived at a step that deals with the air filter. In the absence of feedback, the operators have no way of knowing if steps for checking the oil were performed correctly or completed at all. The operators are only aware of an action taken to enter a value for engine oil. Then, without further explanation, they are directed to check the air filter. Even though automatic branching was intended to make the job easier, it was found to be confusing for the operators.

As a result, Computer-Based Procedures that utilize automatic branching, and other modes of automation as well, should be evaluated for usability before they are implemented in the field Parasraman, Sheridan, and Wickens [18] compiled a list of evaluative criteria that could help designers decide what should be automated, and if so, how and to what extent automation should be applied. One criterion having relevance to computer-based systems addressed the extent that automation affected mental workload. If, it could be determined that automation did not notably reduce the level of mental workload, then other solutions should be considered. Additionally, Parasraman, Sheridan, and Wickens [18] identified the impact of automation on situation awareness. Accordingly, if changes in system state are left up to another agent, the operator is less aware of the surrounding environment. As such, the operator loses the opportunity to be actively engaged in the process. As one might suspect, loss of situation awareness can place people in the role of being a passive decisionmaker. As for applications specific to Computer-Based Procedures, if situation awareness is found to be lacking, the operators will be unable to perform the work at their fullest potential.

Above all, Parasraman, Sheridan, and Wickens [18] support the notion that some aspects of automation hinder human performance. For these reasons, it is important to thoughtfully evaluate what the merits of automation might be and decide if and to what extent the technology is fitting to the product. By choosing to carefully weigh the benefits and limitations that automation offers, a more ordered and intuitive navigational structure will be realized.

3.4 Observe Theories of Multimedia Learning

Limited feedback was found to be one of the more troublesome issues facing the usability of Computer-Based Procedures [14]. Ideally, the system should: keep the operator appraised of the status of the operation; provide indication as to what operations are permissible; facilitate detection and recoverability of errors; and provide a capability for data validation [13]. Regrettably, it was discovered that the amount of feedback that would nominally be required to keep the operator in the loop was grossly inadequate.

This deficiency was particularly notable for the soft button controls used to operate an electronic procedures system [13]. One button, the “Complete” button, was intended to be pressed after each step was executed. But, clear feedback that would indicate a pressed button was unavailable. An automatic feed forward to the next step in the series was the only way the operator knew the button was pressed.

Because feedback was deficient, the operators would press the button multiple times, particularly if the system did not immediately respond to their request. Since the buttons failed to respond as if they were pressed, the operators were continuously frustrated trying to decide if the button was working or not. Multimedia elements, such as words, pictures, sound, and animation can be used to promote feedback in an online environment.

Software applications that utilize multimedia have generally been shown to take full advantage of the cognitive processes humans would ordinarily have at their disposal. This claim is based on Paivio’s dual coding theory [19, 20]. The premise being that learning, and the acquisition of knowledge follows a two-channel structure: visual and auditory. This theory further advocates a visual and auditory system that is truly independent. Because the channels are independent, information is gathered simultaneously, through the eyes and ears, without the imposition of a burdensome workload.

Therefore, in accordance with the tenants of dual coding theory, people learn at a much deeper level when words and pictures are combined, as opposed to words alone. As such, multimedia is suited to foster the learning that takes place through two-channels. Mayer [21] and other researchers over the years have extended some of these ideas. Drawing upon Paivio’s dual coding theory, Mayer’s model of multimedia learning is shown in Fig. 5. Starting from the left side, the operator receives the Multimedia Presentation, a format composed of pictures, words, and audio input. The signals are then transmitted to Sensory Memory where it is received by the eyes and ears. Next, Sensory Memory transfers the sounds and images gathered through the senses to a verbal and pictorial model of the user’s experience. In turn, the channels are integrated in Working Memory, and the subsequent information is stored in Long-Term Memory.

Fig. 5.
figure 5

(Adapted from: [21]).

A model of multimedia learning as it relates to the act of pressing a soft button on a touch screen.

Full size image

For a more concrete example, Mayer’s model shows how multimedia elements were used to enhance the “Complete” button. The enhanced button is presented in Fig. 6. This button takes advantage of the two-channel structure. Visually, the button utilizes an icon that symbolically directs the operator to go forward. A text label, “Complete”, was also added to supplement the icon image.

Fig. 6.
figure 6

A “Complete” button that’s been enhanced to display the use of words and pictures. (Color figure online)

Full size image

Color further enhances the level of feedback. For instance, the red background change in color (e.g., red to green), after it’s been pressed. The feedback could even be augmented with an auditory cue, a “beep” to further reinforce the amount of feedback.

Obviously, some feedback is better than no feedback at all. But it is important to take a step back and consider the best way to fit feedback in a multimedia framework; and particularly, within a framework that leverages human cognition. Moreover, Mayer’s model of multimedia learning provides a practical way for considering how people’s capabilities can be optimally matched to multimedia elements. Yet, even though the benefits of multimedia in human-system interaction are recognized; Mayer [21] forewarns that too much multimedia can have a negative effect on transfer and retention. Therefore, consider excluding words, pictures, or sounds that are deemed as irrelevant to the multimedia presentation.

4 Discussion and Conclusions

Computer-Based Procedures are starting to emerge as a viable alternative to paper-based procedures. And at least for the near future, it is unlikely that the idea of providing an operator with an explicit set of instructions will be a thing of the past.

Like any new technological application, the user requirements pertaining to the design, development, procurement, and implementation for a computer-based procedure system deserves special consideration. If comprehensive user requirements are in place, there is a good chance that issues, such as those mentioned here, can be avoided. It is also important to point out that user requirements for a computer-based procedure system should not be restricted solely to the user-computer interface. Ultimately, for the project to be successful, user requirements should be included in all elements of the organization, not necessarily at the intersection between user and computer. Otherwise, without a broad comprehensive set of user requirements, project failure at the point of implementation, at the “last mile”, will likely occur [22]. (p. 139).

Finally, comprehensive user requirements are not always sufficient unless the stakeholders having an interest in Computer-Based Procedures are willing to alter their thinking. To alter one’s thinking entails having the flexibility to realize that Computer-Based Procedures are a new idea. As a new idea, it is unrealistic to think this technology should be forced on old trades, traditional ideas, and previously set ways of doing business [23]. While such approaches may have worked at one time for paper-based procedures, they will not work for Computer-Based Procedures. In “The Medium is the Massage,” McLuhan, Fiore, and Agel [24] explained what it would be like to be in the present while refusing to give up on the past, “The past went that-a-way. When faced with a totally new situation, we tend always to attach ourselves to the objects, to the flavor of the most recent past. We look at the present through a rear-view mirror. We march backwards into the future.” [24] (p. 74–75).

There are examples throughout history that have born witness to the “rear-view mirror” analogy. One example happened in the very early days of the motion picture industry. At that time, the art of making a film was drawn from the format for plays performed on stage. The director would fix the camera in the middle front row, to mimic what the view would look like from someone sitting in the audience. Because the position of the camera was fixed, if actors left the stage, the camera continued to be aimed at the vacated scene. At the time, the director did not appreciate how the camera might be used to follow the actors beyond the confines of the stage [25].

From this simple beginning, the craft of making a movie matured from a novel way to record stage plays to the form seen today. Of course, numerous other examples abound. Norman [23] cites other examples where old ideas steadfastly remained in the presence of new emergent technologies. But, as organizations move from paper to paperless, it is essential to recognize that the change forces us to see the world through a different lens. A lens that, at the end of the day, hopefully produces something the users will want.