Keywords

1 Introduction

Trying to identify what digital medical technologies are today is a practically unresolvable task. Over the last few years, we have seen a radical change in these technologies; they have become not only extremely sophisticated and complex but also capable of maintaining evolving relationships with their users.

It is obvious that the benefits of this evolution are absolutely valuable, but several issues are linked to the evolutionary lines of medical devices and are often related to usability. In fact, the usability of digital medical devices (i.e., the level of effectiveness and efficiency provided by the device in interactive environments) and the level of user satisfaction in certain contexts of use [1] are definitely not satisfactory at present. Users of medical technology are among the most diverse, including doctors, nurses, technicians, administrators, and patients themselves or their families. Each type of user has different and sometimes conflicting needs, skills, and traits. The tools themselves can be so different that combining them under a single label seems excessively summary as well as inappropriate. We could define the term “medical device” to refer once and for all, say, to devices without any relational quality beyond those impressed on them by their designers (e.g., a scalpel). But the notion of a medical device also encompasses e-health records, robots that assist or take the place of human operators, and applications that inform and help patients, thus creating a dialogue between patients and healthcare professionals; often these systems create ecosystems in which, in addition to one or more human counterparts, they interact with each other.

On closer inspection, the problem of usability of medical devices cannot be tackled in isolation and must concern managerial and administrative aspects of the entire healthcare system. However, within this complexity, the design of each medical device must be tailored to the specific activities, objectives, competencies, and skills of those they assist. Just as the considerations relating to their evaluation both in the development phase and during their actual use are indispensable. Given the complexity of the socio-technical systems that are responsible for providing health services, the insufficient usability of medical devices is one of the major problems affecting the efficiency of the entire system and, more importantly, the health of the patient [2, 3]. Many studies in recent years have highlighted problems related to the usability of medical devices that have resulted in negative effects for the health of patients [4]. This paper will focus on a few studies in particular to shed some light on the nature of the problems and their consequences.

2 Some Studies on Medical Devices

Errors related to the use of medical devices can lead to serious damage to the patient and represent a frequent and sometimes silent cause of death. Numerous studies on errors in medicine clearly show a direct link between usability problems and errors, patient readmission, and injuries [5, 6]. As an example, it is worth noting that adverse events related to patient-controlled analgesia were reported almost 2500 times between 2003 and 2004. This figure seems even worse when we consider the fact that over 9000 adverse events occurred in the 5-year period from 2000 to 2005 [7, 8]. These errors are often attributable to bad interface design: cognitive ergonomics guidelines are very often violated though they could be beneficially applied to the design of healthcare systems and medical devices in order to minimize the chance of adverse events.

Nowadays, electronic medical devices have spread to all hospital and extra-hospital environments, including operating rooms, emergency rooms, radiology departments, laboratories, emergency vehicles, intensive care units, and even homes. The development process and the performance of these devices are affected by an infinite number of variables that are not always considered and whose consequences are not always foreseen. Among these variables it seems obvious to consider that the pervasive use of these instruments is compromised by sound interference, low lighting, reflections, and electrical interference. However, it is less commonly appreciated that operators, even experienced ones, are forced to continuously update their operational skills and develop new automatisms [9]. Performance can be further limited by stress, fatigue, and the application of incorrect procedures. This is compounded by the wide range of devices that the operator uses over the course of a day. The physical, sensory, and mental capacities of health workers are variables that must be taken into account, especially since the use of equipment increases considerably from year to year.

Devices can only be used safely and effectively when the user, the activity, the context of use, the stress and fatigue levels, and the interactions between all these factors have been adequately evaluated during the design process. Keeping in mind the complexity of the factors involved, Schaeffer et al. [10] have studied the consequences of using two different models of insulin infusion pumps for patients. In their study, they linked the design principles of human factors with those of user interface design. More specifically, the authors correlated the programming errors of the instrument during data entry with different error categories, considering parameters such as blood glucose level or amount of carbohydrates. As a result of their evaluation, they were able to point out that, if certain developmental inefficiencies in the usability of the instrument had remained unresolved, adverse events could have occurred due to the release of inappropriate amounts of insulin, potentially having very serious consequences.

Other aspects relevant to the design of medical devices include population aging and consequent changes in users, such as reduced sensory capacities (e.g., decreased vision or hearing) and motor and cognitive abilities. Maša Isaković et al. [11] investigated elderly patients’ ease in using a glucose self-monitoring application not specifically designed with them in mind. Their analysis showed that the most important points when designing an application for the elderly are the very basic aspects of the user interface: the size, visibility, and comprehensibility of buttons and symbols. For this purpose, the various distorting and illusory effects of perceptual processes should also be taken into account [12, 13]. Furthermore, the age-related deterioration of memory and a lack of familiarity with technology can cause user confusion and therefore lead to error. Although making applications suitable for all users would be a very difficult if not impossible task, creating different profiles for different accessibility groups seems to be the best performing solution. In this case, the involvement of users from the earliest stages of design has been shown to be essential.

In development teams, collaboration between people with different skills is fundamental. The study conducted by Fairbanks et al. [14] on defibrillators highlighted the importance of collaboration between clinicians and human factor experts. This study has pointed out that, although the collaborative design and the ergonomic evaluation of the user interface are well-established practices in the aeronautical and nuclear industry, most medical devices are not tested in all the contexts in which they will be used and with all future users. Defibrillators are perhaps the medical devices most exposed to critical situations. Problems such as the presence of too many buttons in a small area, the need for excessive scrolling to get to the desired setting, and lack of feedback in an emergency can be lethal. Whereas the market today offers many choices, an ergonomic approach should also guarantee the simplification of the organization of the health system, increasing the quality, efficiency, and safety of care. Managers and experts should consider medical devices that are truly functional for clinical-assistance activities and that aid workers in carrying out their daily tasks with greater satisfaction and effectiveness, with the added benefit of consequent savings in terms of a reduced need for training.

One of the aspects of evaluation studies that can prove to be difficult is the recruitment of subjects. Healthcare personnel, doctors, and nurses are often overworked professionals, which makes it difficult to involve them in evaluation circumstances beyond their working environments. To overcome this obstacle, Bond et al. [15] had evaluation sessions take place during a conference attended by end users, allowing direct interaction with the medical devices they wanted to evaluate. More specifically, their evaluations concerned one software application, a medical diagnostic tool (ECG viewer), and a medical research tool (electrode misplacement simulator). From their study, interesting possibilities emerge for conducting evaluations in a way that does not involve taking the end users inside laboratories, but instead finding large gatherings of end users and taking the opportunity to have end users interact with the device at their convenience.

Another study is particularly interesting for at least two reasons: it is a longitudinal analysis that was carried out in two different phases over 7 years, and concerns users who are sometimes neglected in usability studies, that is, radiology technicians. In this study [16], six radiology technicians were observed while performing radiological examinations (i.e., Computed Tomography) as part of their usual activities in a hospital emergency department. The authors considered variables such as the occurrence of errors and the time needed to conduct various aspects of the examination. The same technicians were interviewed about their thoughts on the diagnostic system they were using regarding aspects such as ease of use, cognitive effort required, and the possibility of it leading to errors. The results of this analysis highlighted several problems related to the usability of the system, both in itself and as part of a broader operational context. Most notably, it became apparent over the years that all noticeable improvements, such as a reduction in the time needed to carry out the examination, were in fact attributable to an increase in the skill of the operators. The system updates made between the first and second evaluations had not led to any improvement in usability.

3 Beneficiaries of Usable Medical Devices

Direct and indirect users of medical devices have different needs and characteristics. To varying degrees, any individual can take part in the improvement of device usability and effectiveness for their particular context and needs. In any case, however different the needs may be, the following objectives are universal when improving usability:

  • To reduce the number of accidents and deaths due to incorrect, ineffective, or inappropriate use of medical devices.

  • To improve the ease of use for users as well as users’ well-being.

  • To comply with current regulations and cultural needs of the various countries.

Based on these considerations, Wiklund et al. [17] stated that ensuring adequate usability of medical instruments is as much a moral imperative as a path to economic benefits (see Fig. 32.1).

Fig. 32.1
figure 1

The main beneficiaries of the increased levels of usability of medical instruments and their benefits

Full size image

Efforts to ensure adequate usability must involve all beneficiaries. Each user group is a vector of specific needs and requirements and is called to participate in the evolutionary development of medical devices.

4 Usability Evaluation

Various government authorities (see, for example, the FDA’s webpageFootnote 1) have recommended that the activities and processes necessary to ensure the usability of medical devices should be divided into (a) processes relating to devices not yet on the market and (b) processes relating to products already in use.

For devices not yet on the market, the indications, essentially addressed to the manufacturing companies, guide the preparation of a final report that clearly expresses the results achieved regarding the usability of the product.

Therefore, the report must contain clearly and in detail:

  • Any conclusions reached during evaluation.

  • A description of the device’s target users, applications, and contexts, and any training necessary for its operation.

  • A description of the user interface.

  • Any problems highlighted during evaluation.

  • An analysis of the risks and dangers associated with the use of the device.

  • A summary of preliminary results.

  • A description and categorization of tasks along with their critical aspects.

  • The details of the evaluation, based on the principles of human factors.

Following the release of the product on the market, issues may arise such as potentially or actually dangerous circumstances, errors during use, and improper use, even if the product was developed following correct usability guidelines.Footnote 2 In this case, the priority becomes the clear communication of the event to the manufacturer of the device so that the device’s usability may be improved further. Here also, the FDA recommends the preparation of a report describing with as much precision as possible the context in which the adverse event occurred. In particular, the following information is required:

  • The device type, manufacturer, brand, and lot number if applicable.

  • The exact location of the event.

  • If a patient or device operator was harmed.

  • If there was a problem with the device itself, such as a defect, malfunction, and break.

  • Whether someone was directly operating the device at the time of the event, and if so, who.

  • If there were other therapies being used on the patient at the time of the event that may have caused or contributed to the event.

Evidently, the FDA’s statements have as their conceptual basis user-centered design (UCD) [18] (see Fig. 32.2). UCD considers the user its focal point, rather than technology. In the realization of a product, the specific needs, traits, and limitations of the end user are the most important functional needs. To this end, development must be carried out at different levels by multidisciplinary teams of marketing representatives, developers, technicians, and usability experts. The goal of UCD is to understand what users really want and need in order to produce interactive tools that foster a productive and satisfactory dialogue between user and device.

Fig. 32.2
figure 2

The process of user-centered design (UCD) in which, starting from user and market analysis, iterative evaluation cycles, with the user as the focal point, are carried out before the product launch

Full size image

What this team is specifically called to do can vary depending on many factors, from the company’s positioning to how many products the company has already put on the market, from how innovative the product is to the type of target users and whether that type is more or less precisely defined. In “The invisible computer” [19], Donald Norman states that it is extremely important to perform a preliminary assessment of user needs and then move on to a market analysis. This sequence of analysis is already itself an iterative cycle, since the market analysis should serve to re-evaluate and reframe users’ needs in terms of their definitive requirements and characteristics. Based on the results obtained, a series of iterative cycles is initiated, towards the specification of the product (concept design), its realization in the form of prototypes which gradually grow closer and closer to the final product (implementation), and experimentation on the usability of the product in the field before the product is definitively released (product launch).

Each of these cycles aims to make user-device interaction more fluid, responsive, and error-free. Any difficulty encountered by the user poses a challenge to form new design hypotheses leading to alternative design solutions. Ultimately, UCD aims to maximize the usability of a product. ISO 9241 [1] defines usability as the “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” To achieve adequate usability, designers must be able to respond adequately to these questions:

  • Effectiveness: does the product support the user in achieving their objective?

  • Efficiency: does the product allow the user to achieve their goals without unnecessary effort, without wasting time, and without generating mistakes?

  • Satisfaction: does the use of the product represent a positive experience for the user, meeting not only instrumental requirements but also cognitive and emotional needs?

As defined, these questions must be answered in the affirmative, taking into account the specified users and the specified contexts of use.

4.1 Methods for Usability Assessment

The methods that have been developed over the years for the assessment of usability can be divided into two main categories: those carried out exclusively by experts and those involving a variable number of the end users. The methods carried out exclusively by experts are observational or inspectional in nature, while those involving end users, whether conducted in the laboratory or in the field, almost always consist of having the users perform tasks with the device in order to measure the distance between what the users perceive and how they interpret it, as well as between intention and action. The time and resources available are very often the main factors in the choice of methods used for the collection of end user requirements. For this reason, the most pragmatic and rapid methods are described below, since they allow the collection of large amounts of data in a reasonably short time while minimizing costs, possibly by leveraging several methodologies at once. The methods that involve expert analysis include cognitive task analysis, heuristic evaluation, cognitive walkthrough, and the Delphi Method. One method that involves both experts and end users is contextual inquiry. Users, on the other hand, are usually involved in usability tests and focus groups.

  • Cognitive task analysis

    Cognitive Task Analysis is a technique—more accurately a set of techniques—which examines the cognitive nature of tasks by analyzing and breaking down the activities into specific actions, identifying their relative frequency and difficulty [20]. A fundamental part of the technique is the elicitation of the experts’ own knowledge, even tacit knowledge. This type of analysis allows the designers to create a practical hierarchy of interventions and therefore plan the redesign of the system around the most critical elements.

  • Heuristic evaluation

    Heuristic evaluation is a low-cost technique that works well for medical devices and can generally detect the main problems in a rather short time. It can also be very useful in cases where contextual observation cannot be applied or where confidentiality may be restrictive.

    The technique is usually based on guidelines or heuristics which are used by experts to inspect the technology under evaluation [21,22,23]. The guidelines have been refined over the years in the field of cognitive ergonomics. The application of this technique usually requires at least two experts to independently explore all components of the system and detect any violations of the appropriate guidelines, noting the severity of each violation. The detection of violations may also provide indications for the redesign of the system.

  • Cognitive walkthrough

    Like heuristic evaluation, cognitive walkthrough is entrusted to experienced staff and not to end users [24]. Often following task analysis, a heuristic evaluation requires an expert to complete a series of tasks to test the system’s comprehensibility and learning curve for a novice user. Essentially, in situations where the availability of end users and financial resources proves restrictive, experts may leverage their theoretical and practical knowledge of usability issues to evaluate a product. Another advantage of this method is that the evaluator’s sensitivity eliminates the bias often found in less experienced users who attribute shortcomings in performance to some weakness in the device rather than their lack of experience.

  • Delphi Method

    After identifying all user-related requirements for a product, it may be difficult to set priorities. The Delphi method is by definition an iterated sequence of steps with the aim of ensuring that opinions converge as iterations progress [25]. Starting with fairly generic questions, experts discuss relevant issues and exchange opinions which can then be cross-referenced to provide the basis for a second session of deeper elaborations. From time to time, a summary of the previous session’s discussion can be provided to the experts. In the third and usually last meeting, a series of questions are asked in order to arrive at a conclusive consensus on the importance of the different factors considered.

  • Contextual inquiry

    Contextual inquiry is a particularly pragmatic technique suitable for the design of a new product or the redesign of an existing one. The activity of the expert tends to focus on the user: the expert asks the users questions about their experience as they interact with the device, such as questions about what is happening and why, and any possible improvements to the system [26]. Designer and user collaborate in this way in an almost symbiotic relationship to uncover information essential for product development. To implement this methodology in highly complex healthcare contexts it is often necessary to use simulations of real scenarios to avoid situations that could be dangerous for operators and patients.

  • Usability test

    Usability tests conducted with users can identify and quantify different variables such as the frequency and type of errors made by users when interacting with a device, the time needed to complete different tasks, and the frequency and nature of requests for support [27, 28]. Whenever possible, these tests are conducted in the laboratory and involve a limited number of users that are representative of end users in general. In preparation for the execution of the test, it is important to establish the preliminary activity of selecting the tasks the user must try to perform and the techniques for recording the user’s activity. In addition to highlighting critical issues of which the user is normally unaware, usability tests are also commonly chosen to collect information on the user’s experience, satisfaction, and opinions on variables such as the attractiveness and the perceived usefulness of the system.

  • Focus groups

    Focus groups are widely used, not only in healthcare but in all contexts that adopt a user-centered approach. A focus group consists of one (or more) experimenter(s)/facilitator(s) and a group of participants representing the end users [29]. The fundamental component of its success is the ability of the facilitator to animate the discussion among the group of people in search of useful ideas for the evaluation and/or development of a device. For focus groups to work it is essential that the facilitator follows simple rules such as providing everyone with clear, basic information about the purpose of the meeting, using terminology that is easily understandable by all, listening respectfully, and employing a strong dose of empathy.

    Adequate tools (e.g., notes, audio recordings, video recordings) must be integrated into the realization of a focus group to keep track of all the information, whether implicit or explicit, provided by the participants.

4.2 The Usability Assessments in Reality

To date, there are no clear indications on which methodologies to adopt in order to maximize the amount of information gained from the evaluation of medical equipment. Very often the path followed by evaluators is the one that appears to be the most informative given the limitations of the context within which evaluations are to be carried out [30]. In order to identify the path usually taken by the various researchers involved in the evaluation of medical devices, Campoe [31] conducted a meta-analysis on relevant studies published between 1993 and 2012. Following preliminary research, she identified 886 papers that, after several rounds of refinement, were reduced to only 18. The analysis of these 18 papers showed that half of the studies on the usability of medical devices adopted only one analytical method which was either based on heuristics or involved end users. The other half adopted an approach integrating two or more methods.

The results of this review are not always methodologically sound. For example, the users had not been described in any detail: in most studies, only the demographics of the users were described, and in six of the studies not even those were considered. However, even when studies did report more information on user characteristics such as their level of professional experience, they did not use these variables to account for the results [31].

The task descriptions did not fare much better. In fact, only six studies reported in some detail the tasks used during the evaluation, relating them to the complexity, duration, and frequency of the operational process. However, the major weakness found in these studies lies in their lack of explanation of the reference environments used to evaluate tools that are often used in direct conjunction with other devices or that may have varying uses as determined by the operational context. The analysis by Campoe [31] showed that 11 studies reported the kind of environment in which the evaluation test took place (i.e., the hospital, operating room, etc.), but only 2 described the physical characteristics of the environment in which the device would be actually used. Complete negligence was found regarding the treatment of technical, social, and cultural characteristics of the reference environments: none of the 18 studies treated these characteristics as relevant factors for the usability of the device under evaluation.

5 Conclusion

The solution to this sort of methodological anarchy among usability studies cannot come from the enforcement of rules, codes, laws, or compliance with standards. The medical device ecosystem is in fact undergoing continuous and rapid evolution, which often produces solutions that exceed the development goals contemplated by rules and standards. In fact, regulatory systems can often steer the development of medical devices towards solutions that have little to do with everyday practice or even hamper day-to-day operations [32].

Therefore, many aspects of the processes that ensure the usability of medical devices need to be reformulated or reframed in an absolutely innovative way. In this regard, Vincent and colleagues [32] have argued that, with respect to several innovations in the medical field, it is not useful to reference pre-existing norms, standards, or well-established evaluation procedures. For example, 3D printers enable the rapid production of precisely shaped components. But there are obvious difficulties regarding the quality control of the components produced. If we then consider the usability of personal, mobile medical devices, it is clear that the standards developed so far do not effectively guide the realization of products to really meet the needs of each user. Finally, the issue of ensuring usability can be further exemplified by the fact that the increasingly ubiquitous smartphone applications, including various health-related applications that are now within each individual’s reach, can receive formal certifications and approvals not on the basis of their real content, but mostly of how they are presented.

Technological advancement, as far as medical instruments are concerned, opens up obvious and serious problems, difficulties that clearly pose a challenge that has yet to be solved. However, the most innovative medical devices are also those that promise hitherto unexpected possibilities and benefits for the health of patients and broaden the horizon for the development of novel methods that will ensure proper usability [32].