1 Introduction

User-centered design (UCD) becomes increasingly important for product designers and manufacturers since many supplier markets like that of power tools shifted to a buyer market [1]. Thereby, the quality of the products continuously equalizes between the different manufacturers. When the objective quality of the products in terms of the technical functionality is comparably high, the subjective perception of the products becomes increasingly important for the purchase decision [2]. The aim of UCD is to optimize the interaction between the user and a technical system so the user becomes more efficient and effective, and is more satisfied during application. UCD offers great benefits for human-machine systems, where the user is in strong physical interaction with the product. The user is in the flow of power and information with the technical product during the use of such products like power tools [3, 4]. The designer of such products has the aim to increase the perceived usability of a product. The evaluation of the perceived usability is influenced by different usability aspects that depend on a bunch of influences as human factors, product properties and environmental factors [5]. The identification and implementation of the usability aspects in the development process are often challenging, costly, and time-consuming.

In the present state of research, there are already established methods for identifying usability aspects or designing usability studies [6,7,8,9,10]. ISO 16982 summarizes several state-of-the-art methods that can be used for the evaluation of usability [11]. These methods are introduced, and assistance is given for selecting the appropriate methods for usability studies. However, there are no specific instructions guiding designers through the design process to identify the usability aspects relevant to the perceived usability of their products and in setting up a study to evaluate these usability aspects [12]. Besides, several studies exist, which analyze certain usability aspects, such as the ergonomics of different tool handles, vibration comfort, or ease of use, to optimize a product [2, 13,14,15,16,17,18,19,20]. All of these studies are individually designed to analyze a defined usability aspect of a product. It is costly to transfer them to new situations. Lack of a generic study design that structures existing state-of-the-art methods and supports the product designer throughout the process of evaluating perceived usability by being applicable to any human-machine system is a problem [12]. In this way, the effort that would be needed otherwise to adapt the existing methods to a new case, can be reduced considerably. Less obvious usability aspects that can also have a strong influence on the perceived usability of a product are usually neither captured nor identified with these methods [13]. This can currently not be guaranteed by the use of the ISO 16982. A structured processes is needed for an efficient and objective detection of the usability aspects that can be used for UCD-driven development is [12, 21]. Based on an approach by Matthiesen et al., the Usability Study Evaluation Process (USE-Process) is developed by Germann et al. [5, 12]. Using this generic process, product designer should be supported in the evaluation of the perceived usability of a human-machine system. The USE-Process has been developed over many years of power tool testing at IPEK—Institute of Product Engineering at Karlsruhe Institute of Technology by using established state-of-the-art methods and developing them in numerous studies for the evaluation of perceived usability.

For a better understanding of the complex perceived usability Germann et al. introduce the thinking model “Perceived Usability Model” (Fig. 1) which is part of the Usability Study Evaluation Model (USE-Model). The perceived usability of the product depends on many different usability aspects. The combination of all usability aspects together results in the usability perceived by the user. Every usability aspect is influenced by the complex interaction of human factors. In the beginning of the USE-Process, it’s the aim to identify the different usability aspects as well as human factors, product properties and environmental factors that can be structured by the Perceived Usability Model. The information in the model helps afterward for not to miss any relevant influence by the evaluation of the perceived usability.

Fig. 1
figure 1

The Perceived Usability Model: The thinking model by Germann et al. explains how the interation between human factors, product properties, and environmantel factors influancing the usabilty aspects and the perceived usability [5]

Full size image

The USE-Process that guides the product developer through the user centered design process contains 3 main steps consisting of the identification of relevant usability aspects, the subjective evaluation of the product based on the selection of relevant usability aspects, and the quantification of results of this evaluation by using selected measurement technology.

Fig. 2 presents the basic structure of the process, as well as the most important methods used in the individual steps. The USE process can be applied to identify potentials for the new product generation as well as to validate the first prototypes of a new generation. Many consumer products are developed in successive generations. A new development is therefore usually based on an existing product. In addition, there are various reference products on the market with which the new product can be compared.

Fig. 2
figure 2

The Usability Study Evaluation Process (USE-Process): The generic process for identification of relevant usability aspects, evaluation under objective conditions, and quantification [5]

Full size image

However, the state of the research currently lacks evidence of the generic applicability of the process. With the following research question, this paper studies the generic applicability and added value of the USE-Process. The question arises how well this process solves the problems and therefore an experiment was conducted in which a new tool was examined. The experiment investigates how applicable the process is, and how well unknown usability aspects can be detected that are relevant. By evaluating usability aspects under objective conditions and quantifying the influencing factors, such as product characteristics, products can be improved. The quantified usability aspects can then be used as aims for product development.

To answer this question, the process is applied to the evaluation of the perceived usability of a newly designed cordless screwdriver. In Chap. 2, the result, how the USE-Process is applied to the usability evaluation of the cordless screwdriver, and a selection of the results of this usability evaluation is presented to discuss the research question. First of all, a short overview about the USE-Process is given in the following.

2 Application of USE-Process to a codeless screwdriver

The section below shows the application of the USE-Process for the example of cordless screwdriver applications, which address the professional usage of medium to high loads on construction sites. The scenario for the application of the USE-Process to the development process of screwdriver is motivated by an example of a German power tool manufacturing company. The transferability of the generic process to other power tools can be assumed by applying the process to a new power tool. The results of the perceived usability of the cordless screwdriver are used to discuss the impact of the standardized procedure on the objectivity of the perceived usability assessment and the advantage to identify subconscious usability aspects, too.

2.1 Application of step 1 identification—field study to build up knowledge for development of the laboratory study

A field study was carried out on three different construction sites with carpenters, drywall builders, and installers of two different companies for the identification of usability aspects for the perceived usability. The construction sites were selected with the expertise of product developers and sales people from the power tool company. The workers represent the users of cordless screwdrivers. The field study was divided into four phases according to the analyzing process:

2.1.1 General query of user information and working conditions

First, the users are interviewed about the working conditions and typical applications where cordless screwdrivers are used by the enterprise (see Fig. 3). A wide range of applications is identified for cordless screwdrivers of the medium power class. The usage of specialized screwdrivers for applications with different requirements is less common. The testing applications have to cover up that wide range in consequence. Also, the work of the users of cordless screwdrivers scatters over a wide range of applications.

Fig. 3
figure 3

Interview of professional users in the field study—the first step of USE-Process

Full size image

2.1.2 Specific query of the features of each application

The second phase of the field study is the interview of the users about the use of cordless screwdrivers for the individual applications. The interview helps to obtain more information about the different applications and about problems, which the users have when using the cordless screwdriver. Knowledge is gathered about subconscious usability aspects that are important to understand the perception of the cordless screwdriver, the applications, and the way the cordless screwdriver is used by the professional worker. Fig. 4 shows an extract of the general questionnaire that is used for the survey of screwdriver users.

Fig. 4
figure 4

Extract from the general questionnaire for the survey of users on the construction site

Full size image

2.1.3 Spontaneous evaluation of blackened power tools

After phase 2, the professional users are asked to evaluate two different blackened cordless screwdrivers. The users talk about new criteria. They are for example of the cordless screwdriver the noise of the motor and the feeling when pressing the trigger (Fig. 5) without knowing the brands and prices of the power tools.

Fig. 5
figure 5

Comparison of two blackened cordless screwdrivers for an objective evaluation of technical usability aspects by a professional worker

Full size image

2.1.4 Use of power tools for each individual

In the last part of the field study, the worker is recorded with five cameras (Fig. 6) and a microphone while using cordless screwdrivers in different applications. The individual applications are analyzed to reproduce them in the laboratory. An example for the different applications for the cordless screwdriver identified during the field study is given in the following:

  1. 1.

    Connecting roof bars with 200 mm partially threaded woodscrews

  2. 2.

    Mounting of claddings with 160 mm partially threaded woodscrews

  3. 3.

    Drilling in wood for metric screw connections

  4. 4.

    Preparing holes in drywall for electrical installation with hole saws

  5. 5.

    Drilling in metal (thickness 8 mm)

  6. 6.

    Mounting pipes with 30 mm clamps and M4 screws

  7. 7.

    Installation of cable clamps

Fig. 6
figure 6

Installation of cable clamps recorded in the long shot and from the users view (left corner)

Full size image

2.1.5 Selection and specification of the relevant applications

The evaluation of the field study is carried out within the framework of a workshop. Through the collaboration of different experts, an objective evaluation of the use cases and usability aspects can be made, even if it is not possible to rate the data recorded in the field study quantitatively. All applications of the field study are rated by a group of ten product designers and testing engineers in the categories load on the power tool and load on the user. The result is shown in Fig. 7. Five clusters with similar loads for the cordless screwdriver and the user are furthermore identified. The applications screwing woodscrews sized 6 × 160mm and 10 × 220mm, hole sawing in drywalls, drilling in 10 mm steel plates, and step drilling in steel sheets are selected for further testing representing these clusters. Mounting pipe clams for example is not selected, because the load on the power tool and the user is too low to obtain relevant results for the perceived usability of the cordless screwdriver. The laboratory study design for five selected test applications is shown in Fig. 7.

Fig. 7
figure 7

Selection of five test applications for the cordless screwdriver using the Device-Load-User-Load Model after the rating of all 26 relevant applications from the field study by the load on the power tool and the load on the user

Full size image

2.2 Application step 2 evaluation—laboratory study to evaluate the perceived usability of a power tool with adjustable conditions

This section shows the laboratory setup and the experimental design for the subjective evaluation of the usability aspects identified as relevant to the cordless screwdriver during the field study. The results for the usability aspects “torque shock perception” and “perceived overheating” are furthermore presented.

2.2.1 Selection of reference power tools and preparation of the subjects

Two reference power tools are selected for evaluation of the cordless screwdriver. Both are in the same power class and very common for professional work on construction sites.

Ten 19 to 25-year-old male subjects, who are used to analytic power tool evaluation, are trained by a video about the handling of the cordless screwdriver and about usability aspects, which are identified during the field study as important for the perceived usability of a cordless screwdriver.

2.2.2 Creation of the experimental design

The laboratory test is conducted at the Power-Tool-Testcenter at IPEK that is specially designed for manual testing of power tools. The design of the test cases for the five test applications (Table 1) is based on the working conditions at the construction sites seen during the field study. The laboratory study design for five selected test applications is shown in Fig. 8 (left site). Some of the applications are divided into two different slots when the professional workers perform the application in different poses. Fig. 8 shows the different working poses of horizontal work and working overhead for the example of hole sawing (A and B).

Table 1 Timetable for laboratory testing and subjective evaluation. Every subject goes through all test application. In every test case, he/she starts with the two reference power tools before he/she is using the investigated screw driver. After each test application the evaluation is done for each power tool separately
Full size table
Fig. 8
figure 8

The laboratory study for the cordless screwdriver is divided into five selected test applications: screwing woodscrews sized 10 × 220 mm (1) and 6 × 160 mm (2), hole sawing in drywalls (3), drilling in 10 mm steel plates (4), and step drilling in steel sheets (5). For hole sawing, the working poses were also varied: horizontal work (A) and working overhead (B)

Full size image

The sequence of the different applications is described in Fig. 8. The first sequence represents the orientation phase. The subjective evaluation after each application is also documented.

2.2.3 Evaluation scheme for the perceived usability

A query is used for the subjective evaluation of the perceived usability. The questionnaire is filled out after every test case for all three cordless screwdrivers. The query is structured according to the usability aspects that were identified during the field study as relevant for professional workers. The sub-questions for evaluation of the usability aspects are part of the general database at IPEK. Each question can be scored from 1 to 7. The evaluation categories were:

  1. 1.

    ergonomics

  2. 2.

    handling

  3. 3.

    performance

  4. 4.

    preceived power

  5. 5.

    preceived working result

  6. 6.

    sound and vibration

  7. 7.

    versatility

Personal comments on subjective opinions are also possible. This paper focuses on the overheating of the power tool and the torque shocks on the user’s wrist to subsequently describe the results for evaluation of the usability aspects.

2.2.4 Results of the subjective evaluation

To compare the perceived usability of the cordless screwdriver with both reference power tools and to decide about the perceived usability of the cordless screwdriver. The results of the subjective evaluation of the usability aspects and the results of the measurements had to be analyzed and statistically interpreted. The subjective opinions of the subjects can be collected while comparing the three power tools in the first orientation phase by using the think-aloud method. The 148 comments of the subjects are clustered in Fig. 9 into usability aspects and displayed with the frequency at which every cluster is mentioned. The most frequently mentioned usability aspects are the level of the perceived housing temperature and the perceived torque shocks the users feel at their wrists.

Fig. 9
figure 9

Frequency of the usability aspects mentioned by the subjects during the orientation phase of product testing (148 comments)

Full size image

The usability aspects “overheating” and “torque shocks” are considered in the following as an example for the usability aspects that are evaluated in the study for the cordless screwdriver. The boxplots in Figs. 9 and 11 show the results of the subjective evaluation of the usability aspects “overheating” and “torque shocks” by the subjects after every application. The usability aspect “overheating” of screwdriver C is rated significantly better than that of the other two screwdrivers in Fig. 10. No significant statement can be made for the subjectively perceived difference of the torque shocks between the three power tools depending on the similar results and the high scattering of the rating in Fig. 11.

Fig. 10
figure 10

Subjective evaluation of the temperature for the usability aspect ‘overheating’ for the three different screwdrivers by ten subjects for five applications from 1 = cool to 7 = super-hot; In the application “metal 2”, the cordless screwdriver C is rated with 6, thus there is no scattering, only in two cases, it was rated once with 4 and once with 5

Full size image
Fig. 11
figure 11

Subjective evaluation of the usability aspect ‘torque shock’ for the different screwdrivers and applications by ten subjects in five applications from 1 = comfortable to 7 = painful; In the application “drywall”, the cordless screwdriver B is rated with 5, thus there is no scattering, only in two cases, it was rated once with 6 and once with 7

Full size image

2.3 Application of part 3 quantification—selective use of measurement technology to quantify subjective evaluation

The section below shows the usage of a mobile data logger with integrated sensors for the rotational speed to quantify the relevant product properties for the usability aspect “perceived torque shocks” that act on the user and an external pyrometer to quantify the perceived “heating” of the power tool during application. Moreover, the results are compared to the results of the subjective evaluation of the corresponding usability aspects.

2.3.1 Selection of measurement technologies to quantify usability aspects

The temperature of the housing is measured using a manual pyrometer to quantify the difference in overheating between the three cordless screwdrivers. The positions of the three measurement spots near the motor are shown in Fig. 12. A mobile measuring system placed between the cordless screwdriver and the battery pack uses an integrated gyroscope to measure the rotational speed of the cordless screwdriver around the tool bit. The rotational speed of the power tool stresses the wrist of the user and is a measurement parameter for the intensity of the torque shock, e.g. at the end of tightening woodscrews.

Fig. 12
figure 12

Cordless screwdriver with point for temperature measurement and mobile measurement technology and the electronic board for integration of the measurement technology in the power tool

Full size image

2.3.2 Results of temperature and rotational speed measurement

The results of the subjective evaluation are compared with the recorded measurement data to determine correlations between the subjective evaluation by the user and the quantitative product properties of the power tool. The understanding of the relation between the perceived usability aspect displayed in the Perceived Usability Model (Fig. 1) and the product properties supports the product designer how to optimize the usability aspects. The results of the measured product property housing temperature (Fig. 13) confirm the subjectively perceived usability aspect of the overheating behavior between screwdriver C and the other screwdrivers (Fig. 10). In contrast, the maximum values of the rotational speed (product property) in Fig. 13, which is an indicator for the subjectively usability aspect perceived torque shocks, show a high scattering for the different applications similar to the scattering of the subjective evaluation of this usability aspect. It is impossible to make any statements about significant differences between the cordless screwdrivers due to the strong dispersion. The high scattering of rotation speed is one of the reasons why no significant difference in the subjective evaluation of perceived torque shocks exists, even though perceived torque shocks are one of the most important usability aspects identified in the orientation phase of the laboratory tests.

Fig. 13
figure 13

Results of measuring the housing temperature for the comparison to the subjectively perceived overheating behavior of the three types of cordless screwdrivers. Screwdriver C is significantly cooler than the other screwdrivers

Full size image

3 Discussion

The presented paper addresses the problem that a generic process which guides product developers through the cost and time-efficient evaluation of the perceived usability of products like power tools is missing where the user is in the direct flow of power. The state-of-the-art methods for evaluating the perceived usability, which is necessary for the success in user-centered design (UCD) in this sector, are often specialized for one single product or usability aspect. Adjustments that are necessary to transfer the specialized methods from the original to another case are the major obstacles to save time and money. The generic Usability Study Evaluation Process (USE-Process), for the identification of usability aspects, evaluation of the usability aspects, and the quantification of the product properties by measurement technology was developed to overcome this problem. It is based on the state of the research and on own iterative studies and can be applied to any power tool. The process is suitable for the individual use of perceived usability evaluation on different power tools due to its generic nature and the application to different power tools in preliminary studies. The transferability of the USE-Process to any new power tool is now investigated using the example of a study on cordless screwdrivers. Extensive impressions of the test persons are recorded and evaluated in a standardized form through the standardized application of methods such as the think-aloud-method or the use of questionnaires. Only those use cases identified as relevant need to be replicated in the laboratory as concrete test cases through the structured evaluation and selection of applications from the field study. This allows the development of more targeted laboratory studies to save time and effort. Automated evaluation options can be implemented through the possibility of reusability and standardization, which significantly reduce the evaluation effort of the studies and increase the added value of the usability evaluation for the product designer.

In addition to a significant reduction in effort, the process also aims to record the opinions of the subjects in a structured and objective way so that they can be used for product design. The relevant usability aspects for the user are analyzed in the field study to objectively evaluate the perceived usability of a product. In contrast to methods where usability aspects are determined in advance, the USE-Process ensures that all usability aspects relevant to an objective evaluation of the perceived usability are considered. The use of structured questionnaires and the execution of the perceived usability evaluation in a laboratory study under controlled conditions helps to reduce the influence of disturbing factors such as different workplace conditions or the different brand perceptions by the subjects. The evaluation can be performed with a relatively small number of subjects.

By quantifying the subjective evaluation with measurement technology, an understanding of the relationship between the evaluation of a usability aspect and the expression of a product property can be established. The statement about the perceived usability of the power tool can be verified with objective values this way. The measurement technique support by interpreting the results of the subjective evaluation, as in the subjective evaluation of the torque shocks during the fastening process. They help to decide whether any scatter in the evaluation of a usability aspect depends on the uncertainty of the test person’s evaluation or on the system itself. A complete evaluation of the perceived usability by measurement technology is not conceivable at this time due to the complex interaction of the different usability aspects. It is therefore necessary to combine the subjective evaluation by the query with the objective results of measurements.

The aim of a generically applicable process to evaluate usability aspects can be achieved with the USE-Process, however, overall the following limitations have to be made. The importance of the usability aspects and the test environments need to be evaluated individually dependent on the product and its users and can therefore not be standardized. Despite the high standardization in the subjective evaluation of the perceived usability and the use of measurement technology, disturbance variables coming from the user or the device can influence the result of the perceived usability evaluation. The selection of unified measurement technology is complex. Therefore, the quantification of the subjective evaluation of usability aspects can only be carried out for usability aspects which are of particular importance for the perceived usability of the product or which occur frequently and can be recorded in a standardized way. The use of measurement technology cannot replace the subjective evaluation of the perceived usability by the subjects due to the complex interactions between the usability aspects. The ability to objectify the evaluation and to reduce the scattering makes a target-oriented use of measurement technology necessary. Both steps should be conducted therefore in the process.

The study about the applicability of the USE process for the user-centered development of a cordless screwdriver has shown the advantages of using this generic process when the human is in the power flow of the machine. But there still exist limitations of the process. A hardware prototype of the power tool is needed for the USE-Process as it is presented in Germann at al. [5]. Also, two reference systems are necessary for the physical testing in the laboratory study. This can be a problem at an early stage of product development, when just a virtual prototype is availed. Further, the process is specially designed for products when the human is in the power flow of the machine. It would be interesting to see how far this process can be transferred to the user-centered development of products in which humans are primarily involved in the information flow with the machine, such as interfaces for operating machines or the software design of apps, and whether the user-centered development of these products can also be improved as a result. This question cannot be answered with the present results and requires further investigation. It may be possible to perform the laboratory study on virtual prototypes in a semi-virtual validation environment. The first step of the USE process would still be useful and necessary to define which product features are represented in the semi-virtual environment and how the evaluation of the prototype in the semi-virtual environment should be designed to simulate the evaluation behavior of the addressed users.

4 Conclusion

The development of user-centered systems, e.g. of power tools, that interact strongly with the user is a challenging task. Many usability aspects have to be considered during the development to increase the system’s perceived usability due to the very versatile and complex use of power tools. In this contribution, the generic USE-Process is presented, that has the goal of reducing the cost- and time-intensive process steps in the user-centered development of power tools. The presented process uses already existing and profiled methods and develops a generic and structured approach by an iterative adaptation and combination of different power tools. The USE-Process is divided into a field study in which usability aspects that are relevant to the perceived usability are identified. Based on the applications observed in the field study, steps are presented to set up a test environment that allows a reproducible and objective evaluation of the perceived usability and thus the possibility to improve the development of power tools. In the last step, the results of the subjective evaluation are quantified by selected measurement technology.

The generic process offers the following benefits relative to the current state of research:

  1. 1.

    The structured process of the field study ensures the identification of all relevant usability aspects, which must be considered in the UCD of a power tool.

  2. 2.

    Its generic structure makes the evaluation of the perceived usability easier and more efficient since the effort for adaptation to a specific product is significantly reduced.

  3. 3.

    The standardized procedure for subjective evaluation of perceived usability and the quantification of the results through the use of measurement techniques makes the perceived usability evaluation more objective and allows reducing the number of subjects for the perceived usability evaluation that leads to a further reduction of effort.

  4. 4.

    Quantification of the subjective evaluation through measurement technology helps the product designer to understand the connections between the results of individual usability aspects and the product properties of a product.