User-Centered Design of an Intuitive Robot Playback Programming System

. A central part of a robot programming system is its user interface. Oftentimes, its design and development are neglected, although the user interface is the connection between the robot and the user. In this work we present the application of a user-centered design process to a robot programming system interface. The iterative process consists of three steps, namely analysis, design, and evaluation that may be repeated until a desired outcome is obtained. The evaluation shows that with the help of the adapted user-centered design process, it is possible to improve existing user interfaces to become more intuitive and easy to use for non-experts from domains of assembling and construction


Introduction
Usually, programming robots is done by experts by writing source code. This approach is time-consuming and costly and therefore robots do not play a big role in small and medium sized enterprises. To reduce the cost and time needed to program a robot system, a fast and intuitive programming system that can be used by non-experts with little to no programming experience is needed. In [1] we described our fast playback-programming system that is developed for non-experts. Programming the robots with this system is done by manually guiding them along the desired path. Afterwards, the recorded robot program is displayed along a timeline in the user interface. The target group for this programming system consists of non-experts that usually work in domains of assembling and construction. However, the playback-programming system still needs to be improved in terms of usability and intuitiveness. Therefore this work shows an approach towards how to design an intuitive user interface for touch screens and how to evaluate it. Touch screens tend to allow more intuitive interactions than keyboard-mouse combos [2]. Our contribution is the process of how an interface for an intuitive robot programming system that is controlled with a touch screen may be designed. At the beginning we describe our iterative three-stage design process consisting of analysis, design, and evaluation, where the changes to the existing robot programming system are shown. After that we show the application of the design process using the example of playback robot programming. Finally we show the outcomes of the evaluation steps and how the user interface improved compared to the previous version.

Related Work
To get an overview on the field of designing intuitive robot programming systems, we have to take a look at two fields of study. The first part deals with designing interfaces in general, and the second part shows current research in intuitive robot programming systems.
Design is usually an iterative process. This approach can be found in many process models. Four approaches to the design process were selected as examples. All of these approaches relate to a user-centered design process. The individual phases differ greatly in terms of formulations and details. A uniform picture is not recognizable at first glance. The design process as it appears in the German DIN EN ISO 9241-210 standard contains the phases plan the user-centered design process, understand and specify context of use, specify user requirements, produce design solutions, evaluate designs against requirements, and design solution meets user requirements. The design process according to [3] consists of the phases analysis, design, evaluation, and revision. The design process according to [4] contains the phases analyze, design, develop, evaluate, and implement. The Design Thinking process according to [5] contains the phases understand, observe, point of view, ideate, prototype, and test. All these design processes can be generalized to an iterative, three-step design approach that can be used to improve the interface design of a robot programming system. These steps are shown in Figure 1 and are called analysis, design, evaluation. This work does not deal with the implementation step.
After taking a look at existing design processes, we have to look at existing intuitive robot programming systems to specialize our design process on robot programming systems. Recent research is trying to make programming robots as easy as possible by utilizing methods that do not require the users to write source code. There are multiple approaches that try to achieve this. One approach is to use the Programming by Demonstration approach that is either guiding the robot manually through its task [6] or that is learning the robot motion by observing the user with sensors [7]. Other intuitive approaches are to utilize hand held devices [8], smartphones [9], touch screen [10], visual programming in CAD [11], or virtual reality [12] to program robots.
All these approaches have in common that they need some kind of interface to exchange information with the user. Most of the times, a screen is used to interact with the robot, but the design process itself for the development of an intuitive user interface is not shown. Therefore, a need for a design process for robot programming user interfaces can be identified. With the next sections, we want to fill this gap by introducing our design process for robot programming user interfaces.

Design Process for Robot Programming Systems
The research aspect of this paper is to take the proven contents of a user-centered design process and apply them exploratively to the design of an interface for a robot programming system. The goal is to design an intuitively operable interface. For this purpose, the state of the art will be examined in relation to the design process for interfaces. The three phases analysis, design, and evaluation are studied and selected methods are transferred to the robot programming system. The choice of methods is based on the approach of using intuition and personal experience to obtain an optimal result [13].

Analysis
The analysis phase is the first step in the user-centered development of interfaces. In the analysis, the interaction partners, man and machine, are examined. The investigation of the user behavior gives a complete picture of the work tasks, user groups, and working environment [14, p. 39]. One approach of the analysis is the division into goal definition, basic idea, and context of use [15, pp. 177 ff.]. Methods of analysis include field observation, identification of personas, structured, and free interviews [14, pp. 47 ff.]. In the intuitive robot programming system, the context of use consists of the operator, the robot, and the object for handling. In addition, the basics of the theory of intuitiveness are researched in order to work out valid methods within the process.

Design
The design phase deals with the transfer of requirements from the analysis phase to an interaction platform [14, p. 65]. One approach is to divide it into different aspects: screen design, information design, interface design, interaction design, animation design, and sound design [16, pp. 12 ff.]. Methods for designing interfaces can be divided into sketching and prototyping. Methods of sketching are manual sketches, computer-generated layouts, storyboards, and wireframes. Prototypes can be paper prototypes, video prototypes, Wizard-of-Oz prototypes and physical prototypes [17, pp. 115 ff.].
Here, the implementation of the design has the goal to develop an intuitively operable interface. For this purpose, a paper prototype and later an interactive virtual mockup is generated.

Evaluation
The evaluation phase defines test criteria with verifiable parameters for specific tasks. The interface can only be assessed in predefined contexts of use and depending on specific user types [18]. Methods of evaluation are divided into quantitative and qualitative methods, heuristic evaluation, and general usability investigation methods [16, pp. 315 ff]. The prototypes here will be evaluated with methods that measure the intuitiveness of the interface. Since the researched evaluation methods measure the usability of the interface, an adaptation of the evaluation method is necessary here. The MINERIC Toolkit is designed to evaluate robotic systems regarding intuitiveness [19].

Summary
Existing design processes aim to develop an interface that improves its usability. The research object of the robot programming interface intends to develop an interface that improves its intuitiveness. The approach is to use an already developed theory of intuitiveness in the context of human-technology interaction and to explore methods that design an intuitive interface within the adapted design process.

Example: A Playback Programming System
After describing the design process in detail, we verify it by showing its execution on an example. In this example, we want to improve the already existing user interface for playback programming [1] in two iterations.

First Iteration
The first iteration contains the phases analysis, design, and evaluation. In the analysis phase, a research on the state of the art is carried out. This includes the research on programming systems, standards, and on scientific literature to analyze the theory of intuitiveness. In the design phase, insights are incorporated into the interface design, which results in a paper prototype. The paper prototype is evaluated with three usability experts in an expert review.

Analysis
In the analysis phase, various systems from the field of video and audio editing are tested. Systems with and without touch screen functions are used to analyze how elements of a recording timeline can be edited in the systems. The topic of intuitiveness in the context of human-technology interaction was examined by a working group in 2006 [20]. 24 experts for man-machine systems were asked how intuitive work is related to the seven principles of dialogue design, as described in the standard DIN EN ISO 9241-110. The similarity of the principles in connection with intuitiveness was inquired [21]. Design The interface is oriented towards word processing software (e.g., Microsoft Word, Open Office Writer). It is designed in accordance with the standard DIN EN ISO 9241-110. Several dialog principles have been incorporated into the design. Conformity with user expectations was adopted by implementing well-known conventions in the toolbar and icons that correspond to the functions in word processing software. Self-descriptiveness has been implemented by functions that are communicated nonverbally, only icons are shown on the buttons of the toolbar. Suitability for the task was adopted by active buttons that are clearly visible and buttons, like Paste, that appear only when a section of a recorded robot program along a timeline has been copied to the clipboard.
A paper prototype is created for evaluation, which is shown in Figure 2. First modifications are made to the original system for keyboard-mouse combo. The arrangement of the buttons on the toolbar is included. For loops and branch functions, the additional dialog window is replaced by buttons.
The interaction is based on the operation of word processing software with keyboard-mouse combo. In word processing software, a word is marked by clicking on it and dragging the mouse along the word. In the paper prototype, a part of the robot program along the timeline can be marked by touching it with the finger and moving the finger along the recorded timeline. When a section of a robot program is highlighted, the edit functions become active. The interface consists of a toolbar with functions. The functions are divided into file management functions (new file, open file, save file), undo/redo functions, edit functions (copy, paste, cut, delete), and special functions (loops, branches) [1].
When recording a new robot program for a timeline, the interface changes to the recording mode and the robot has to be programmed by manual guiding. The open gripper button is only active when the gripper of the robot is closed and the close gripper button only is active when the gripper of the robot is open. The resulting paper prototype is shown in Figure 2.
Evaluation The evaluation is conducted as an expert review. The two methods Cognitive Walkthrough [22] and Heuristic Evaluation [23] were used for this. It was done with three usability experts who also do this job in their profession. The paper prototype from the design phase was evaluated in aspects of intuitiveness. It was done individually with each expert. First, in a Cognitive Walkthrough, the expert was guided through the individual functions of the system. Three tasks to be solved were described. At the end, every expert had to complete the questionnaire for the heuristic evaluation.

Second Iteration
The second iteration again contains of the phases analysis, design, and evaluation. In the analysis phase, further research on standards and design laws is done. In the design phase, insights are incorporated into the interface design, which result in an interactive virtual mockup. The MINERIC toolkit is used to evaluate the virtual mockup with 20 workers from the assembling domain [19].

Analysis
In the analysis phase, more guidelines from standards and design laws are used. The design of a virtual mockup requires a closer examination of the coloring, shaping, and composition of individual interface elements. Here, the standard DIN EN ISO 9241-110 serves again as a support to meet important criteria of usability.
Design laws play a role in the design of intuitive user interfaces outside the ISO standards [21]. Several design laws can be found in the literature [15]. The law of similarity states that related elements are recognized by similarities. The law of proximity states that spatially close elements are recognized as belonging together.

Design
The suggestions for improvement, which were written down in the expert review, have been incorporated into the design. The interface has also been designed in the second iteration according to DIN EN ISO 9241-110 and design laws of the analysis step.
Conformity with user expectations has been further developed by changing the vocabulary, so that terms from information technology (e.g. capture stimulus) have been changed to everyday terms (capture image). Self-descriptiveness is adopted by designing the record button especially present when a recording is expected. Suitability for the task is implemented by designing all buttons consistently inactive, if the function is not available at the time. In the record mode, the standard interface is heavily dimmed. Law of proximity is taken up by clustering buttons in the toolbar to show that they belong to different topics. Law of similarity is taken up by icons for open gripper and close gripper that look similar in record mode, so they can be recognized as belonging together in the topic, but they differ in colors so they can be distinguished. The interactive virtual mockup is shown in Figure 3. Evaluation During the second iteration of the design process, we conducted a user study according to [19]. The study took place on the 23rd and 24th September 2019 with 20 participants. All participants were workers from the assembling domain aged 21 to 57 with a mean age of M = 38.7 years and a standard deviation SD = 9.7 years. Ahead of the study, each participant saw a short video of the programming system and got an introduction into the interface. Then, each of the three different tasks were explained to the participants. During the study, the participants had to demonstrate the tasks to a scaled, non-automated robot model with the help of the designed interactive virtual mockup. The first task was a pickand-place task with post editing where two red cylinders had to be picked up from a conveyor belt and deposited in the corresponding box, so that both, the recording and the editing of robot programs can be evaluated. The second task was a sensor-based repetition, where objects had to be picked up from the conveyor belt and deposited in the corresponding box, as long as the objects were red cylinders. The third task was a sorting task where objects had to be picked up from a conveyor belt and had to be sorted according to their color. A schematic representation of the tasks is shown in Figure 4.

Evaluation Outcomes
For each of the two iterations of the design process, an evaluation phase has been performed. The first time, an evaluation with usability experts was conducted, and the second time an evaluation with non-experts from the domains of of assembling and construction was conducted.

Evaluation with Usability Experts
In a first evaluation, we used a Cognitive Walkthrough in combination with a Heuristic Evaluation to identify possible improvement points with the help of usability experts. The main comment was, that the usage of the system needs to be consistent. For some functions, sections had to be marked first and then the function was triggered, for other functions it was vice-versa. Another comment was, that the used vocabulary should be adjusted to the target group, so it is understandable for them. The last major comment from the experts was, that functions that are expected to be used next should be placed more prominently within the graphical user interface. With the help of these major comments and some minor notes, the interactive virtual mockup was designed.

Evaluation with Domain Experts
The purpose of this evaluation was to get an intuitiveness rating of the improved interface from non-experts, namely workers from the assembling domain. Since intuitiveness is defined as a combination of effectivity, efficiency, and satisfaction according to the MINERIC toolkit, we used the presented approach to evaluate the prototypes. The detailed evaluation process is described in [19].
The effectivity score is shown in the left chart of Figure 5. It is measured on a scale from 0 and 100. The rating for the designed mockup is M = 69. This indicates a rather effective to use system. Furthermore, the score for each task shows that the effectivity is continously increasing, which indicates that there is a learning effect and it is easier to use the mockup with each task.
The efficiency score is shown in the middle chart of Figure 5. Its value ranges from 0 to 220, where lower scores indicate a better efficiency. The overall rating of M = 39.2 is directly translatable to a little effort to use the interface. This means that it is quite easy to use the designed mockup. A slight increase of the efficiency over the three tasks may be explained with the fact that the last two tasks contained sensor-based loops and branches that are harder to understand.
The satisfaction score is shown in form of the QUESI score and its subscores in the right chart of Figure 5. It is measured with five subscores on a scale from 1 to 5, where 5 means that the participants are fully satisfied with the system and 1 means that they are not satisfied. An overall QUESI score of 3.7 shows that the participants were rather satisfied. The subscores of the QUESI score are subjective mental workload (W), perceived achievement of goals (G), perceived effort of learning (L), familiarity (F), and perceived error rate (E). The score of 4.2 for the perceived achievement of goals shows that the participants were able to achieve their goals, whereas a score of 3.4 for the familiarity shows that the participants were familiar with the interface, but there is room for improvements.
Overall, the three evaluated scores, namely the effectivity, the efficiency and the satisfaction show that with the help of the design process, the interface for the playback programming system was improved and is usable by non-experts working in assembling and construction domains.

Conclusion
To sum up, we applied a three-step user-centered design process to an interface for a robot programming system. With the example application of the design process to improve an existing user interface in the field of playback programming, we have shown that the user-centered design process results in an intuitive user interface that is usable by non-experts from domains of assembling and construction. Limitations are the early stage of research of the theory of intuitiveness for human-robot interaction and the use of a scaled, non-automated robot model for the evaluation. Future work may be done by also taking the implementation phase into account when designing the user interface.