Keywords

1 Introduction

In order to integrate and certify a direct human-robot collaboration (HRC), four types of operations are permissible according to [1, 2]. The paper at hand deals with the research on the collaborative operations speed and separation monitoring (SSM) as well as power and force limiting (PFL). With the ISO/TS 15066 [3], system integrators are supported by easy to use principles for the estimation of the dynamic separation distance in SSM and the configuration of permissible robot speeds in PFL. However, problems arise in the practical implementation of these two operations. In SSM, the safety zones are often oversized due to different simplifications such as a punctual and constant directional operator speed as well as conservative values for the robot stopping distances. While PFL on the other hand allows the closest level of interaction, it also defines the most stringent safety requirements. Thereby, the permissible robot speeds depend on the affected characteristic body areas. If no exact determination of these body areas can be performed during the process, the practical implementation of PFL usually avoids larger translatory robot motions at the height of the human head and limitations of the maximum Cartesian speed to 250 mm/s by a time consuming path planning. In both types of operations the mentioned circumstances can lead to cycle time losses and wasted space in the respective production system.

1.1 Related Work

The separation distance \(S_p\) to be maintained applying the SSM depends especially on the motion behaviour of the operator and robot [4]. Applying an extended path planning strategy, advantages can be achieved compared to the classic human-robot coexistence, since the robot motion can be dynamically adjusted to its environment [5,6,7,8,9]. Despite this, it is noticeable that the motion behavior of the human operator is either often neglected or facilitated. However a simplified consideration of the operator has been examined in some approaches [7, 10, 11]. Nevertheless, there are no suitable concepts for the differentiation and localization of individual body regions and the use of the acquired status information for an adaptive robot path planning. This results in the loss of usable workspace due to conventional assumptions such as a constant operator speed of 1.6 m/s. In conclusion it can be established that none of these approaches consider the complete human musculoskeletal system within an HRC environment. According to the PFL specification, the forces and pressures acting on the operator in the event of a collision must not exceed the biomechanical thresholds of the affected body areas according to [3]. Numerous studies have been carried out in this regard to correlate factors such as maximum forces or stresses with respect to the onset of injuries and pain perception [12,13,14,15]. In a more recent study, new pressure thresholds were determined and assessed with regard to a revision of the ISO/TS 15066 [16]. A Risk reduction can be achieved either by safety control systems or inherently safety features on the robot. In recent years, scientific work has focused on safety-related implementations as well as the investigation of collision consequences and influencing values (e.g. effective masses) in case of contact [14, 17,18,19,20]. Nonetheless, the PFL method is still in a research stadium and up to now rarely suitable for industrial practice. A main reason for this are the non-applicable areas for transient contact scenarios according to [3]. This complicates the design of safe and efficient HRC scenarios as well as the certification process. Based on the related work, a need for action can be derived with respect to an extended human modeling within the SSM and PFL type.

1.2 Novelty and Objective

The paper at hand deals with the development and investigation of an extended human modeling and the kinematic state consideration of specific body areas. This approach will be embedded into a new hybrid collaborative operation (HCO), which combines the SSM and PFL method [27]. In order to estimate the motion data we will apply a previously developed HRC simulation tool. The main objectives are the reduction of safety zones and the increase of productivity within HRC systems under the early consideration of relevant safety requirements. Section 2 introduces the applied simulation tool and provides an overview of the HCO method. A new approach for the kinematic state determination of the specific body areas based on a digital human body model is discussed in Sect. 3. The evaluation of this approach is carried out by means of a designed and simulated HRC scenario in Sect. 4. Section 5 summarizes the results of this paper and discusses future research topics.

2 Preliminary Work

2.1 Simulation Tool for Human-Robot Collaboration

Subsequently, an HRC planning and simulation tool, which has been developed within the BMBF research project KoMPI (fund number 02P15A060), will be applied for the consideration of the human motion behaviour and the kinematic state estimation of single body regions. Using this tool, both human operators and robots can be analyzed in a collaborative assembly environment and evaluated in terms of ergonomics and automation aspects. The underlying system architecture consists of two main components: A robot and peripheral simulation, and a human and process simulation [21]. The extended concept is based on the integration of evaluation functionalities for the field of automation technology (e.g. collision-free robot path planning) into the proprietary software solution ema Work Designer (emaWD) of imk automotive GmbH [22, 23]. emaWD enables a temporal evaluation and ergonomic analysis of the designed production processes. A central part of emaWD is a realistic simulation and visualization of human body motions, whereby a digital human model (DHM) can be parameterized and controlled via a simple and intuitive task-based process modeling [24]. The part of the robot and peripheral simulation is based on the Robot Operating System (ROS) [25]. ROS is widely used in robot research and enables a holistic simulation of robot systems, sensors and the environment.

Figure 1 shows the concept of the data transmission between the emaWD-ROS interface (EMA_ROS_PLUGIN) and ROS itselfs by means of the animated human motion in emaWD. Hereby, the human model is reduced to \(n_{br}=18\) body regions. A /humanPublisher-Node collects the appropriate information about the individual and time-dependent kinematic poses in a user defined ROS message Human.msg and published it on the /kompi/human topic. In the kompi_interface package this message will be received and interpreted by a corresponding callback function. By using the incoming data and simplified geometrical primitives, an approximated human motion is then reconstructed on the planning_scene of MoveIt! [26]. Hence, this dynamic environment can be considered in the estimation of a collision-free robot path.

Fig. 1
figure 1

Transmission of virtual human motion data in the HRC simulation tool

Full size image
Fig. 2
figure 2

Kinematic Relation between robot’s TCP and a specific body area k

Full size image

2.2 Method of a Hybrid Collaborative Operation

The simulation based description of the human motion behaviour, discussed in this paper, will be embedded in a new hybrid collaborative operation method (HCO) for HRC applications. Assuming a pre-defined robot trajectory, the main objective of the HCO is to achieve a safety-oriented and efficient configuration of the Cartesian robot speed [27]. With respect to the specified safety strategies of SSM, HCO involves a pure speed adaption and no alternative path planning. Thereby, the robot system must comply with the normative requirements of the PFL and be equipped with an external sensor system for workspace monitoring according to the SSM. The procedure of the HCO method is divided into three stages, which are briefly described as follows:

  1. 1.

    Kinematic state determination: The kinematic states of the interaction partners are determined at the tool center point (TCP) of the robot (assumption in this paper) and \(n_k \in \mathbb {N}\) characteristic body areas k of the operator (cf. Fig. 2). They include the local position vectors \(\boldsymbol{r}_{r} \in \mathbb {R}^{3\times 1}\) and \(\boldsymbol{r}_{h,k} \in \mathbb {R}^{3\times 1}\) as well as linear velocities \(\boldsymbol{v}_{r} \in \mathbb {R}^{3\times 1}\) and \(\boldsymbol{v}_{h,k} \in \mathbb {R}^{3\times 1}\). Based on the kinematic positions, the respective collision vectors \(\boldsymbol{r}_{c}\) and actual distances \(S_{k}\) between the operator and robot can be formulated as:

    $$\begin{aligned} S_{k} = \left\| \boldsymbol{r}_{c}\right\| = \left\| \boldsymbol{r}_{h,k} - \boldsymbol{r}_{r}\right\| . \end{aligned}$$
    (1)

    For the sake of clarity, the index k does not occur in the further steps unless they have to be used explicitly. A key aspect in the SSM and PFL method is the description of the absolute directional speeds

    $$\begin{aligned} v_{rc}= \boldsymbol{v}_{r}\, \frac{\boldsymbol{r}_{c}}{\left| \boldsymbol{r}_{c}\right| }, \quad v_{hc}= - \boldsymbol{v}_{h}\, \frac{\boldsymbol{r}_{c}}{\left| \boldsymbol{r}_{c}\right| } \end{aligned}$$
    (2)

    and the collision angles

    $$\begin{aligned} \varphi _{rc} = \cos ^{-1}\left( \frac{\boldsymbol{v}_{r}\, \boldsymbol{r}_{c}}{{\left| \boldsymbol{v}_{r}\right| \, \left| \boldsymbol{r}_{c}\right| }}\right) ,\, \varphi _{hc} = \cos ^{-1}\left( \frac{\boldsymbol{v}_{h}\, \boldsymbol{r}_{c}}{{\left| \boldsymbol{v}_{h}\right| \, \left| \boldsymbol{r}_{c}\right| }}\right) \end{aligned}$$
    (3)

    between both interaction partners [6, 8, 9].

  2. 2.

    Safety measurement according to SSM: The second stage of the HCO includes the determination of the current protective separation distance

    $$\begin{aligned} S_{p}(t_0) = \underbrace{v_{hc}\, (T_{r} + T_{s}(v_{rc}))}_{S_h} + \underbrace{v_{rc}\, T_{r}}_{S_r} + S_s(v_{rc}) + S_{m} \end{aligned}$$
    (4)

    according to [3] as well as the proof of the required safety condition \(S \ge S_p\), direction of motions via \(\left\{ \varphi _{hc}, \varphi _{rc}\right\} \) and relative speed \(v_{rel}=v_{rc}+ v_{hc}\) between the human operator and robot. With regard to Eq. (4), \(T_r\) denotes the reaction time of the robot system. \(S_m\) includes the inaccuracy of the applied monitoring system in the sense of a constant minimum distance. The variables \(T_{s}(v_{rc})\) and \(S_s(v_{rc})\) describe a speed-dependent braking behaviour of the robot in Cartesian space regarding category stop 1 (cf. [1, Annex A]). In case all required conditions \(S \ge S_p\), \(\varphi _{rc} \ge \pi /2\) and \(v_{rel} \le 0\) are violated, a reduced robot speed \(\widetilde{v}_{rc}(S,S_p(t_0))\) is desired to maintain the separation distance according to the SSM type (cf. [8]).

  3. 3.

    Integration of PFL Requirements: In the third stage of the HCO method, the SSM will be combined with the safety-related requirements of the PFL. If a reduction of the robot speed should be performed in stage 2 (\(0<\widetilde{v}_{rc}<v_{rc}\)), a remaining potential collision speed \(v_{c} = v_{rc}- \widetilde{v}_{rc}\) is specified, which will be compared with the permissible collision speed \(v_{p,k}\) according to the transient contact model of the PFL [3]. Hence, a modification of the original robot speed would only be required within the HCO method if \(v_c > v_{p,k}\) applies. Thereby, \(v_{p,k}\) is decisively influenced by the maximum permissible transfer energy and the biomechanical properties (effective mass, stiffness) of the concerned specific body area k, which usually changes with a high probability during the collaboration scenario.

For further details of the HCO method, please refer to [27]. In context of the this method, the present paper deals with the determination and usage of the kinematic human states based on virtual motion data. These informations are necessary in order to firstly get a knowledge about the critical and hazard body areas, secondly to determine the local safety distances \(S_{p,k}\) and thirdly to obtain the body area-dependent permissible collision speed over the entire process.

3 Kinematic States of Human Body Areas

As part of the virtual acquisition of human motion data and the kinematic state determination, the DHM ema and the human-specific subtasks created by the planner are applied. In this regard, Sect. 2.1 briefly summarizes the export of the human behaviour simulated in emaWD via the developed ROS interface EMA_ROS_PLUGIN and the reconstruction of the kinematic states within the robot and peripheral simulation. The two parameters poses and velocities of the transmitted message Human.msg provide the spatial positions \(\boldsymbol{r}_{br}\in \mathbb {R}^{3\times 1}\) and rotations \(\boldsymbol{R}_{br}\in \mathbb {R}^{3\times 3}\) as well as the linear and angular velocities \(\boldsymbol{v}_{br}\in \mathbb {R}^{3\times 1}\) and \(\boldsymbol{\omega }_{br}\in \mathbb {R}^{3\times 1}\) of the individual body regions. In the next step, the geometric relationship between these body regions br and the specific body areas k according to [3] has to be described in a formulation.

Table 1 Assignment of the specific body areas k according to [3] to the body regions br of the reduced EMA model (l = left; r = right)
Full size table

The assignments of the body areas k to the respective body regions br are listed in Table 1. For the sake of clarity, the listing is limited to the body’s head, center and the upper limbs. In addition to the designation of the ema body regions, the local displacements \(\boldsymbol{r}_{k}^{br}\in \mathbb {R}^{3\times 1}\) of the body areas with respect to the parent coordinate system \(\mathcal {K}_{br}\) are given. The Cartesian coordinates of \(\boldsymbol{r}_{k}^{br}\) were determined geometrically using the underlying ISO/TS 15066 specification and the human model type 50th percentile, male of the applied simulation tool. With the transformation of \(\boldsymbol{r}_{k}^{br}\) into the global frame \(\mathcal {K}_w\), the spatial position of the specific body area k is given by

$$\begin{aligned} \boldsymbol{r}_{h,k} = \boldsymbol{r}_{br} + \boldsymbol{R}_{br}\,\boldsymbol{r}_{k}^{br}. \end{aligned}$$
(5)

In addition, considering the linear velocity \(\boldsymbol{v}_{br}\) and the rotational velocity \(\boldsymbol{\omega }_{br}\) in the origin of \(\mathcal {K}_{br}\), the following relation applies with Euler’s derivation rule:

$$\begin{aligned} \boldsymbol{v}_{h,k} = \boldsymbol{v}_{br} + \boldsymbol{\omega }_{br} \times \boldsymbol{r}_{k}^{br}. \end{aligned}$$
(6)

4 Evaluation

Afterwards the human motion consideration within the HCO method is carried out by means of an exemplary HRC scenario. In the scenario, the TCP of a Kuka LBR iiwa 14 R820 moves along a pre-defined circular path \(\boldsymbol{x}(t)\) (cf. Fig. 3). An extended investigation of the entire kinematic chain has been done in a primary work [8]. The characteristics of the SSM measurement and the influence of the extended human modeling discussed in this paper is summarized in Fig. 4.

Fig. 3
figure 3

Simulated motion of the human operator and the Kuka LBR iiwa 14

Full size image
Fig. 4
figure 4

Safety distance characteristic and influence due to the human modeling

Full size image

First of all, Fig. 4a illustrates the current distances \(S_k\) and dynamic safety distances \(S_{p,k}\) according to Eq. (4) of four selected body areas. The operator initially approaches to the robot’s workspace and then walks around the circular and contrary trajectory of the TCP. So a reduction of the actual distances can be be noticed in the beginning of this scenario. After 3.04 s, the minimum distance is reached at 0.57 m by means of the left shoulder joint (\(k=6\)). Moreover, the operator penetrates the safety zone in the segment 1.55 s \(<t<\) 3.71 s with at least one body area. The partial oscillation of \(S_{p,k}\) can be explained by the periodically varying speed \(v_{h,k}\) during the operator’s motion. This effect is intensified especially in the upper limbs (see e.q. \(k=16\)). Figure 4b shows the variation of the critical body area \(k_{cr}\) for the proposed scenario. This parameter results from the minimum time-to-collision value TTC between the several body areas and the TCP. For further details please refer to [8]. Even within this short process and the selection of only four body areas, a strong shift of \(k_{cr}\) can be recognised. This emphasizes the high relevance for the reproduction of the entire human musculoskeletal structure regarding the SSM method. Furthermore, the extended human modeling will also effect on the permissible collision speed \(v_p\) according to PFL, since whose characteristics strongly depend on the properties and thresholds of the respective body areas. With Fig. 4a and b, the left arm nerv (\(k=16\)) appears to be the most critical area with a share of 57.55 % over the hole process. Accordingly, this area will primarily be observed for the next point. In Fig. 4c the composition of \(S_{p,16}\) due to the individual terms of Eq. (4) can be seen. Beside the required minimum distance \(S_m\), the speed-dependent courses of \(S_r\), \(S_h\) and \(S_s\) are apparent. It can be stated that these terms may become negative, since the direction of motion of the interaction partners is considered. This is significant regarding the objective of a close cooperation and the minimization of the separation distance in a HRC environment. Nevertheless, \(S_p\) must not be less than \(S_m\) according to [3]. From the displayed courses it can be noticed, that \(S_{h}\) has got the highest influence on \(S_p\) with an average of 0.48 m (75.61 %) and a maximum value of 1.66 m at \(t=1.1\) s (95.35 %). In contrast, the terms \(S_r\) and \(S_s\) resulting from the robot’s responsiveness and stopping behaviour have a comparatively small influence, with an average of \(\bar{S}_r=0.05\) m (10.35 %) and \(\bar{S}_s=0.098\) m (14.04 %).

Following this, the part of \(S_{h,k}\) both in the conventional interpretation with \(\bar{v}_{hc}=1.6\) m as well as in the speed-dependent variant of the body areas are shown in Fig. 4d. According to Eq. (4) the reaction time \(T_r\) and the stopping time \(T_s(v_r)\) affect the value of \(S_{h,k}\) besides the actual operator speed \(v_{hc}\). Due to the proportionality of the stopping values to the actual robot speed, a non-linear course can be seen even in the conventional interpretation. Thereby, a maximum value is reached at \(t=1.71\) s. When comparing the courses of the different body areas, the obvious potential of a possible distance reduction can be recognized, as long as the local velocities \(v_{hc,k}\) of the operator are taken into account. In this scenario an average reduction of 0.88 m which ranges from a minimum of -0.54 m (increase) to a maximum of 1.8 m could be achieved. It should be noted that an exact modeling and consideration of the human motion can also lead to an increase of the dynamic separation distance. This can be seen for example in the time interval 0.97 s \(<t<\) 1.25 s.

5 Conclusion and Future Work

The paper at hand deals with the investigation of a new concept for the modeling and consideration of a human body motion within HRC environments. A central objective is to assist planning engineers in the design stage of HRC system development. Moreover it is intended to improve hybrid production scenarios between human operators and robots under the consideration of the normative safety conditions. In an exemplary HRC scenario different key factors in the calculation of the separation distance (according to SSM) has been investigated. Especially the extended consideration of the human motion data was evaluated. The results illustrate appropriate potentials for the reduction of oversized safety zones compared to the conventional operating methods. The kinematic state determination of the human body areas, as the first stage of the HCO method, has been integrated into the ROS based planning and simulation tool (cf. Sect. 2.1). Due to the ROS-Industrial initiative, real control interfaces of various industrial robots are provided by different vendors. In this context, future work will include a further evaluation of the HCO method using different robot systems and interaction scenarios. This will be carried out in the learning and research factory of the chair of production systems.