Advertisement

Logistics Research

, 9:23 | Cite as

A methodology to develop collaborative robotic cyber physical systems for production environments

  • Azfar Khalid
  • Pierre Kirisci
  • Zied Ghrairi
  • Klaus-Dieter Thoben
  • Jürgen Pannek
Open Access
Original Paper
Part of the following topical collections:
  1. Dynamics in Logistics: Digital Technologies and Related Management Methods

Abstract

The paper identifies the need for human robot collaboration for conventional light weight and heavy payload robots in future manufacturing environment. An overview of state of the art for these types of robots shows that there exists no solution for human robot collaboration. Here, we consider cyber physical systems, which are based on human worker participation as an integrated role in addition to its basic components. First, the paper identifies the collaborative schemes and a formal grading system is formulated based on four performance indicators. A detailed sensor catalog is established for one of the collaboration schemes, and performance indices are computed with various sensors. This study reveals an assessment of best and worst possible ranges of performance indices that are useful in the categorization of collaboration levels. To illustrate a possible solution, a hypothetical industrial scenario is discussed in a production environment. Generalizing this approach, a design methodology is developed for such human robot collaborative environments for various industrial scenarios to enable solution implementation.

Keywords

Cyber physical system Human robot collaboration Collaborative robotics 

1 Introduction

The manufacturing horizon for Industry 4.0 [1] comprises a paradigm shift from the automated manufacturing toward an intelligent manufacturing concept. The exclusive feature in Industry 4.0 is to fulfill the real-time customer demand of variations in products in a very small lot size. This will enable a manufacturing system to meet individual customer requirement without wasting time for setup and for re-configuration of an assembly line. The intelligent manufacturing implementation may take place though the concept of internet of things (IoT) [2], in which each participating component has a specific IP address. Due to the availability of big data in IoT, the manufacturing system characteristics can be predicted precisely like predictive maintenance, robustness in product design and adaptive logistics. In this context, the smart manufacturing setup or a smart factory [3, 4] and logistics system have to fulfill the mass customization [5] demand in a flexible manner.

For a smart robotic factory to work in the context of Industry 4.0, high productivity and flexibility is the demand of the future. To cope with this issue, robots may take most of the workshare in future manufacturing, yet the human worker has to stay in the work area either in supervision role or for the jobs for which the robots cannot be trained. The constant human presence in or near the work area of intelligent robot leads to a shift regarding safety. The conventional approach is to expose human workers up to a limited extent to the robot and with appropriate safety control that leads to full stoppage (safe hold) of a machine in case of worker violation of the robot workspace. This causes interruptions and resetting procedures to be activated which reduces productivity. The futuristic approach is to implement robotic applications where robot and human workers can coexist and collaborate safely. In this setting, the robots share the same workspace with human counterparts and perform activities like raw material handling, assembly and industrial goods transfer.

Due to the presence of more than one million conventional (non-collaborative) working robots in the industry [6], converting the present day conventional robots to collaborative ones presents a lot of revenue potential. These conventional robots cannot be replaced with new collaborative robots (see Table 1) in manufacturing areas because of the huge financial cost involved. One approach to convert these conventional robots into collaborative ones is by making their environment intelligent, e.g., by putting sensors around the robot working area in addition to the capturing of human worker motion. This way, multiple conventional robots will be able to collaborate with humans. This will be an advantage for the manufacturers as capital investment on newly developed collaborative robots may not be required. To establish such a collaborative environment, a cyber physical system (CPS) needs to be established which takes care of all the necessary requirements of communication, safety, security, sensors and electronics. This will also allow even very large payload robots to carry out the tasks in a collaborative manner as is the case in the small to medium payload robots shown in Table 1. In one such attempt in MIT [7], the human motion capturing sensors are used with a non-collaborative robot. The virtual component resembling the actual scenario of man and robot is used to calculate the distance between the robot and the human. Based on the real-time distance calculation, the robot controller is given the task by an external module to systematically reduce the speed. This way, a generalized solution is sought to make a conventional robot intelligent.

2 State of the art in collaborative robotics

The state of the art development in collaborative robotics has roots in the technologies arriving from the humanoid robotics, artificial intelligence and exoskeletons, which were developed over the last two decades. The basic objective of such robotic humanoids is to work in household and medical applications to attend the needs of disabled and old people. In the industrial domain, there is only a recent trend for the development of intelligent collaborative robots. Table 1 shows many examples for such collaborative robots that can work alongside humans without creating hazardous situations. So far, the collaborative robotics is developing fast in industry and it is estimated that the collaborative robotics sector will grow to US$1 billion by 2020 [6, 8]. This growth is driven by small to medium manufacturing, electronic manufacturing and allied services provider companies. For industries looking for such agile manufacturing technologies, robot manufacturers develop collaborative robot designs which are suited for small- to medium-size product handling and other operations.
Table 1

State of the art collaborative robots

Robot

Application area

Specifications

Main sensors

Capabilities

ABB Switzerland, Yumi—IRB 14000 [9]

Mobile phone, electronics and small parts assembly lines

Payload—0.5 kg

Reach—559 mm

Repeatability—0.02 mm

Foot print size—399 mm × 497 mm

Weight—38 kg

Velocity—1500 mm/s

Acceleration—11 m/s2

Camera-based object tracking

Collision detection through force sensor in joint

Dual arm body

Pause motion upon collision

Action resumption only by human through remote control

Collision free path for each arm

Rethink Robotics, Boston, USA, Sawyer [10, 11]

Machine tending, circuit board testing, material handling, packaging, kitting etc.

Payload—4 kg

Reach—1260 mm

Repeatability—±0.1 mm

Weight—19 kg

Camera in wrist

Wide view camera in head

High-resolution force sensors embedded at each joint

Force-limited compliant arm

Seven DOF single arm robot

Touch screen on the main column for instructions

Context-based robot learning

Universal Robots, Denmark, U10 robot [12]

Packaging, palletizing, assembly and pick and place etc.

Payload—10 kg

Reach—1300 mm

Weight—28.9 kg

Velocity—1000 mm/s

Repeatability—±0.1 mm

Foot print size—Ø190 mm

Force sensors embedded in joints

Speed reduction is directly programmed

Six DOF in single arm

Collision detection

Robot stops upon collision

Speed reduction to 20% on workspace violation

NASA, USA, Robonaut 2 [13]

International Space Station, space robotics

Payload—9 kg

Reach—2438 mm

Weight—150 kg

Velocity—2100 mm/s

Finger grasping force—2.3 kg

Stereo vision camera

Infrared camera

High-resolution auxiliary cameras

Miniaturized six-axis load cells

Force sensing in joints

Dual arms with complete hands and fingers

Each arm has seven DOF

Each finger has three DOF

Elastic joints

KUKA, Germany, LBR iiwa 14 R820 [14]

Machine tending, palletizing, handling, fastening, measuring

Payload—14 kg

Reach—820 mm

Weight—30 kg

Repeatability—±0.15 mm

Torque sensors in all axis

Force sensors in joints

Contact detection capability

Reduction in velocity and force upon collision

Single arm robot with seven axis

In Table 1, multiple examples show dexterous robots comprising of single or dual arms that have multiple degrees of freedom (DOF). In most of the cases, the tool end-effector repeatability shows the capabilities of modern collaborative robots to handle intricate tasks. All of these robots can work and collide gently with humans on the factory floor as the joints are developed with internal force sensors. The arms and heads are equipped with high-resolution cameras, even 3D cameras for tracking. In some cases [15, 16, 17], visual markers are used for fast recognition and tracking, on every tool which are needed to the robot to complete the job. All the robots have programmable compliance, such that they can be trained for the new job on the shop floor. Yet, the maximum payload capacity varies from 0.5 to 14 kg, i.e., small- to medium-sized payload. Collision detection, instant hold upon collision and speed reduction upon violation of workspace are the common implemented technology features. It seems that there is a paradigm shift in the role of robots in industry and services from conventional unintelligent robots to collaborative ones. Also, these recent developments range from small- to medium-scale payload applications in human–robot collaboration (HRC), paving the way for heavy robots to become collaborative as a next step in industrial collaborative robotics. A very recent example is of FANUC’s CR-35iA [18] capable of carrying 35 kg payload with category 3, performance level (PL) (d) safety certification, according to ISO 10218-1:2011.

The paper has two basic objectives: The first aim is to identify the collaborative schemes and formulate a formal grading system; secondly, to define a CPS for human–robot collaboration in industrial scenarios and develop a methodology that can search for appropriate solutions in a given industrial scenario down to sensor level. The latter allows us to convert conventional heavy payload robots to intelligent ones for any industrial setup. Further detailed considerations for an equipped external environment for such robots are derived from pre-defined safe CPS according to the scenario requirement and collaboration level sought. The approach is initiated by studying implemented robot safety schemes and then evolving effective collaboration schemes. Once the collaborative schemes are sorted, some key indicators are introduced for formal categorization of industrial collaborative scenarios with examples of few selected sensors. A hypothetical collaborative example is presented to identify the sensor level requirements for a given industrial scenario. The paper is summarized with a design methodology for the development of such CPS in the context of variation in industrial scenarios.

3 CPS in human robot collaboration

The proposed approach is to exhibit safe intermediate HRC without passive safety mechanisms (e.g., fencing). In order to realize this, extra safety and protection measures need to be implemented for a collaborative robotic CPS (CRCPS). These safety and security (protection) requirements are based on the level of interaction between humans and robots on the shop floor to increase productivity. Security is moreover closely related to safety as both these system level properties have to be considered concurrently. Security essentially protects the systems from humans as attackers and the safety physically protects humans from the systems (e.g., avoiding collisions). In fact, the approach in the design of CRCPS is to merge the safety and security concerns just like designing industrial facility, control and risk assessment that consider both aspects [19]. However, in this paper, only the safety aspects are considered for CRCPS development because security can be studied in this specific case only once a safe HRC system is ensured. Security is left as the future direction of current research on CRCPS development to secure a ‘safe HRC system’ from the cyber-attacks.

A CPS is a smart system in which the computational and physical systems are integrated to control and sense the changing state of real-world variables [20]. The success of such CPS relies on the sensor network and communication technologies that are reliable, safe and secure. In CPS, all the functional components are in modules and interconnected (wirelessly) in the production line or in the smart factory. Even raw materials and machines are connected to the network cooperating with human workers through human–machine interaction (HMI) systems. Hence, the CPS platform evolves its architecture to engineer across the digital–physical divide and removing the borders among the key technologies. In particular, the CPS for manufacturing and production [21, 22, 23, 24, 25, 26, 27, 28, 29] may consist of electronics, computing, communications, sensing, actuation or robot, embedded systems and sensor networks. The CPS in manufacturing needs other resources like flexibility of the manufacturing system, the manufacturing scenario and the adaptability of changing assembly tasks [30], in addition to HMI technologies and other typical CPS modules. For the application in HRC, the deployment of a full scale CPS accounts for the human worker as an inherent part of the system. To state the CRCPS definition, the three components are clearly evident in the model with detailed adaptor modules (see Fig. 1). The CRCPS structure is inspired by anthropocentric CPS (ACPS) [16, 29, 31], mainly due to the cohesion of the human as an inherent module.
Fig. 1

Structure of CRCPS: detailed components, modules, adaptor technology modules and interconnected links

The human component (HC), the physical component (PC) and the computational component (CC) represent the three main integrated entities. The interaction among the three entities depends upon the advent of the enabling adaptor technologies. The HC is well connected through different adaptor technologies, e.g., accurate human position tracking technology is essential adaptor in the CRCPS. The CRCPS is a highly automated system as it removes the boundaries between the composite elements, thus preferring their operational interactions. There are various HMI technologies based on human senses of vision, acoustics and haptics. The proposed CRCPS can utilize vision system for detection, tracking and gesture recognition of human workers. The robots can also be commanded using acoustic signals from humans (e.g., voice control). Additionally, a variety of sensors and actuators can provide the interaction between HC, CC and PC. There are standard interactions shown between the components which have to contribute with a role. Adaptor technologies are scenario dependent and can be seen as plug and play devices. There are other optional scenario-dependent interactions between the standard components and adaptors in CRCPS.

The CRCPS is an extension of the CPS and for that reason must show compliance to the system level properties of a CPS. For this, CRCPS must exhibit properties like integrality, sociability, locality and irreversibility. Moreover, it must be adaptive, autonomous and highly automated [32]. Integrality for CRCPS means that its functional components are well integrated to perform self-organizing tasks like learning and adaptation. The ability of CPS to interact with other CPS through different communication technologies defines the sociability. It will encompass not only devices but also integrates humans as well. As an example, if the two CRCPS are functioning in a close physical distance, then the worker belonging to a CRCPS must be able to interact safely with the robot that belongs to the other CRCPS. Locality introduces the computational, human and physical capabilities of a CPS, as bounded by spatial properties of the environment. Irreversibility of the CPS makes it self-referential in timescale and state-space. The adaptive characteristic makes the system self-organized and evolving. The autonomy [16] refers to the roles of functional components and the CPS itself as capable to make independent decisions.

4 Collaboration classification

For CRCPS industrial environment, a smooth overlapping of workspace zones of robots and humans is considered in which both can interact. The formal grading of the human–robot collaboration involves the level of interaction between the two entities. The level of interaction can be formalized based on the distance between the two entities, workspace share level and the complexity of collaborative tasks which both are performing mutually. Many human avoidance schemes based on human activity prediction or human and robot position estimation at the same time [33, 34, 35], risk prediction control [36] and augmented reality [37] are considered to be implementable in an interactive environment. There are also fatalities reported [38] in countries where usage of robots is intensive despite putting all the safety and protection protocols. For example, in Germany, such accidents range from 3 to 15 annually from 2005 to 2012. Note that this rate relates to accidents without any collaboration between humans and robots. There is also an issue of mental strain on humans in addition to the physical interaction of robot and human. It is discussed by Arai et al. [39] that by restricting the moving area and moving speed of robots, the mental strain of a human operator remains low. Also, the prior accurate information of robot motion is essential to decrease the strain on a human operator. In this context, there is general need to classify the collaboration level and specific to heavy payloads, it is obligatory to reduce the level of risk in HRC.

To formally grade the HRC, the safety approaches in practice must be known first. All the examples shown in Table 1 follow at least one safety approach during human robot interaction. Safety schemes based on position prediction and building intelligent environment [40] around robots are summarized. The intelligent environment means to equip the robot environment with appropriate monitoring sensors to make it aware of situation, human, safety zone and distance. However, the four basic principles of safety protection of working with robots are described in [41, 42]. Here, these approaches are outlined briefly.

A common approach using small size robots is to provide guidance manually or reduce the robot speed as per requirement. This manual approach is open loop, without sensing, has high HRC level, is restricted to small size robots and depends on the defined risk assessment. The basic safety approach can be termed as ‘complete isolation’. In this approach, a specified work zone is covered with sensors like laser scanner or proximity sensor. In this case, the robots must stop at the human access to the work area. These systems are sensor dependent, closed loop and have almost no HRC level attainment (see Fig. 2 for collaboration schemes).
Fig. 2

Collaboration classifications: a robot on safe hold against human violation, b speed reduction if the worker is in the robot work zone, c robot touching the human with a pre-defined calibrated force

The third approach is the speed and separation monitoring through vision-based systems or other possible techniques. Speed reduction schemes of robot can be applied with a possible stop or speed reduction in case of worker enters the dangerous zone. This safety concept uses multiple integrated sensors and an effective sensor fusion technique to develop a fast, reliable real-time monitoring solution for HRC. High HRC level attainment is possible but poses challenges to the risk assessment in case of a failure of a monitoring function. The speed monitoring can be integrated with separation monitoring, in which human avoidance algorithms are used in a dynamic human tracking context. A small active area around the human position is marked and continuously updated for the human motion in the robot work zone, forcing the robot to actively avoid such a space. The last concept is the force monitoring through the use of force sensors. This system will also work with the help of a vision field which will guide the robot in case of a human presence. The robot speed and acceleration reduction will take place according to the level of force allowed to hit a specific part of the worker’s body. This scheme demands integration of force sensors in addition to the sensor technology required for basic area monitoring. The scheme provides highest level of HRC attainment but poses a challenge to the risk assessment in case of failure of any monitoring function.

By looking at different collaboration techniques, it is possible to categorize these by several parameters. Figure 3 shows the collaboration level from low to high. There are four equally weighted key performance indicators (KPIs) selected to contribute in the overall HRC grading scheme. These indices are PL, safety distance (SD), risk (R) and the reaction time (RT). PL is taken as the ‘mean time to dangerous failure’ (MTTFd) and defined in the EN ISO 13849-1 based on the average number of cycles per year until 10% of the components have a dangerous failure.
$${\text{MTTF}}_{\text{d}} = \frac{1}{10}E(x)\quad {\text{for}}\,x = \hbox{min} \{ P({\text{failure}}\,{\text{of part}} \cdot {\text{cycle/year}}) = 1\}$$
(1)
E is the mean time until 10% of the components have a dangerous failure or the component operating time is restricted to E. The units of this indicator are expressed in years, e.g., the MTTFd range for electromechanical components is 100–200 years. This means that the component needs replacement after 10% of the MTTFd value. A period of 20 years as a component replacement time is set as a goal according to the standard and can be taken as the maximum value for this indicator.
Fig. 3

HRC grading scheme: four KPIs on the left and grading calculation is on the right side

The second indicator is the SD calculated between human and a working robot. The SD formula for a human working with an industrial robot is given in EN ISO 13855.
$${\text{SD}} = (K \cdot T) + C$$
(2)

SD computes the minimum SD from the risk zone. K is the speed of the man approaching to collision with the robot (mm/s). T is the robot’s follow-up time in (s) to stop completely, once the brakes are applied. C is the additional distance (mm) for safety compliance that depends on the sensor’s capability or resolution. In case of multiple sensors used in a system, the sensor with lowest resolution can decide the resolution of the overall system if any sensor fusion technique is not used. Calculations with various sensors show that SD = 0.5 m is not possible even with very fast sensors (see Table 3). Yet, no human robot collaboration can be implemented if SD is larger than approximately 2 m.

The 3rd indicator is calculated based on the manufacturer specifications according to the number of unsafe components used and is termed as risk (R).
$$R = \frac{U}{U + S}$$
(3)

The ratio of the number of unsafe components (U) divided by the total number of components is referred as risk (see Eq. 3), where S is the number of safe components used in the system. According to the PL range specified above, i.e., electromechanical component below 100 years MTTFd, is considered unsafe. Additionally, each sensor product itself may be specified based on the number count of safe and unsafe components used. In a CRCPS perspective, all the used sensor components and equipment can be marked safe or unsafe. The minimum risk can be specified when all the components used are safe. The maximum risk can be checked according to a benchmark or left on the designer’s disposal or risk assessment based on the ISO 12100:2010.

The fourth index to gauge the effectiveness in collaboration is through the data delay rate (D i ) of the sensors (see Eq. 4). The diversity of sensors used in a designed CRCPS may have an asynchronous data transmission rates. Data delay rates (ms) are important as the delay time from every sensor counts on the overall system’s RT to respond in a case where any sensor fusion technique is not used. Thus, the overall delay time of the system is the key indicator, enabling the robot to initiate the safety protocol in time to avoid any hazard. Larger delay time can affect the robot’s RT adversely and hence reduce the effective HRC attainment. The other variable is the number of sensors (N) installed in a CPS system. In the case, the system consists of a number of heterogeneous sensors, this variable represents the number count of slowest sensors. Here, k is a constant. It is zero for a completely isolated systems and one for all other monitored systems.
$${\text{RT}} = k(D_{i} \cdot N)$$
(4)

After looking at different collaboration techniques and the performance indicators, we now formalize the collaboration grading scheme. Figure 3 shows the grading pattern of HRC from low to high. On the right side in Fig. 3, the HRC grades are specified, where a1 shows the highest level of HRC attainment and d 2 the lowest.

All of the above-mentioned indicators are converted to the corresponding indices on the scale of 0–1 (divide by the best KPI value).
$$I_{j} = \frac{{{\text{KPI}}_{j} }}{{\left( {{\text{KPI}}_{j} } \right)_{b} }}$$
(5)

In case of SDI and RTI, inverse scale is used as the best values are the smallest, e.g., 0.5 m for SD calculation is very difficult to achieve. On the right side in Fig. 3, the collaboration grading is specified based on the sum of all the four indices with equal weights, resulting in a maximum score of 4. Here, ‘a’, ‘b’, ‘c’ and ‘d’ correspond to a scale of 3–4, 2–3, 1–2 and 0–1, respectively. This way, the collaboration attainment is divided into four large categories, where each category is comprised of two sub-categories.

5 Sensors catalog

To assess the HRC attainment level, it is necessary to compute all the KPIs for a given collaboration context. For this purpose, the safety schemes and the possible risk-reduction approach mentioned above are further explained at the sensor level. The HRC schemes are studied to incorporate sensor level requirements of the CRCPS and generate a sensor catalog for each type of collaboration. The sensor catalog is a sensor library that can be established with various sensors of diverse specifications and can be integrated in the design methodology of the CRCPS. This catalog together with performance indicators forms the basis of an optimization algorithm to generate a list of possible feasible solutions for any given industrial scenario. It may also reveal nonexistence of any feasible solution. One of the basic conditions for CRCPS implementation is the known positions of human and robot in real time. In some cases, it is also important to know the extents of the assembly and the scenario for the operation.

Scenarios are the possible situations in which an industrial process can take place, e.g., a large automobile engine held by the robot gripper is presented to the worker for an industrial process like quality inspection, drilling, seal adhesion, fastening. [43]. In any given scenario, real-time location information of body parts of the worker is important. For example, a motion sensor installed on an arm can give real-time information about the arm position. Yet, if the position information of worker hand is not included, the estimation of assembly size and worker hand size must be taken into account. A different example is of vision sensors employed for the position information of human worker which must be workable in different lighting conditions, e.g., in low visibility or in a rough industrial environment. Similarly, the communication must be fast enough for an immediate and accurate response of the robot which exemplarily could be the case for a low distance, safe wireless network. Overall, the system must comply the relevant safety standards like EN ISO 13849-Part 1 and 2 and EN ISO 13855. These standards provide principles, safety requirements and guidance for the design and integration of safety-related parts of control systems.

Table 2 defines the collaboration level of different safety approaches, employable risk-reduction schemes and the basic sensor pack currently available, to implement a safety concept. The solutions can be found based on the industrial scenario and HRC level sought. For the speed and separation monitoring case, inertial measurement units (IMU) are employed in addition to the basic area and position monitoring sensor systems. Active human avoidance algorithms are part of the solutions in addition to the applied sensors. Similarly for force-monitoring-based HRC system, the basic area and position monitoring will be a requirement for implementation of the CRCPS in addition to the force sensors. In force monitoring, different types of geometry adapted tactile sensors are available to be installed at the robot joints, with shock-absorbing properties for safe collision detection and touch-based interaction. Force sensors of different force ranges can be used for assessment of force exposure limits for different human body organs. However, use of force sensors in robot joints is a new trend in collaborative robotics as shown in Table 1.
Table 2

Collaboration concepts and required technologies

Collaboration concepts

Collaboration level

Risk-reduction approach

Technology (sensors employed)

Manual operation

High HRC but for small robots only

Physical ergonomics based assessment

No sensors, passive protection guards

Complete isolation

Robot stoppage on workspace violation

HRC: 0

Robot workspace or path calibration

Laser scanner, proximity sensor, light curtain

Speed and separation monitoring

High-level interaction

Robot workspace calibration

Robot speed calibration

Separation distance calibration

External instructions to robot controller

Cameras, IMU

Human avoidance algorithm

Force monitoring

High-level interaction

Force calibration

Force monitoring: force sensors, torque sensors, load cells

Table 3 shows the computation of some indices that are checked for different employable sensors in the CRCPS. While these data were obtained for specific sensors only, it may still be regarded to hold similarity for sensors of these classes. The number of sensors is selected according to the practical requirement for such a system. For example, to check the worker entry into the robot workspace, only one laser scanner is required. In order to monitor the worker position through a vision system, a minimum of two cameras is needed for full field coverage. Moreover, to design a worker vest, a total of four IMU`s are required at minimum to cover the body front, back and arms.
Table 3

Indices computation for sensors: security laser scanner, time of flight camera, motion tracking inertial measurement unit and quality assist ultrasonic sensor

Indices

Security laser scanner (16 Hz)

ToF camera (20 Hz)

Motion tracking IMU (60 Hz)

Quality assist ultrasonic sensor (50 Hz)

Data delay rate (D i ) (ms)

62.5

50

16.6

20

Sensor detection capability (d) (mm)

70

145

38

40

Number of sensors (S)

1

2

4

2

Additional distance based on sensor resolution (C) (mm)

448

1048

192

208

Safety distance (SD)a (mm)

1120

1720

864

880

RT (ms)

62.5

100

66.64

40

PLIb

0.5

0.5

0.5

0.5

SDI

0.48

0.25

0.63

0.62

RIb

0.6

0.6

0.6

0.6

RTI

0.6

0.375

0.58

0.75

Total

2.18

1.725

2.31

2.47

a K = 1600 mm/s, T = 0.42 s, C = 8(d − 14)

bAssumed values

It is noted that the SD is large in case of a camera system as compared to other sensors that makes HRC nearly impossible. Moreover, SD = 1 m is required in any case for the deployment of safety speed reduction scheme, e.g., if a worker is coming toward a robot with a speed of 1600 mm/s and the robot’s follow-up time is 0.42 s, then the robot must exhibit safety speed reduction when SD <1 m. For the RT calculation, ultrasonic sensors show the best result.

6 Hypothetical application scenario

The core of the CRCPS development is the integration of dynamic characteristics of the individual components. The individual protection components register context, situation, and status of worker, machine, plant, and process and activate protective mechanisms before a hazard, e.g., a collision, can occur. The production process will run without threats and interruptions and will achieve the level of security and safety meeting legal requirements on an industrial floor. Symbiotic human–robot collaboration [32] is defined for a fenceless environment, in which productivity and resource effectiveness can be improved by combining the flexibility of humans and the accuracy of machines. CRCPS can enable such HRC with the characteristics of dynamic task planning, active collision avoidance, computational intelligence [44] and adaptive robot control. Humans are part of the CRCPS design in which human instructions to robots by speech, signs, hand gestures or other adaptor technology are possible during collaborative handling, assembly, packaging, processing or other tasks. All of these industrial tasks require a solution for HRC specifically in the domain of conventional medium and heavy payload robots, as there is no such solution exists so far.

Figure 4 shows a monitored area in which a human and a robot are interacting for completion of an industrial task. The vision system can be established through overhead 2D cameras or a 3D stereo vision camera and an additional laser scanner to cover any violation of robot workspace by a human worker. The vision system is providing the real-time location information of the worker, to the system. The robot system is programmed to reveal its end-effector position in all six DOF. The vest, which the worker will wear all the time, contains multiple IMU fitted at various body locations of the human worker thereby providing position and rate information to the CRCPS. The same can be proposed for an IMU fitted helmet for accurate head positioning information. These IMUs contain six sensors, i.e., three gyros for the three angular deflections and three accelerometers for linear acceleration measurement.
Fig. 4

HRC in CRCPS design: a hypothetical industrial scenario example

A pre-defined safe distance margin enables the system to identify if the worker is near to the robot. Speed or acceleration reduction can be started suddenly upon identification of a dangerous situation and may lead to full stoppage of the robot until worker leaves the safe distance limit in the workspace. The robot will continue its job from the point it went in to full stop. There is an interaction mode in which either the hands or the worker voice can be utilized to train the robot. For this, different hand gestures can be used to train the robot in the interactive environment. For force-monitoring system, force reduction approach is applied suddenly, once the SD margin is reached. Force sensors can provide an additional feature in the case of touching the human worker. Force calibration for different body organs is a must requirement in order to design such systems. Joints of new collaborative robots are equipped with force sensors, torque sensors and load cells. However, conventional robots without force sensors in joints cannot be used in the force reduction and monitoring approach. Collaborative robots as shown in Table 1 have the capability of collision detection and hold operation once collided with human worker. Force calibration on the basis of collision forces that are below any threshold of human pain level is required. There are recently developed [45, 46, 47] guidelines on contact forces based on biomechanical experimentation.

In such a CRCPS, multiple sensors integration and computational intelligence schemes like human tracking, human avoidance and intelligent use of multiple sensory data can be implemented. Due to the resource exhaustive nature, the real time and software issues arise in the embedded systems distributed intelligence. The integration between the cyber and physical layer requires communication and synchronization of the embedded system software that introduces complexity, limiting performance of real-time system [48] and the emerging problems due to the compromised cyber-security during the product life cycle. To cater for such issues, there are overhead controlling and self-verification approaches [49, 50, 51]. Such approaches can be useful in dealing with unusual system behavior within CPS modules and to find out the actual cause of the malfunction. These system integration approaches in CPS research include intelligent sensor fusion techniques, intelligent modular synchronization and different layers of protection checks and verification schemes depending upon the allowed overhead.

7 Generalized methodology for various industrial scenarios

In addition to the safety concepts, HRC attainment level and the sensor technology employed for a particular solution, there may be multiple industrial scenarios for which a generalized methodology can be established. Figure 5 shows the general methodology for building a CRCPS in a given industrial scenario. The methodology starts from an HRC industrial scenario from which the detailed customer requirements are generated. The methodology shows criteria based on several collaboration indices. The indices are evaluated based on the sensor level information from the sensor library that can be established on the basis of state of the art sensor technology and holds vital specifications information in a software form. Once an initial set of sensors is selected, an optimized solution is searched between the collaboration indices and the sensor specifications selected from the library. The final solution of the optimization algorithm is matched to customer-specific requirements for the CRCPS design. If the result is unfeasible, the requirements are then adjusted according to the presented solution. Once customer requirements are met, the solution is implemented.
Fig. 5

Methodology for implementation of CRCPS in various industrial scenarios: key components are the sensor’s library and collaboration indices

Figure 6 shows the optimization procedure in the design methodology for CRCPS. Detailed sensor specifications from the sensor library are used to tabulate the initial data from the selected sensors. The upper and lower bounds of the specifications are set as part of the data input. These are the input constraints applied and can be changed by the user if the final optimized solution does not come up to the customer requirements and expectations. The initial data are populated using a suitable design of experiments (DOE) technique, e.g., factorial method, Taguchi or randomization. After spreading the initial population, multi-objective genetic algorithm runs that is selected due to the characteristics of directional crossover, fast convergence and objective function penalization. Multiple objective functions are defined according to the collaboration indices or KPIs mentioned in Fig. 3. In the optimization process, total number of iterations t is calculated according to the size of the initial population times the selected number of generations (N gen). Once the number of iterations reaches N gen, the algorithm stops and presents the final solution for technology selection. The optimized solution is the set of sensor specifications that can achieve best possible KPIs. By using optimization algorithms like genetic algorithm or optimization techniques using heuristics, the global optimum can be reached in the final solution that can avoid the local optimum traps.
Fig. 6

Optimization data flow in CRCPS design methodology

Apart from the data flow in the CRCPS methodology, there can be various industrial scenarios based on real-life industrial situations like in an assembly line in which a single worker may interact with multiple robots or vice versa. There can be technology solutions, other than camera systems for the scenarios of varying illumination conditions at different day timings. The systematic evolution of scenarios is based on the technologies delivering worker position information in the CRCPS as the robot gripper position is known from any scenario. Table 4 summarizes those technologies comprising of sensor systems and software algorithms that are required to complete basic industrial tasks. An intelligent multiple worker tracking system is an example of software modules in addition to the camera systems in case of multiple workers interacting with robot at the same time.
Table 4

Human position recognition systems in different industrial scenarios

Industrial scenarios

Human position recognition systems

IMU

Vision system and radar

Single robot–single worker

Correction factor adjustment with gyros

Human motion capturing software

Worker identification tracker

Human avoidance algorithm

Real-time safety distance computation

Gripper path optimization in human presence

Only radar tracker will work in poor lighting conditions

Multiple robots–single worker

Same as above

Same as above

Single robot–multiple workers

Correction factor adjustment with gyros

Human motion capturing software

Multiple vests (IMUs) must work with robot

Intelligent multiple worker identification tracker

Human avoidance algorithm

Real-time safety distance computation

Worker position estimation in case of obstacles

Gripper path optimization in presence of multiple workers

Only radar tracker will work in poor lighting conditions

Multiple robots–multiple workers

Correction factor adjustment with gyros

Human motion capturing software

Multiple vests (IMUs) must work with multiple robots

Same as above

Figure 7 shows an industrial scenario where the activities are carried out in varying illumination conditions in different parts of the day. This lighting condition normally exists in small and medium enterprises where the factory floor is not completely isolated from the outside environment. In this case, day light camera systems can be compromised to identify the worker position; however, other technologies like radar system and IMU can function normally. Figure 8 shows a scenario in which multiple workers are collaborating with the robot at the same time. For this to implement, a smart multiple tracker needs to be developed. The intelligent tracking system can work for both, radar and camera system technologies, i.e., in this scenario, every worker collaborating with robot must wear a sensor equipped vest, band or helmet all the time. The intelligent tracking system must be able to deal the multiple workers separately. The technical scenario may get difficult in the case of physical obstacles and the two workers coming very close to each other hiding the tracking markers of one of the two. These special cases must be dealt with, by advanced estimation schemes and filters implemented in the tracking system.
Fig. 7

Industrial scenario with varying illumination conditions

Fig. 8

Industrial scenario with multiple workers needs multiple worker identification tracker

The above-mentioned industrial scenarios are summarized in Table 4. Although a generalized solution cannot be presented here, the table with some variations can be considered as generic for industrial scenarios which range from single robot to multiple robots working together with multiple human workers. The possible approaches in order to build HRC system may include options like the use of inertial sensors, vision, radar or any hybrid approach for the human position monitoring. The hybrid option may consider any of the two approaches in a combined way. The real benefit of the hybrid approach is the execution of tasks with high precision as the positioning information from two separate sensor systems will work mutually and compensate for the errors. Additionally, one technology area might be more practical in a given scenario like the vision system can be compromised in some conditions or in the case of vision obstacle. In that scenario, the other sensor technology will keep the system functional.

8 Conclusion

The paper identifies the requirements of HRC in an industrial context. A controlling CRCPS structure for HRC suggests the human worker to be an integrated part for which various interactive technologies can be employed. This approach is different from the common CPS concept in which there are only two components and the cyber component controls the physical system. There is also a definition of ACPS in which the HC takes the integrated role with the cyber and PCs. Also, in the ACPS structure, no role is fixed for any component and the roles take the shape as the CPS learns from the tasks and activities. The ACPS concept is best suited to the machine learning environment, humanoid robotics, small-sized collaborative robotics and other artificial intelligence applications, where the human role is important to train the robot for new jobs or for programmable compliance. For CRCPS, the concept of considering human worker as an integral part of the system and teaching the heavy payload robot for new tasks can be a similar concept. However, the flexibility part in the ACPS model makes it different from the proposed CRCPS structure where the human role is integrated, but with less flexibility. One example of a less flexible human role can be seen where the intelligent heavy payload robots may be restricted from programmable compliance due to safety reasons and the workers dealing with such robots have to use programming pad to train robots for new jobs. This is the basic difference in the concepts that differentiate CRCPS from ACPS.

It is found that state of the art collaborative robots is being developed for small- and medium-sized manufacturing applications and the solutions to convert for large number of existing conventional robots to intelligent ones are in infancy stage. Also, the unintelligent heavy payload robots do not have a collaborative solution yet. In this context, different safety approaches of working with robots are studied and potential detailed solutions are identified. Although, it is difficult to identify and integrate all the collaboration and industrial process requirements, but some performance indices are established for collaboration like PL, SD, risk of using unsafe components and RT based on data delay rate. These indices are realized with their best and worst values to form a scale from 0 to 1. A sum of these KPIs based on equal weights is used to reveal a grading pattern in HRC. However, it is important to further study unequal weights according to the role of every KPI in the collaborative process. An example is SD because this parameter has a decisive role in defining collaboration. HRC cannot take place at all without having the SD value within the prescribed range. Even, it can be stated that other performance indicators are valid only, if the condition for SD is met. There can be other important KPIs which may contribute in the grading system. Those KPIs can be evolved from the requirements of the identified collaborative process. Process KPIs depend upon the complexity, nature and duration of the industrial collaborative task. A general mapping of process KPIs and integration with the existing grading system is the future extension of this work.

Safety schemes are graded for HRC attainment level, and the sensor level requirements for each collaborative mode are identified. It is found that the force-limiting approach has the highest HRC level but it is only possible with force sensors in joints of the robots. Joints with force sensors are not the common feature in conventional robots, thus force-limiting approach cannot be applied to convert more than million conventional working robots in industry to intelligent ones. Speed and separation monitoring are the appropriate approaches that can be utilized in the case of conventional robots.

The paper is also focused on the technology limits and real-time issues for the achievement of high HRC. A collaborative scenario example revealed the detailed sensor requirements of a hypothetical industrial HRC system in which IMU, camera system and laser scanner are the minimum required technological components. For CRCPS design in HRC, a generalized methodology is developed to cater for various industrial scenarios. The methodology shows that a sensor library has to be established and an optimized solution can be searched using collaboration indices and sensor’s specifications. The library is suggested as the sensor specifications are stored in a standard soft form. The library can be further classified into different categories of sensors so that two sensors of the same type but with different specifications can be placed in a unique category. Although the optimized solution will show the best possible available sensors which suits the given industrial scenario, but the user can apply and modify the input bounds on the sensor specifications, if required in the proposed methodology. However, it is challenging to maintain an up to date sensor catalog from various sources and manufacturers. The methodology also mentions the use of unsafe components as a key index. Using a large pool of database for the component selection, the number of unsafe components in a solution can be increased from a certain critical level. The usage of such components will also hamper the industrial certification process of such HRC system.

Notes

Acknowledgements

The authors would like to acknowledge the support of the cLINK (Centre of excellence for Learning, Innovation, Networking and Knowledge), Erasmus Mundus Programme of the European Union and the InSA Project, sponsored by the German Federal Ministry for Economic Affairs and Energy (Project Register Number: HRB 24505 HB).

References

  1. 1.
    VDI VDE (2014) Industrie 4.0 Wertschöpfungsketten, status report. http://www.vdi.de/fileadmin/vdi_de/redakteur_dateien/sk_dateien/VDI_Industrie_4. 0_Wertschoepfungsketten_2014.pdf
  2. 2.
    Gubbi J, Buyya R, Marusic S (2013) Internet of things (IoT): a vision, architectural elements, and future directions. Futur Gener Comput Syst 29(7):1645–1660CrossRefGoogle Scholar
  3. 3.
    Zuhlke D (2008) Smart factory—from vision to reality in factory technologies. In: Proceeding of the 17th international federation of automatic control world congress (IFAC), South Korea, pp 82–89Google Scholar
  4. 4.
    Zuehlke D (2010) Smart factory—towards a factory-of-things. Annu Rev Control 34(1):129–138CrossRefGoogle Scholar
  5. 5.
    Mourtzis D (2016) Challenges and future perspectives for the life cycle of manufacturing networks in the mass customisation era. Logist Res 9(2)Google Scholar
  6. 6.
    Knight W (2014) How human–robot teamwork will upend manufacturing. In: Rotman D (ed) MIT technology review, business report, breakthrough factories. Jason Pontin. https://www.technologyreview.com/s/530696/how-human-robot-teamwork-will-upendmanufacturing/#
  7. 7.
    Lasota PA, Rossano GF, Shah JA (2014) Toward safe close-proximity human–robot interaction with standard industrial robots. In: The 10th IEEE international conference on automation science and engineering (CASE), Taipei, TaiwanGoogle Scholar
  8. 8.
    Collaborative Robotics Market Exceeds US$1 Billion by 2020, ABIresearch (2015) https://abiresearch.com/press/collaborative-robotics-market-exceeds-us1-billion-/
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
    NASA, robonaut R2. http://robonaut.jsc.nasa.gov
  14. 14.
  15. 15.
  16. 16.
    Pirvu B-C, Zamfirescu C-B, Gorecky D (2015) Engineering insights from an anthropocentric cyber-physical system: a case study for an assembly station. Mechatronics. doi: 10.1016/j.mechatronics.2015.08.010 Google Scholar
  17. 17.
    Griffiths S (2014) Meals on wheels! Robots deliver food they’ve cooked to diners in a Chinese restaurant—and they even make small talk. http://www.dailymail.co.uk/sciencetech/article-2725845/Meals-wheels-Robots-deliver-food-cook-diners-Chinese-restaurant-make-small-talk.html
  18. 18.
  19. 19.
    Kriaa S, Pietre-Cambacedes L, Bouissou M, Halgand Y (2015) A survey of approaches combining safety and security for industrial control Systems. Reliab Eng Syst Saf 139:156–178CrossRefGoogle Scholar
  20. 20.
    NIST (2013) Foundations for innovation in cyber-physical systems, workshop report. http://www.nist.gov/el/upload/CPS-WorkshopReport-1-30-13-Final.pdf
  21. 21.
    Schirner G, Erdogmus D, Chowdhury K, Padir K (2013) The future of human in-the loop cyber-physical systems. IEEE Comput 46(1):36–45CrossRefGoogle Scholar
  22. 22.
    Peissner M, Hipp C (2013) Fraunhofer institute for industrial engineering IAO. In: Spath D, Weisbecker A (eds) The potential of human–machine interaction for the efficient and networked production of tomorrow. Fraunhofer Information Center for Planning and Building IRB. Fraunhofer Verlag, Stuttgart. https://www.iao.fraunhofer.de/lang-de/images/iao-news/studie_future_hmi-en.pdf
  23. 23.
    Horvath I (2014) What the design theory of social–cyber–physical systems must describe, explain and predict? In: Chakrabarti A, Blessing LTM (eds) An anthology of theories and models of design. Springer, New York, pp 99–120CrossRefGoogle Scholar
  24. 24.
    Biamino G (2012) A semantic model for socially aware objects. Adv Int Things 2(3):47–55CrossRefGoogle Scholar
  25. 25.
  26. 26.
    Kochan A (2006) Robots and operators work hand in hand. Ind Robot Int J 33(6):422–424CrossRefGoogle Scholar
  27. 27.
    Kruger J, Nickolay B, Heyer P, Seliger G (2005) Image based 3D surveillance for flexible man–robot-cooperation. Ann CIRP 54:19–23CrossRefGoogle Scholar
  28. 28.
    Winkler B (2007) Safe space sharing human–robot cooperation using a 3D time-of-flight camera. In: International robots & vision show: technical conference proceedings, Robotic Industries Association, Rosemont, Chicago, IL, USA, pp 1–8Google Scholar
  29. 29.
    Zamfirescu CB, Pirvu BC, Schlick J, Zuehlke D (2013) Preliminary insides for an anthropocentric cyber-physical reference architecture of the smart factory. Stud Inform Control 22:269–278Google Scholar
  30. 30.
    Krüger J, Lien TK, Verl A (2009) Cooperation of human and machines in assembly lines. CIRP Ann Manuf Technol 58(2):628–646CrossRefGoogle Scholar
  31. 31.
    Khalid A, Kirisci P, Ghrairi Z, Pannek J, Thoben K-D (2016) Safety requirements in collaborative human–robot cyber physical system. In: 5th international conference on dynamics in logistics (LDIC), Bremen, Germany. Lecture notes in logistics, pp 39–48Google Scholar
  32. 32.
    Wang L, Törngren M, Onori M (2015) Current status and advancement of cyber-physical systems in manufacturing. J Manuf Syst. doi: 10.1016/j.jmsy.2015.04.008 Google Scholar
  33. 33.
    Koppula HS, Saxena A (2013) Anticipating human activities using object affordances for reactive robotic response. In: Proceedings of the robotics: science and systemsGoogle Scholar
  34. 34.
    Petric T, Zlajpah L (2013) Smooth continuous transition between tasks on a kinematic control level: obstacle avoidance as a control problem. Robot Auton Syst 61(9):948–959CrossRefGoogle Scholar
  35. 35.
    Balan L, Bone GM (2006) Real-time 3D collision avoidance method for safe human and robot coexistence. In: IEEE international conference on intelligent robots and systems, pp 276–282Google Scholar
  36. 36.
    Sanderud A, Thomessen T, Osumi H, Niitsuma M (2015) A proactive strategy for safe human–robot collaboration based on a simplified risk analysis. Model Identif Control 36(1):11–21CrossRefGoogle Scholar
  37. 37.
    Green SA, Billinghurst M, Chen X, Chase JG (2008) Human–robot collaboration: a literature review and augmented reality approach in design. Int J Adv Robot Syst 5(1):1–18Google Scholar
  38. 38.
    Davison P (2015) How does the accident in Germany affect industrial robot safety? Robotics online. Robotics Industries Association. http://www.robotics.org/content-detail.cfm/Industrial-Robotics-News/How-Does-the-Accident-in-Germany-Affect-Industrial-Robot-Safety/content_id/5555
  39. 39.
    Arai T, Kato R, Fujita M (2010) Assessment of operator stress induced by robot collaboration in assembly. CIRP Ann Manuf Technol 59(1):5–8CrossRefGoogle Scholar
  40. 40.
    Kuli D, Croft E (2007) Pre-collision safety strategies for human–robot interaction. Auton Robot 22:149–164CrossRefGoogle Scholar
  41. 41.
    EN ISO 10218-1:2011 Robots and robotic devices—safety requirements for industrial robots—part 1: robots (ISO 10218-1:2011)Google Scholar
  42. 42.
    EN ISO 10218-2:2011 Robots and robotic devices—safety requirements for industrial robots—part 2: robot systems and integration (ISO 10218-2:2011)Google Scholar
  43. 43.
    Kirisci PT, Pannek J, Ghrairi Z, Thoben K-D, Lawo M (2015) Mensch–Roboter–Kollaboration in cyber-physischen Arbeitsumgebungen. Robot Ind 4.0 Manag 31(1):43–47Google Scholar
  44. 44.
    Xing B (2014) Novel computational intelligence for optimizing cyber physical pre-evaluation system. In: Khan ZH, Shawkat Ali ABM, Riaz Z (eds) Computational intelligence for decision support in cyber–physical systems. Springer, Berlin, pp 449–64Google Scholar
  45. 45.
    ISO/TS 15066:2016 Robots and robotic devices—collaborative robotsGoogle Scholar
  46. 46.
    Behrens R, Elkmann N (2014) Experimentelle Verifikation der biomechanischen Belastungsgrenzen bei Mensch-Roboter-Kollisionen: phase I. Fraunhofer-Institut fuer Fabrikbetrieb und -automatisierung IFF, MagdeburgGoogle Scholar
  47. 47.
    Research project No. FP-0317 (2013) Collaborative robots—investigation of pain sensibility at the man–machine–interface. Institute for Occupational, Social and Environmental Medicine at the Johannes Gutenberg University of Mainz, Germany. Final reportGoogle Scholar
  48. 48.
    Mosterman PJ, Zander J (2016) Cyber–physical systems challenges: a needs analysis for collaborating embedded software systems. Softw Syst Model 15:5–16CrossRefGoogle Scholar
  49. 49.
    Porquet J, Schwarz C, Greiner A (2009) Multi compartment: a new architecture for secure co-hosting on SoC. In: International symposium on system-on-chip (SOC), pp 124–127Google Scholar
  50. 50.
    Sepulveda J, Gogniat G, Pires R, Chau WJ, Strum MJ (2013) Security enhanced 3D communication structure for dynamic 3D-MPSoCs protection. In: Symposium on integrated circuits and systems design (SBCCI), pp 1–6Google Scholar
  51. 51.
    Drechsler R, Kühne U (2014) Safe IP integration using container modules. In: International symposium on electronic system design (ISED), pp 1–4Google Scholar

Copyright information

© The Author(s) 2016

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • Azfar Khalid
    • 1
    • 3
  • Pierre Kirisci
    • 1
  • Zied Ghrairi
    • 2
  • Klaus-Dieter Thoben
    • 1
    • 2
  • Jürgen Pannek
    • 1
    • 2
  1. 1.University of BremenBremenGermany
  2. 2.BIBA-Bremer Institut für Produktion und Logistik GmbH (BIBA)BremenGermany
  3. 3.Department of Mechanical EngineeringCapital University of Science & Technology (CUST)IslamabadPakistan

Personalised recommendations