Cross-industry standard test method developments: from manufacturing to wearable robots
- 245 Downloads
Manufacturing robotics is moving towards human-robot collaboration with light duty robots being used side by side with workers. Similarly, exoskeletons that are both passive (spring and counterbalance forces) and active (motor forces) are worn by humans and used to move body parts. Exoskeletons are also called ‘wearable robots’ when they are actively controlled using a computer and integrated sensing. Safety standards now allow, through risk assessment, both manufacturing and wearable robots to be used. However, performance standards for both systems are still lacking. Ongoing research to develop standard test methods to assess the performance of manufacturing robots and emergency response robots can inspire similar test methods for exoskeletons. This paper describes recent research on performance standards for manufacturing robots as well as search and rescue robots. It also discusses how the performance of wearable robots could benefit from using the same test methods.
KeywordsWearable robot Exoskeleton Cross-industry Artifact Standards Grasping
Wearable robots, such as exoskeletons, are part of a broad category that includes systems that guide humans to assist them in moving their bodies as well as human-guided systems that augment body motions and forces for added speed or strength. Wearable robots can be partial or full body systems and are currently being developed in many countries around the world (Wolff et al., 2014).
Wearable robots have current or potential applications in rehabilitation (Szondy, 2015), elderly care (Dale, 2014; Leber, 2014), military operations (Tucker, 2015), and manufacturing (Stinson, 2014). The International Organization of Standardization (ISO) 13482 personal care robot safety standard was developed to provide safeguards for elderly or other persons using wearable robots, such as exoskeletons, and provide some cross-industry (Bostelman, 2010) consideration to manufacturing, military, or other industries. Although ISO Standard 13482 has been published, it includes no normative references to directly assess risks or hazards, design, verification, installation, or validation. Also, Herr (2009) suggested that there are other types of exoskeletons as yet undeveloped which may offer, for example, a “significant decrease in the metabolic demands of walking or running”, for which some measures for the standard may be required.
A few studies have attempted to measure the performance of exoskeleton systems to increase the wearer’s speed, strength, and endurance. However, the proposed measurements are targeted towards specific situations and cannot be generalized. Repperger et al. (1990) conducted an experiment to evaluate the performance of a human being wearing an exoskeleton, using Fitt’s law to characterize the performance. Liu et al. (2004) proposed a method to control a lower extremity exoskeleton and described experiments investigating how to measure the human and exoskeleton zero-moment point (ZMP). A control mechanism based on a hydraulic pressure valve for a lower extremity exoskeleton was presented by Guo et al. (2012). The results of a performance test on the hydraulic pressure control system were used to adjust different parameters to improve the system. Asín-Prieto et al. (2015) suggested a regression-based method to reconstruct speed-dependent and angular trajectories, to provide a more natural gait when using wearable rehabilitation exoskeletons. Although this testing methodology seems efficient, it is still applicable only to the specific case under study. Schabowsky et al. (2010) proposed a new hand exoskeleton rehabilitation robot (HEXORR) together with appropriate sensors and enablers. They conducted a pilot study to investigate the performance of the device. However, the measures used were limited to the scope of the study. Maciejasz et al. (2014) surveyed robotic devices and exoskeletons for upper limb rehabilitation. Their review discussed various aspects of these devices (e.g., application field, target group, type of assistance, mechanical design, control strategy, and clinical evaluation), but no information was provided regarding a standard way to measure their performance.
Exoskeleton technology and collaborative industrial robots both require safe human-robot performance and capabilities. However, unlike for collaborative industrial robots, there are currently no standard test methods for measuring the safety and performance of wearable robots. Technological improvements to non-wearable (collaborative) robots, such as industrial robots, mobile robots, and mobile manipulators, have allowed robots and humans to work side by side or robots to work with other robots (Fryman and Matthias, 2012). Collaborative robot safety standards, i.e., ISO 10218-2 and ISO Technical Specification (TS) 15066, have been developed and continue to evolve.
Safety and performance test methods are being developed so that manufacturers and users can evaluate and compare the capabilities of emergency response robots (Jacoff et al., 2001) and industrial robots against the requirements of their applications and particular tasks. Test methods for these industries could provide valuable insights for wearable robot standards, including what metrics should be considered, what safety and performance test methods should be developed, and how generic test methods might demonstrate a measure of safety and/or performance.
This paper will begin with identifying the types of wearable robots used in manufacturing industries that require safety and performance testing, and considering metrics for testing these systems. Standard test methods that have been, or are currently being, developed for emergency response and industrial collaborative robots will be discussed. This will be followed by a brief discussion of the process of test method development. Lessons learned and basic concepts from response and industrial robot areas will then be considered in planning the development of test methods for wearable robots.
2 Types of wearables to be tested
Herr (2009) suggested that metabolic energy cost can be reduced when wearing some types of parallel-limb exoskeletons and other shoes. This is one measure of safety and performance that can be used to define exoskeleton usefulness. However, other metrics that are not currently in the literature go beyond metabolic cost. Herr (2009) also described exoskeletons that can provide increased lift capacity, although there is little supporting information available on the use of these systems by a variety of people (i.e., of various sizes, shapes, genders, ages, etc.).
Duration: maximum time that a task can be performed with the SUT compared to that when performing the task without the SUT.
Speed: velocities that can be achieved and sustained with the SUT compared to those achieved when performing the task without the SUT.
Pose: uncertainty accuracy/resolution (e.g., precision to move to a commanded location) and repeatability (e.g., move to the same commanded location more than once) for the SUT to position and orient the operator’s arm or leg as commanded. The positioning error of a tool or device when held by the controlled arm or leg is the measured component.
Back drivability or control force: force required to resist component reaction or move any or all components of the SUT when they are both driven or not driven.
Put-on/take-off complexity: difficulty in putting on or removing the SUT.
Ease of use: simplicity of initial training and ease of control of the SUT as it allows or improves task completion performance.
Vertical maneuvering: capability, speed to traverse inclines, steps, and undulating terrain.
Horizontal maneuvering: capability, speed to traverse forward, back, and side to side.
These metrics need to be refined and detailed rubrics should be provided to define the possible range of values and allow better quantification of these measures. Other metrics are listed by Wolff et al. (2014) for exoskeletons being considered or used for rehabilitation, including comfort, cost, portability, battery life, range of use, and several others related to maneuvering the body.
3 Test methods from non-wearable robots
The market for non-wearable or collaborative robots has been increasing recently, perhaps in part due to ISO 10218-2 and ISO/TS 15066 approvals, as well as research activities. The National Institute of Standards and Technology (NIST) has been performing research on collaborative robots within its performance of collaborative robot systems project (Marvel et al., 2014) as part of the robotics for smart manufacturing program.
Robots for flexible factory environments are limited by their inability to coordinate, communicate, and understand their actions, roles, and task statuses to collaborate effectively and efficiently with others. Limitations are driven by both the absence of tools and protocols needed for describing collaborative functions, and the complete lack of metrics for assessing how well robots can work together and with humans. The above project (Marvel et al., 2014) is in the process of providing the methods, protocols, and metrics necessary to evaluate the collaborative capabilities of robot systems.
Similarly, emergency response robotics is being researched at NIST within the robotics test facility (Jacoff, 2013), which is a laboratory for developing standard methods for measuring robot performance. The facility houses artifacts and equipment for measuring how well robots perform a variety of tasks that abstract real-world challenges. The application domains supported by this facility include urban search and rescue, bomb disposal, military ground operations, disaster response, and manufacturing. Artifacts are designed to be abstract representations of the environment and task challenges that a particular requirement addresses. Experiments are conducted by running a wide variety of robots through the prototype test methods to understand how best to capture data and to refine the physical artifacts and methodology.
The wearable robots community can leverage experience gained from performance test method development and applications from research on both manufacturing collaborative robotics and search and rescue robotics. The following sections describe industrial and response robot standards and test methods that have aspects that could be considered for the development of wearable robot standards.
3.1 Industrial robot standards and test methods
Industrial robot standards and working documents
ISO 10218-1, 2: robots and robotic devices—safety requirements for industrial robots—Parts 1 and 2
ISO/TS 15066: robots and robotic devices—safety requirements for industrial robots—collaborative operation
RIS 15.06-2012: American national standard for industrial robots and robot systems safety requirements
ISO/DIS 18646-1: robots and robotic devices—performance criteria and related test methods for service robot—Part 1: locomotion for wheeled robots
ANSI/ITSDF B56.5-2012: safety standard for driverless, automatic guided industrial vehicles and automated functions of manned industrial vehicles
ASTM F45.02: navigation (performance) for driverless automatic guided industrial vehicles (WK48955)
ASTM F45.02: docking (performance) for driverless automatic guided industrial vehicles (WK50379)
RIA 15.08: working group on mobile industrial robots safety
The moveable barriers increase the path confinement per trial. An automatic guided vehicle (AGV) or mobile robot is to traverse the reconfigurable path without contacting the barriers. The vehicle performance is measured by how well it follows the path without touching the barriers as their width decreases.
The RMMA can be reconfigured to be horizontal (Fig. 3) or vertical, as well as positioned below or above the mobile manipulator. The RMMA allows for a non-contacting manipulator pose to align a laser retroreflector with reflector fiducials on the artifact to within a few millimeters, dependent upon required uncertainty measurement. Static base, indexed base (i.e., stop and measure the RMMA followed by moving to a new position, stopping and measuring at the second position), and dynamic base positioning can be tested using the RMMA.
Another test used for evaluating the performance of AGVs, mobile robots, and mobile manipulators is obstacle detection and avoidance (Bostelman et al., 2015).
Grasping is another area in which performance test methods could be considered. A proposed roadmap for dexterous manipulation (Falco et al., 2014) includes impact areas focused on several aspects of dexterous arm and hand performance, including sensing, motion, control, and applications.
3.1.5 Test methods
Hand mechanics: position control, torque control of fingers/digits, grasp capacity (e.g., graspable object size and mass), grasp types supported, accuracy, and repeatability.
Tactile sensing: normal forces and pressure, force and impact sensitivity, location of touch, functional tasks, quasi-static and dynamic effects on grasp stability, in-hand manipulation of objects, and touch sensitivity (e.g., using touch to control finger position/force).
Draft test methods are being developed for robotic hands and advanced grippers under a Metrics Working Group for an Institute of Electrical and Electronic Engineers Technical Committee on robotic hand grasping and manipulation (Falco et al., 2013; 2015).
Hand exoskeletons that could benefit from industrial gripper test methods are being embedded in an astronaut’s glove (Favetto et al., 2012) and used as hand exercise devices (Sarakoglou et al., 2007).
The aforementioned roadmap (Falco et al., 2014) also includes dexterous robot arms, proposing fewer complex performance metrics than for dexterous grippers, e.g., reachable volume (i.e., the positions and orientations that an arm can achieve within the workspace), operational space (i.e., the positions and orientations in which the arm and/or hand can effectively perform the required operation), confined space access, and grasping objects while in motion.
3.2 Response robot test methods
Several performance standards have been created through the ASTM International Standards Development Organization under the E54 Committee for homeland security applications (Bostelman et al., 2015).
Potentially relevant standards (noted by ‘ASTM’), working documents under development (indicated by ‘WK’ prior to a number), and planned standards for wearable robot and passive system test methods
Capability of robot
Standard or working document
Mobility, confined area terrains and obstacles
Gaps (ASTM E2801); hurdles (ASTM E2802); inclined planes (ASTM E2803); stair/landings (ASTM E2804); gravel (WK35213); sand (WK35214); continuous pitch/roll ramps (ASTM E2826); crossing pitch/roll ramps (ASTM E2827); symmetric stepfields (ASTM E2828)
Maneuvering, sustained speed (ASTM E2829); maneuvering tasks, towing grasped/hitched sleds (ASTM E2830); maneuvering tasks, post/hole slaloms; search tasks, random mazes with complex terrain (ASTM E2853); navigation tasks: hallway labyrinths with complex terrain (WK33260); confined space voids with complex terrain (WK34434)
Image acuity (WK42363); ranging: spatial resolution (planned); localization and mapping: hallway labyrinths with complex terrain (planned); localization and mapping: wall mazes with complex terrain, sparse feature environments (planned)
Door opening and traversal tasks (WK27852); heavy lifting: surrounding area (WK44323); dexterous inspection (planned); dexterous retrieval (planned)
Current response robot test methods have been, or are being, developed to make it simple to measure, for example, how well a robot navigates around an obstacle on a level floor. Incrementally, more challenging conditions can also be tested, for example, to measure how well a robot navigates inclined planes, steps, undulating floors or complex terrains, and around obstacles (Fig. 5). Additionally, the navigation and obstacle avoidance tests can be combined with vision tests, since most response robots are tele operated. This combination also provides a human-in-the-loop test where a robot’s pitch and roll can skew the operator’s reference frame for the images provided by the onboard camera(s), thus hindering robot control. Each test generically simulates a particular capability that response robots must possess to be useful in critical situations. For example, undulating floors or complex terrains may appear in collapsed buildings where search and rescue robot missions are required.
4 Test method development
Industrial robot and search and rescue robot test methods are being developed in a similar manner. In the case of ASTM F45 performance standards development, the mobile robot and AGV industries were surveyed to establish their current and potential system capabilities to meet specific user application requirements. In the case of ASTM E54.08.01 response robot standards development, the process began with in-depth workshops with emergency responders to identify key performance metrics and deployment scenarios, particularly focusing on urban search and rescue operations. Over 100 requirements were identified over the course of three workshops, which were used to guide the test method development process (Messina et al., 2005). Over time, additional requirements were added from new constituencies, such as bomb squads (e.g., for counter-vehicle-borne improvised explosive devices).
Test method development begins with establishing metrics and, as with any experiment, the isolation of variables and hypothesized results follows. Test methods that allow simple, isolated measurements of capabilities, for example, navigation, can then be broken down into simple-through-complex tests. For example, open-area navigation of a straight line, followed by the addition of a curve, and then added obstacles in the path, and lastly, increasingly narrow path confinement, is one simple test method. In the response robots test suite, the configuration of a robot under test is to remain unchanged through all the test methods. In other words, if a heavier battery is used to extend the robot’s endurance in the power/energy tests, it must be in place during mobility tests, such as stair climbing or inclines, where a changed center of gravity may affect performance. This provides realistic information about configuration tradeoffs.
Ideally, the method should not require expensive, resource-intensive measurement systems and procedures, and thus minimalist test method apparatus design and use must be considered. Apparatus materials should resemble the actual robot application environment and be readily available, relatively inexpensive, and simple to construct as in the apparatuses shown in Figs. 2 and 5 for industrial and response robot test methods, respectively. Alternatively, the need for high precision measurement may require a different approach. The RMMA shown in Fig. 3 was designed and machined to be relatively precise compared to the positioning capability of a mobile manipulator. Even in this case, it is expected that the components can be fabricated through additive manufacturing (3D printing) to save cost and avoid machining, while maintaining the required precision.
The test method administration, procedures, and reporting methods are established. Periodic reviews of draft test methods with potential end users and robot developers, resulting in iterative improvement of the design and procedures, are desirable for ensuring that the standards are useful and usable.
5 Cross-industry test methods
This section discusses how industrial and response robot navigation, docking, combined navigation and docking, and grasping test methods could be applied to wearable robots. Methods developed for industrial and response robot performance tests can help minimize the development process or guide designs for wearable robots. For example, one type of navigation surface may be applicable to one manufacturer’s exoskeleton but not to another. Increasing complex terrain navigation may also show limitations throughout the robot development process. Similarly, an exoskeleton motor, spring, and/or counterbalance may be tuned for lifting or manipulating heavy loads but not for threading a needle. More specific applications of previously discussed concepts follow.
Wearable robots for lower body movement can perform tests similar to manufacturing mobile robots and AGVs demonstrating navigation through confined areas. For example, barriers or a series of objects can be placed along a path that the human must follow while wearing the robot. The walls can be moved closer to the path and if the human collides with the barriers or objects, the metrics of stability, maneuverability, and velocity can be measured. An additional test could be to test avoidance or maneuverability when obstacles suddenly appear in the human’s path.
Similarly, wearable robot navigation tests can be performed using response robot artifacts and methods. For example, inclined planes, undulating floors, stairs, and various complex terrains such as sand, gravel, or wet floors can be navigated while avoiding obstacles in the path.
Wearable robots or passive exoskeletons that allow human arms to move and hold tools for longer periods of time at intended locations could be measured using the RMMA. The human can instead carry a laser retroreflector or insert pegs in holes on such an artifact using a variety of geometric patterns and RMMA configurations. Similar to the mobile manipulator (Fig. 3), fiducials detectable by an optical tracking system can measure the wearable robot motion if higher precision measurement data are required. This fine motion detection data can be used to further refine wearable robot motor tuning.
Both navigation and docking can be combined for full-body exoskeleton (i.e., legs and arms) access and dexterity tests. For example, the human in the exoskeleton would repeatedly move from a different location to the RMMA, similar to tests for the mobile manipulator. Once at the RMMA, the same docking test would be administered. The results of this test could show the time for a human, wearing leg and arm exoskeletons, to repeatedly approach and be positioned to reach the RMMA (using leg exoskeletons) followed by the time to transition from full-body motion to arm-only motion (using arm exoskeletons) when controlled by the exoskeleton.
Dynamic tests can also be administered with the RMMA moving relative to the human and the same alignment task performed as previously described. Additionally, both the humans with the exoskeleton and the RMMA can be moving while alignment or peg insertion tasks are performed.
A key could be picked up, inserted into a keyhole, and rotated.
A ball is picked up, grasped using the fingers and palm, moved using only the fingers to the finger tips, and then rolled using only the fingertips.
Bars of variable diameters each attached to a spring, or thin to thick ropes each attached to a weight, are grasped and pulled and the force is measured.
A doorknob is grasped with the hand and rotated using the wrist and/or a hand-wheel is grasped with the hand and rotated using the wrist and arms.
A needle is threaded or a wrist watch-size gear is placed on a post and meshed with other similarly sized gears.
Repeated exercise of the fingers is followed by performing the above tests.
Much of the experience gained in the development of metrics and test methods for manufacturing and response applications can be applied to wearable robots. Manufacturing robotics is moving towards human-robot collaboration and response robotics is moving towards robot deployment in place of people. Test methods for both robot types are being developed to measure their performance and match it to the task at hand. Similarly, active exoskeletons are worn by humans to move body parts, and passive exoskeletons are already being used to allow humans to extend their productivity and endurance. Safety standards now allow both manufacturing and wearable robots to be used. However, performance standards for both systems are still lacking. Test methods developed for manufacturing and response robots can be directly applied to wearable robots, as described in this paper. Almost direct cross-over between these industries appears feasible and associated performance standards can also be developed for wearable robots. Future research should include the demonstration and testing of wearable robots using test methods similar to those developed for manufacturing and response robots.
- Bostelman, R.V., 2010. International standards efforts towards safe accessibility technology for persons with disabilities: cross-industry activities. Stand. Eng.Google Scholar
- Bostelman, R.V., Hong, T.H., Messina, E.R., 2015. Intelli-gence level performance standards research for autono-mous vehicles. Int. Robot Systems Workshop, p.48–54.Google Scholar
- Campbell, P., 2007. Revolutionizing Prosthetics 2009 Team Delivers First DARPA Limb Prototype. Applied Physics Laboratory, Johns Hopkins University. http://www.jhuapl.edu/newscenter/pressreleases/2007/070426.aspGoogle Scholar
- Dale, B., 2014. A New Industry Flexes its Servos. http://fortune.com/2014/08/27/exoskeletons-wearable-robotics/Google Scholar
- Falco, J.A., Marvel, J.A., Messina, E.R., 2014. A Roadmap to Advance Measurement Science in Robot Dexterity and Manipulation. NISTIR 7993, Intelligent Systems Divi-sion, Engineering Laboratory, National Institute of Standards and Technology, Gaithersburg, USA. https://doi.org/10.6028/NIST.IR.7993CrossRefGoogle Scholar
- Fryman, J., Matthias, B., 2012. Safety of industrial robots: from conventional to collaborative applications. Proc. 7th German Conf. on Robotics, p.1–5.Google Scholar
- Jacoff, A.S., Messina, E.R., Evans, J., 2001. Experiences in deploying test arenas for mobile autonomous robots. Performance Measurement for Intelligent Measurement Systems Workshop, p.1–8.Google Scholar
- Leber, J., 2014. A Wearable Robot Suit That Will Add Power to Your Step. http://www.fastcoexist.com/3035535/a-wearable-robot-suit-that-will-add-power-to-your-stepGoogle Scholar
- Messina, E., Jacoff, A., Scholtz, J., et al., 2005. Statement of Requirements for Urban Search and Rescue Robot Per-formance Standards. http://www.nist.gov/el/isd/ms/upload/Prelim_Requirements_Report.pdfGoogle Scholar
- Stinson, L., 2014. Navy’s Exoskeleton Could Make Workers 20 Times More Productive. http://www.wired.com/2014/09/navys-exoskeleton-could-make-workers-20-times-more-productive/Google Scholar
- Tucker, P., 2015. The Very Real Future of Iron Man Suits for the Navy. http://www.defenseone.com/technology/2015/01/very-real-future-iron-man-suits-navy/102630/Google Scholar