Keywords

1 Introduction

Mobile robots and mobile manipulators have been popular research topics [1,2,3,4,5]. But, in general mobile manipulators have been further investigated recently and are now becoming commercial tools for industrial use [6,7,8]. In research, considerations have focused on the coordination of movements of the robot and the base since redundant degrees-of-freedom (DoF) exist by adding the moving base. An example mobile manipulator consists of a six DoF robot arm (manipulator) mounted onboard a wheeled base (e.g., automatic guided vehicle (AGV) or mobile robot) with two translational and one rotational DoF in the horizontal plane for a total of nine DoF [9]. Some mobile manipulators have more or fewer DoF and may also be equipped with vertical axis motion control of the robot arm base.

As with robot arm or AGV performance, it is important for manufacturers and users of mobile manipulators to implement performance measurements to understand their system capabilities for appropriate application. For example, a user may wish to apply a mobile manipulator to assemble an engine having relatively high tolerances and associated costs as compared to inserting relatively low tolerance and cost rivets into sheet metal covers.

Measurements of the performances of mobile manipulators performing standard tasks (poses and motions) are non-existent except for simply ensuring that the task has been more or less completed. Robot performance measurements may include path comparison and path drawing, Cartesian and polar coordinate measuring, triangulation, optical tracking, inertial measuring, as well as the difference in position and orientation, or pose, of mainly the end of arm tooling from the commanded robot pose. Ground truth measurement using motion tracking systems of various techniques provides relatively accurate robot joint, segment, or tool point position information such that comparisons can be made to the commanded pose. Summarizing review of robot, mobile robot, and mobile manipulator performance measurement research shows this as being relatively new to the research community [10].

A survey of research on performance measurement of mobile manipulators [10] was published by the National Institute of Standards and Technology (NIST) as basis for research in the Robotic Systems for Smart Manufacturing (RSSM) Program [11]. The Program develops and deploys advances in measurement science that enhance U. S. innovation and industrial competitiveness by improving robotic system performance and other aspects to achieve dynamic production for assembly-centric manufacturing. Recently, NIST has been measuring performance of mobile manipulators using both a motion tracking system and artifacts designed at NIST. The artifacts can provide an inexpensive, yet low uncertainty method for manufacturers and users to measure the performance of mobile manipulators.

Performance standards, such as ASTM F45 [12], can also benefit from the use of low cost, high accuracy artifacts to develop generic test methods so that manufacturers and users perform similar and comparative tests. Specifically, ASTM F45.02 Docking and Navigation subcommittee is developing work item WK50379 [13] on docking unmanned ground vehicles and their onboard equipment, such as manipulators. In this work, the use of artifacts is being considered as ground truth for mobile manipulator performance measurement.

This paper discusses the design and use of the NIST artifacts, called reconfigurable mobile manipulator artifacts (RMMAs), in measuring mobile manipulator performance. The concept of using artifacts and programmed algorithms to control the manipulator are discussed. An experiment demonstrating feasibility and experimental results is then discussed followed by conclusions that suggest follow-on measurements.

2 Performance Measurement Using Artifacts

The concept includes positioning a mobile manipulator next to an artifact as well as positioning and orienting the end of arm tool (EOAT) attached to the manipulator at specific locations above an artifact to digitally detect fiducials with known uncertainty. The performance evaluation criteria include the:

  • Time to register the mobile manipulator to the artifact

  • Time to move from the registration points to the assembly points

  • Repeatability after registration

  • Number of search steps equating to the initial distance from registration/assembly points

  • Detection of reflectors with varying diameters

Artifacts, called reconfigurable mobile manipulator artifacts (RMMAs), were designed at NIST to include square, circle, triangle, straight and curved lines, and sinusoidal geometric patterns of tapped holes drilled into machined plates with tolerance of ±0.025 mm. The static and index artifact, RMMA-1, is shown in Fig. 1(a) beside a mobile manipulator. A dynamic RMMA, RMMA-2, is shown in Fig. 1(b). Static means a stopped vehicle with the only onboard manipulator moving to detect fiducials. Index means the mobile vehicle moves from one static location to another where the manipulator cannot physically reach all patterns from one stopped location. Dynamic means both a continuously moving vehicle and onboard manipulator to detect fiducials. Figure 1(c) shows a 457.2 mm square pattern of four reflector fiducials located at the square corners of RMMA-1 and Fig. 1(d) shows a close-up of the reflector inside the square tube reflector housing (which also supports a collimator to be explained later).

Fig. 1.
figure 1

Reconfigurable mobile manipulator artifacts (RMMAs) showing (a) static and index version (RMMA-1) and (b) dynamic version (RMMA-2). (c) Static and index version square pattern of reflectors and (d) close-up, top view of an illuminated reflector inside the square tube. The inset in (a) shows the retroreflective laser sensor used to detect reflectors.

Full size image

Both RMMAs can be in a horizontal (as tested in this research), vertical, or other orientation, at short-to-tall heights, and even configured overhead as would be typical of assembly in manufacturing facilities for an unlimited set of performance measurement possibilities. The EOAT was a retroreflective laser sensor (RLS) that emits light to a reflector and is detected by the RLS. A camera, with a light source, could instead be used as the detection sensor, especially with a larger diameter reflector or other spot. For the RLS/reflector concept, no camera software algorithm was required as the RLS connected directly into one of the manipulator digital inputs. The reflectors can have specific diameters depending upon the required uncertainty for their location. The manipulatorFootnote 1 used has manufacturer’s specified repeatability of 0.1 mm [14] and the AGV navigation sensor has manufacturer’s specified resolution of 1 mm [15]. No information is provided by the AGV manufacturers to specify the vehicle performance such as position accuracy. As published in [16], 6.3 mm diameter reflectors were used to test mobile manipulator uncertainty as an initial concept feasibility test.

For our tests, detector-to-reflector distance parallel to the laser axis was approximately 127 mm where the minimum and maximum detection distances are 100 mm and 10 m respectively. The distance would be representative of a programmed waypoint above and in-line with the next manipulator task point aligned to grip or insert a part or perform another task. The desired uncertainty may be, for example, a part insertion alignment tolerance required for a manufacturing assembly process. Moving along this grip/insertion line, parallel to the laser, at the aligned pose to the task point, also provides some knowledge of insertion performance (i.e., if the task point is continuously detected along the grip/insertion line).

Each adapter, to be screwed into the various patterns of holes, supports a background target, a circular reflector, a square tube reflector housing, and a cylinder used as a light collimator. Circular collimators are inserted into three dimensional (3D) printed (blue) square tubes that house a micro reflector, a reflector cover with a specific diameter hole, through which exposes the reflector, on top of the reflector and the collimator on top of the cover. Flat background targets, measuring 7.6 cm diameter with 6.4 mm incremental rings are perpendicular to each collimator and sometimes used as a simple visual cue for the test director when the manipulator does not align with the reflector. The adapter, reflector, housing, target, and collimator can be perpendicular to the flat surface or rotated to pitch angles between ±90° and yaw angles between 0° and 360°. The reflector and the collimator inside can be any diameter, dependent upon the sensor specification and the desired measurement uncertainty. Experiments for this paper utilized 1 mm through 6.3 mm diameter reflector cover holes where the 1 mm diameter hole was used for registering the manipulator to the reflector center.

Without the collimator, as shown in Fig. 1(c), the reflector can be detected at approximately ±20° to the vertical axis. For the collimators used, the reflector can be detected at a maximum 3.2 mm radius from the reflector center. It can be detected at approximately ±7° to the vertical axis when using a 12.7 mm inside-diameter collimator. Collimators could be made with even smaller inside-diameters to force more perpendicular manipulator tool point axis pose to the reflector.

The RLS, shown in Fig. 1(a), shows the sensor mounted in-line and perpendicular to the manipulator tool point. Initial alignment to the reflector can occur using one of several methods briefed in [16]. For our experiments, we aligned the RLS using the manipulator jog mode from the teach pendant until the laser detected the two registration reflectors for both the circle and square reflector patterns. Therefore, we could read directly from the teach pendant the end-effector coordinates to return to during our experiment.

An optical tracking measurement system was initially used as ground truth [17] for comparison to the use of artifacts and to measure all system components simultaneously for test method development. The ground truth system has static positional accuracy of 0.02 mm, however costs approximately 20 times the cost of the artifact concept. Figure 2 shows a snapshot of the optical tracking system markers used as fiducials and positioned on the square pattern of collimators. The tracking system measured the performance of the vehicle, manipulator, and artifact within the same system of reference. In this paper, the focus is to discuss the comparison between only the manipulator and artifact as a low cost, relatively high accuracy measurement method. The potential for artifacts being made using three dimensional (3D) printing could lower the cost further by an order of magnitude, or 200 times, as demonstrated through machining costs of the RMMAs and the 3D printing of parts used with the artifact.

Fig. 2.
figure 2

RMMA with optical tracking system markers on each collimator. Markers are also shown in Fig. 1(a) attached to the retroreflective laser sensor held by the manipulator.

Full size image

3 Experiments

The experiment consisted of moving the AGV from a home position away from the RMMA-1. The AGV control program moved the AGV to the first location where its position and orientation or pose was pre-determined by the AGV control program. AGV orientation angles were programmed to be at 45° with respect to the RMMA-1. Upon completion of the pattern detection for a location, the AGV moved to the second location and pose, and so forth until six locations were completed. The AGV completed the test by returning to the home position.

A modified registration method for registering the mobile manipulator to the RMMA was recently developed that uses the components shown in Fig. 3, including a 3D printed aperture housing, collimator, micro-reflector, and a camera iris aperture. The aperture allows the opening to the reflector to not only be any size, but to also center the opening. This ensures that the two registration fiducials are centered on the reflector even when using a much larger reflector for all other fiducials. The smallest aperture opening used was 1 mm diameter while all other fiducials were 3.2 mm or 6.3 mm diameter. The RLS did not return a ‘detect’ at a smaller diameter than 1 mm diameter aperture setting.

Fig. 3.
figure 3

(a) Components used for registration and (b) registration fiducial mechanism attached to the RMMA.

Full size image

A circular search pattern was used in previous tests [16] to register the manipulator to the first reflector. Once the first fiducial was acquired, it was possible for a registration skew to occur as only the edge of the pattern may have been detected and the opposite side of the second fiducial used for registration could cause an incorrect performance measurement of the mobile manipulator. The circular search began with a step increment chosen to be approximately half of the diameter of the fiducial being tested. For example, an initial step size of 3.1 mm was chosen for a 6.3 mm fiducial to be detected. However, after the initial circle of steps was completed, the step moved to the next larger 3.1 mm circle step radius (e.g., from 3.1 mm to 6.2 mm radius from the start location) and at the same step arc angle (e.g., 15°) causing much larger steps to occur as the circle pattern grew. Instead, a square pattern was tested for the research described in this paper that kept the same step size throughout the entire search. An example of the square step pattern is shown in Fig. 4 where the search begins away from the reflector at the chosen start point (yellow arrow dotted end) and each step moves along the small white or gray arrows until the RLS detects the reflector with the red arrow step.

Fig. 4.
figure 4

Example square step search pattern drawing. The pattern begins with the yellow arrow dotted end and ends when the reflector is detected with the red arrow search step. (Color figure online)

Full size image

The Mobile Manipulator program controlled the manipulator during the tests. It interfaced with the AGV directly to obtain the current AGV position and orientation, and it interfaced with the AGV control program (Transport Structure) running on the Order Manager application to coordinate the motion of the arm with the motion of the AGV. The AGV control program signaled the Mobile Manipulator program when it arrived at one of the stop or test locations. The AGV control program also sent the identification number of the test location. The Mobile Manipulator program read the current AGV pose and used it to compute the initial search location of the two registration reflectors in the target pattern (circle or square). Additional patterns could also have been used in the Mobile Manipulator program.

The manipulator was first moved from a stowed location over the body of the AGV to a staging location directly in front of the AGV. The manipulator was then moved from the staging location to the first of the two registration reflectors. The staging location was chosen so that the manipulator could make a straight line motion from the staging location to a registration reflector located in front of, or to either side of, the AGV without colliding with its shoulder joint. After moving to the first registration reflector, the manipulator performed a square spiral search to determine the exact location of the reflector. When it determined the location of the first registration reflector, the program repeated the process with the second registration reflector. When the locations of the two registration reflectors were determined, the program had sufficient information to compute the locations of the other fiducials in the square or circular patterns. The initial search was not counted as a performance criteria since the mobile manipulator could use various types of registration techniques, such as: physical contact using a touch probe [18], cameras detecting fiducials [19], or laser interferometry, theodolites, and coordinate measuring arms [20]. However, for comparison to repeatability, the initial registration number of iterations count was logged and included in results.

Once the locations of all reflectors in the pattern were computed, the manipulator cycled through them a set number of times – 32 times for each pattern in this experiment. At each reflector, the RLS checked to see if the manipulator was aligned with the reflector.

When the test was completed, the manipulator was moved to the staging location and then the stow location. When the manipulator was back in the stow location, the Mobile Manipulator program signaled to the AGV control program that it was clear to move.

The positions of the index fiducials for the targets were recorded prior to performing the repeatability tests. The AGV was first moved to a location where it could reach both of the index fiducials. The current location and orientation of the AGV was recorded. The arm was repositioned manually until the sensor detected alignment with each of the index fiducials, and the manipulator position was recorded. This information, along with the manipulator base position relative to the vehicle’s coordinate system, allowed the correct manipulator coordinates for the index fiducials to be calculated for an arbitrary AGV location. This allowed the AGV to approach the target/work area from any direction and to compensate for variation in the AGV’s stopping pose.

The calibration of the manipulator base location involved recording the position of one or more fiducials from a variety of locations. Both the AGV location and the manipulator coordinates of the fiducials were recorded. This data was processed using an iterative, non-linear model to find the best value of the base position and orientation

4 Results

The mobile manipulator performance measurement results using the RMMA-1 included only the detection of reflectors for each pattern and after initial registration. By comparison, the initial number of search steps used to register the manipulator at the first reflector was recorded. Results are shown in Table 1. The repeatability performance measurement process began once the mobile manipulator was registered to the artifact after initial registration and moving through the square or circle pattern one time and with the AGV statically positioned at a pattern. Measurements of ‘detect’ or ‘1’ were logged for each RMMA-1 reflector location. If a search was required to find the fiducial after registration, the measurement at that reflector was counted as a ‘no detect’ and the number of search steps was recorded.

Table 1. Test results of the mobile manipulator accessing the RMMA from various stop points (see Fig. 4) and various AGV poses. The gray rectangle in the center of the stop points map shows the approximate RMMA square and circle pattern locations.
Full size table

Table 1 shows: the consecutive position number and programmed AGV position, the AGV pose angle (heading), the circle or square pattern being detected, the total number of reflectors to detect for 32 pattern iterations after the registration pattern, the reflector diameters for each pattern (rounded to whole numbers), the number of reflectors detected and detection percentage, and the initial number of search steps needed to register to the first reflector after the AGV stopped. The AGV stop points programmed are shown in Fig. 5. The lines leading to the stop points indicate the AGV orientation.

Fig. 5.
figure 5

Stop points

Full size image

The table shows very high repeatability results at 97% or above as shown in the “% detected” column of the table. The results demonstrate a good test procedure for determining repeatability of a mobile manipulator to register to and access assembly points within the reflector diameters chosen. Further tests are required to understand direct connections between mobile manipulator performance and system pose, for example, suggesting that AGV pose at 0° provides higher performance than at other angles. Results here do not show this since position 6 included the AGV being at 90° and yet, was repeatable to 100%. Several additional tests are envisioned as well, such as: repeatability of the same pattern followed by the other pattern, both from different AGV poses; using the 1 mm registration reflectors for all patterns followed by the same size reflectors for all, such as 3 mm; and using 1 mm diameter reflectors for all points within each pattern to provide a possible detection limit.

5 Conclusion and Future Work

As discovered in a NIST survey of mobile manipulator research [10], performance measurement of these systems is minimal as compared to robot arms. Measurement methods, such as using optical tracking systems, are useful methods for measuring mobile manipulator performance, although at a much higher cost. The use of known artifacts, called reconfigurable mobile manipulator artifacts, to measure the performance of mobile manipulators is being researched at NIST to demonstrate the feasibility of the test method. The concept of using artifacts demonstrates to potential manufacturers and users of mobile manipulator systems that relatively low cost performance measurement methods exist. Artifacts, such as the RMMA-1 and in the future, RMMA-2, allow an unlimited number of performance measurement configurations. The measurement of mobile manipulator repeatability and accuracy for very low resolution tasks (e.g., positioning bags of product) through very high resolution tasks (e.g., assembly of parts for manufacturing) is achievable through the use of RMMAs. Static and index tests have been completed using this method and have proven feasible. Experimental results show a promising test method to measure performance of mobile manipulators that are to be used for manufacturing assembly tasks, where at least the mobile manipulator tested has the capability to perform assembly to 1 mm positional accuracy or greater. Future test method developments should not only include dynamic mobile manipulator performance measurements, but also include the suggested tests in the results section. Additionally, rapid registration techniques, finer retroreflective laser sensors allowing smaller diameters, and in turn, physically providing peg-in-hole measurements with variable peg and hole chamfers, are expected to provide even higher performance measurements towards assembly applications of mobile manipulators.