DOE Laser-assisted Turntable Calibration and Workpiece Registration

Abstract

The ability to review robot programs before they are executed can be used to correct erroneous programming. In complex processes, such a review can only be achieved by integration of additional peripheral devices and workpieces used into the programming framework. In this work, we present a semi-automatic method for the calibration of a turntable and a workpiece registration based on the turntable calibration utilizing only a DOE laser. The turntable pose is calculated by approaching markers on the turntable on a so called acquisition plane. Based on the calibration, workpieces are registered with intersection points between the laser beams with the turntable rotary plate. We evaluated our approach in terms of accuracy and the amount of time for execution. The resulting poses of the turntable and workpieces are used to load models into the 3D simulation of the programming framework and thus can be used to review the robot program.

1 Introduction

In human-robot collaboration, humans and robots work together on tasks, or a human solves tasks by operating the robot, so that the advantages of both parties are combined. For example, in fibre spraying processes a spray gun is attached to a robot and the robot is programmed through direct guidance, so that the process knowledge of the operator is combined with the robot’s precision and repeatability [1]. This approach, called playback programming or kinesthetic programming [2] is intuitive, since no robotics knowledge is required.

An useful extension to this programming approach is the editing of the robot program after recording the robot motion and before execution [3]. For instance, it is possible to cut out errors in the robot trajectory or to insert repetitions or branches. The editing concept can be extended to simulate the robot path with the goal to review and especially avoid incorrect robot programs. For this, the setup with all relevant objects must be known in the programming framework.

In this work, we present an extension to a kinesthetic programming framework [4], such that it is possible to calibrate an external turntable in relation to the robot and add it into the 3D simulation of the programming framework. Based on the calibration, the user can register workpieces in the 3D simulation of our framework. With this, it is possible to review robot trajectories in the 3D simulation of the framework as shown in Fig. 1.

Section 2 gives an overview of sensors and algorithms that can be used to calibrate peripherals and register objects. In Sect. 3, we describe our approach for the calibration of a turntable. Section 4 describes the method for registration of workpieces. Our extensions are evaluated in terms of accuracy and execution time in Sect. 5. Section 6 summarizes and concludes the paper.

Fig. 1

Given a setup with a turntable and workpiece (left), our goal is to know this setup in the 3D simulation of our programming framework (right)

2 Related Work

If a robot has to work with or on objects, it is necessary to localize these object and to create a virtual representation. For this, different sensors can be used. Multiple colour and depth cameras are commonly used to monitor a work cell [5]. Colour or colour depth cameras attached to a robotic arm are used for pose estimation and motion planning [6] and laser-assisted welding systems utilize already a digital industrial camera combined with a laser [7]. Nevertheless, camera based method need to handle changing lighting conditions [8] and motion blur [9]. Ultrasonic sensors are used to detect empty paint cups [10] and in outdoor applications, such as autonomous driving, radar and lidar are commonly used in case of lighting extremes such as day and night [11]. In our application, fibre spraying, only minor lightning changes occur and the process produces air pollution [12]. This results in an additional effort for cleaning and calibrating sensors. As of this, the used sensor should be as simple as possible.

Most of the challenges of the current problems of object localization are not relevant to our application. As mentioned, highly changing lighting conditions are not to be considered. Also, an approach which can handle a variety of non-stationary objects in a large scene is not required [13]. Additionally, hard real-time requirements are not given and so a fast computation is not needed [14]. Consequently, we decided to use an approach which is not facing these problems.

Insofar, a semi-automatic process in which a user can calibrate a turntable and register workpieces using a low cost DOE laser is sufficient. Thus, there is no need for a complex object localization or additional sensors. Further, a DOE laser can be used as a visual aid for playback programming in general. The only drawback of such a method is the accuracy of the procedure as this depends on the manually performed steps.

3 Turntable Calibration

We present a semi-automatic procedure with which a user can determine the complete pose of the TT in reference to the robot, so that the turntable (TT) can be included into the virtual simulation. After defining general terminology and functions for the calibration, we explain the calibration procedure in detail. In our case, the tilt of the TT is set manually before calibration, which results in the need of a explicit calculation of the tilt angle. In the case, that the tilt could be controlled and set by software, the presented method is simplified.

3.1 Definitions

We define the origin of the world coordinate system in the base of the robot and as a right-handed system. A pose $P = (T,O)$ in our world consists of a position $T := \left( x,y,z\right)$ with $x,y,z \in \mathbb {R}$ and an orientation $O := \mathbb {R}^{3\times 3}$. The set of all poses is called $\textbf{P}$. The extension .x, .y and .z is used to reference x, y or z coordinates of a pose. Further, the plane $\mathscr {A} := \left\{ (x,y,z_A) \mid x,y \in \mathbb {R} \wedge z_A = const \in \mathbb {R} \right\}$ is called acquisition plane. The smallest angle between $\mathscr {A}$ and the laser beam mounted on the robot is called the laser angle . The limitation of $\lambda$ is given by the structure of the used robot cell.

We define a set of markers $\mathscr {M} := \{m_0, m_1, m_2, m_3, m_4\}$ with $m_i \in T$ on the rotary plate of the TT. The marker $m_4$ is located in the centre of the rotary plate. The remaining markers are set on a circle with a known diameter D around the marker $m_4$ and are indexed clockwise, so that ellipse semi-axes are defined by $\left[ m_0,m_2\right]$ and $\left[ m_1,m_3\right]$, as shown in Fig. 2 left. Further, a set of robot poses $\mathscr {P} := \{p_0, p_1, p_2, p_3, p_4\}$with $p_i \in \textbf{P}$ is given, which are obtained by pointing the laser beam at the markers $m_i$ on a fixed $\mathscr {A}$ with a fixed laser angle $\lambda$.

A turntable configuration $TTC := \left( \varphi , T_T, O_s\right)$ consists of a tilt angle $\varphi \in \left[ 0^\circ , 90^\circ \right]$, a turntable position $T_T$, in which the mounting height $z_T$ is known from the structure of the robot cell, and a turntable orientation $O_s := \{left, down, right, up \}$. The orientation $O_s$ is simplified, since the TT can only be tilted to the left, down, right and up, which corresponds to rotations of $0^\circ$, $90^\circ$, $180^\circ$ and $270^\circ$ around the z-axis of the TT, as shown in Fig. 2 right. The intrinsic parameters of the TT are known. Thus, we can use the function $\tau : \varphi \times O_s \rightarrow T_T$, which returns the translation between the centre of the rotary plate and the TT position based on the tilt angle $\varphi$ and the orientation $O_s$.

Fig. 2

The DOE laser beam is used to point on marker from robot positions on the acquisition plane (left). Different mounting orientations of the turntable have to be considered (right)

3.2 Calibration Method

Before each calibration, a reference run of the TT is performed, which guarantees correct marker positions on the turntable. Then, the user approaches the robot via hand guidance, so that the DOE laser beam targets at the markers $\mathscr {M}$ on the rotary plate. The rotational axes as well as the translation in z is locked while guiding the robot, defining $\mathscr {A}$. The number of markers on the turntable is minimal, as $m_4$ serves as redundancy and can be added optionally. Marker $m_0$ is highest point of the circle on which the markers are defined and the farthest away from the tilt axis. During the calibration, the markers are approached in ascending order of their indexing (clockwise) and the corresponding robot poses $\mathscr {P}$ on $\mathscr {A}$ are stored. Based on the recorded $\mathscr {P}$, the turntable calibration TTC can be calculated as follows.

Orientation: As of the structure of the possible orientations, the lines segments $[p_0, p_2]$ or $[p_1,p_3]$ are perpendicular to the world x-axis and can used to determine the orientation. For this, we first compare the y-values of the $p_i$ to choose which line segment is perpendicular to the x-axis and then consider the values of the y-coordinate (down and up) or the x-coordinate (left and right):

$$\begin{aligned} O_s\left( \mathscr {P}\right) := {\left\{ \begin{array}{ll} down, &{} \text {if } \left| p_0.y - p_2.y\right|< \varepsilon \wedge p_1.y< p_3.y\\ up, &{} \text {if } \left| p_0.y - p_2.y\right|< \varepsilon \wedge p_1.y> p_3.y \\ left, &{} \text {if } \left| p_1.y - p_3.y\right|< \varepsilon \wedge p_1.x< p_3.x \\ right, &{} \text {if } \left| p_1.y - p_3.y\right| < \varepsilon \wedge p_1.x > p_3.x \end{array}\right. } \end{aligned}$$

(1)

We allow inaccuracies in approaching the markers with the threshold $\varepsilon \ll 1$. The equation also holds, if $p_0$ and $p_2$ are the same point, which occurs if the tilt and the laser angle both are $90^\circ$.

Tilt angle: The tilt angle $\varphi$ is determined with $p_0$ and $p_2$. Different cases are considered according to the given orientation $O_s$. In case of up and down, $\lambda$ directly influences the distance between $p_0$ and $p_2$. For left and right, the calculations are independent of $\lambda$.

down: The distance $d_x := \left| p_0.x - p_2.x\right|$ is projected from $\mathscr {A}$ along the laser beams to the rotary plate, as shown in Fig. 3 top. By using $\lambda$, $d_x$ and D, a triangle can be constructed. Thus, we can determine $\varphi$ by the ratio of the vertices $d_x$ and the distance D, where $d_x$ depends on both $\varphi$ and $\lambda$:

$$\begin{aligned} \varphi = 180^\circ - \lambda - \arcsin \left( \frac{d_x \cdot \sin (\lambda )}{D} \right) \end{aligned}$$

(2)

up: Analogous to down, a triangle can be constructed, but in this case $\lambda$ does not occur in the triangle directly, as shown in Fig. 3 bottom:

$$\begin{aligned} \varphi = \lambda - \arcsin \left( \frac{d_x \cdot \sin (180^\circ - \lambda )}{D} \right) \end{aligned}$$

(3)

left and right: The markers $m_0$ and $m_2$ are not on a parallel line to the laser beam. However, the distance $d_y := \left| p_0.y - p_2.y\right|$ is independent of $\lambda$. As of this, $\varphi$ can be calculated by the ratio of the recorded distance $d_y$ and D:

$$\begin{aligned} \varphi = \arccos \left( \frac{d_y}{D}\right) \end{aligned}$$

(4)

Position: We move the TT with respect to the emanated laser beams of $\mathscr {P}$ and the TT mounting height $z_T$. Let M be the centre of the recorded positions $\mathscr {P}$. Then the turntable position $T_T$ can be defined with the laser angle $\lambda$, the mounting height $z_T$ and the function $\tau (\varphi , O_s)$:

$$\begin{aligned} T_T := M + \frac{\tau \left( \varphi , O_s\right) .z - z_T}{\tan \left( \lambda \right) } \left( \begin{array}{c} 1 \\ 0 \\ -\tan \left( \lambda \right) \end{array}\right) + \tau \left( \varphi , O_s \right) \end{aligned}$$

(5)

Thus, the turntable calibration TTC is achieved with Eq. 1 for the orientation, Eqs. 2, 3 or 4 for the tilt angle and Eq. 5 for the position.

4 Workpiece Registration

The workpiece registration is also tied to the DOE laser and moving the robot to specific points. In contrast to the turntable, there are no markers defined on the workpieces from outset. To deal with this, we use points which are located on the workpiece surface resting on the rotary plate. With this points we define correspondence pairs between the workpiece and the rotary plate.

Fig. 3

Derivation diagram of the tilt angle calculation for the cases $O_s = \{down\}$ (top) and $O_s = \{up\}$ (bottom)

For each correspondence pair, the user defines first a relevant point on the digital surface shown in the graphical user interface of our framework. Then, the robot is guided by the user, so that the DOE laser points to the corresponding point in the real world. This position is stored and the system calculates the intersection of the laser beam with the upper surface of the rotary plate from the stored robot pose as second point in the correspondence pair. There should be at least three correspondence pairs, since we define the contact plane to the rotary plate. After the definition of the correspondence pairs, the transformation between object and TT is calculated with pose estimation using singular value decomposition [15].

5 Experiments

We evaluate our approach in terms of accuracy and execution time for both proposed extensions. For this, we use the cell proposed in [1] and extend the programming framework presented in [4]. In terms of intuitiveness, a simple wizard-based UI was designed for the TT calibration (Fig. 4 left). Also, it is possible to save and load calibrations. For the workpiece registration, we implemented and compared two approaches: A laser-assisted registration method as proposed in Sect. 4 and the definition of correspondence points by clicking on the upper surface of the TT, so that the laser is not required (Fig. 4 right).

Fig. 4

After turntable calibration in a wizard-based UI dialogue (left), a workpiece is registered via a picking method (right) or a laser-assisted method

5.1 Accuracy of Turntable Calibration

We performed the calibration on various tilt angles $\varphi$. In the context of fibre spraying, $\varphi = 0^\circ$ and $\varphi = 90^\circ$ are most relevant. Additionally, we used different $\varphi \in [0^\circ , 45^\circ ]$ to show the accuracy for general tilt angles. Overall, 30 calibrations were performed: Eleven with $O_s = left$ and $O_s = down$ and eight with $O_s = up$. We neglected $O_s = right$, since the calculation is identical to left. The ground truth position of the TT was gauged via measuring tape and the orientation was measured with a gyroscope sensor with an accuracy of $\pm 0.06^\circ$.

Figure 5 shows the results of our evaluation. The error in the x-axis is consistently smaller than in the y-axis, and both errors scatter in the same size, resulting in an error range of up to $1.5\,\text {cm}$—neglecting the two outliers. For most calibrations, the tilt angle error is in $\left[ -0.2^\circ ,1.1^\circ \right]$. The error of the tilt angle $\varphi$ is smallest for $O_s=left$ and for $O_s=up$ a similar good accuracy is achieved, except one outlier. In case of $O_s = down$, there is a swing to the negative and two calibrations which failed completely with a tilt error of $10.9^\circ$ and $-22.5^\circ$. It is not clear what exactly caused these two failed calibrations.

Fig. 5

Results of the turntable calibration experiment. Left: Error in position x [cm]; Centre: Error in position y [cm]. Right: Error in tilt $[^\circ ]$

Fig. 6

Classes of workpieces for the registration evaluation: Defined corners (left), Rotational symmetric (centre) and workpieces whose contact surface to the rotary plate can not be targeted with the laser (right)

In Summary, the test cases for up and left have, on average, a tilt angle error in the sub-degree range and an average position accuracy in the sub-centimetre range. Although a slightly larger error rate for down, there is overall an accurate result, even though the calibration requires only a minimal number of markers.

5.2 Accuracy of Workpiece Registration

We evaluated the cases of $\varphi = \{0^\circ ; 44^\circ ; 90^\circ \}$ for three different orientations $O_s = \{left; up; down\}$. Further, we distinct workpieces in three different classes. Workpieces with defined corners (hexagon, Fig. 6 left), which can be easily clicked and approached. Rotational symmetric workpieces (tube, Fig. 6 centre), which have not clues for point definition like corners. In addition, workpieces are considered whose contact surface to the rotary plate can not be targeted with the laser (shell, Fig. 6 right). We performed the registration 50 times, with the two mentioned different acquisition methods (laser-assisted and picking) and used a different number of correspondence points each time. The workpieces are bolted in the centre of the TT, so that the position of the workpiece is exactly known with respect to the TT position. Since the definition of the relevant points is left to the user, a presumably optimal choice of points is assumed.

Fig. 7

Results of our workpiece registration evaluation. $a)-b)$ Error in translation [cm]; c) Error in contact plane $[^\circ ]$. d) Error in rotation $[^\circ ]$. The overall results are shown with subsets characterized by the used registration method

We measured the error of the centre position of the workpiece, as well as the error of the contact surface, which is the surface containing the relevant, and the rotation error around the normal vector of the contact surface. Figure 7 shows the accuracy results of all performed registrations. In general, all registrations perform in with a sub-centimetre accuracy. All workpiece classes have a similar spread in the sub-centimetre range and the registrations that utilize more correspondences are more accurate. Regarding the orientation, the contact area matches often perfectly. However, there is up to $1.5^\circ$ contact area error for the shell, independent of the used registration method. The largest outliers in the rotation error are due to use of the picking method. The remaining data points cluster in a range of $\left[ -2^\circ ,2^\circ \right]$ rotation error.

The registration with picking is more accurate than the laser-assisted approach. One reason for this is that workpieces are always mounted in the centre of the rotatory plate and it is more easy to click the points accurately, as the centre can be found relatively well by human eyes. We also notice a slight worse accuracy when tilting is present. Considering that our ground truth data is subject to error, the accuracy of the position is independent of the orientation and tilt of the turntable, as well as the recording method. However, using clearly defined corner points on the workpiece, as well as using a larger number of relevant points, improve the accuracy of the method further. The orientation of the registration is most accurate when the laser assisted method is used. The contact surface rotation is error-free for the tube and hexagon and even the $\approx 1.5^\circ$ contact plane rotation is still usable within our application fibre spraying process.

5.3 Execution Time

We measured the amount of time to perform the turntable calibration and the workpiece registration during the aforementioned experiments. The turntable calibration time is defined as the time span between opening the wizard for the calibration and loading of the turntable 3D model into the simulation with the calculated calibration. In case of the workpiece registration, we measured the time from starting the UI dialogue for registration until the object is shown in the simulation window.

Fig. 8

Measured execution time for calibration and registration [sec]

Most of the calibrations are performed in less than 150 s and need minimal 87 s, as shown in Fig. 8. The outliers result in case of $\lambda = 90^\circ$, at which the approaching with hand guidance is more strenuous. In case of the registration, differences are seen in the used method. The laser assisted method requires a comparable amount of time to the calibration and the picking method is performed with less amount of time. The wide range in the registration duration result from the varying number of correspondence pairs. On average, the amount of time rises with an additional correspondence pair by $\approx 10$ s for picking and by $\approx 20$ s for the laser assisted method.

The picking is carried out more efficiently because of the workpiece mounting and as it is performed only in the GUI. Overall, a fully known setup with workpiece and turntable is achieved in less then five minutes. A Further reduce of this amount of time is possible through regular application.

6 Conclusion

We present a method for the calibration of a turntable and the registration of workpieces based on the calibration. Our method utilizes, in contrast to the current trend in research, not several visual sensors. Instead, we utilize only a DOE laser attached to the robot. The DOE laser is used as a visual aid in approaching individual points, which are used for the determination of the turntable and workpiece poses. For successful calibrations, the translation error scatters in a 1.5 cm interval, while the tilt is in a range of $\left[ -3^\circ , 3^\circ \right]$ off the measured angle. The workpiece registration accuracy is in the interval of $\left[ -1 , 1 \right] \,\text {cm}$. The rotational error of the registration is in the range of $\left[ -6.6^\circ , 3.4^\circ \right]$. For both methods combined, five minutes of time effort is needed. The approach can be used in the production of small batches, which requires in worst case a new registration for each program execution. In future work, the known objects are used to simulate fibre spraying processes. With this, robot trajectories can be verified for correctness before execution and, if needed, optimized to reduce production costs.