Probabilistic movement primitives for coordination of multiple human–robot collaborative tasks

Abstract

This paper proposes an interaction learning method for collaborative and assistive robots based on movement primitives. The method allows for both action recognition and human–robot movement coordination. It uses imitation learning to construct a mixture model of human–robot interaction primitives. This probabilistic model allows the assistive trajectory of the robot to be inferred from human observations. The method is scalable in relation to the number of tasks and can learn nonlinear correlations between the trajectories that describe the human–robot interaction. We evaluated the method experimentally with a lightweight robot arm in a variety of assistive scenarios, including the coordinated handover of a bottle to a human, and the collaborative assembly of a toolbox. Potential applications of the method are personal caregiver robots, control of intelligent prosthetic devices, and robot coworkers in factories.

Appendix: Time-alignment of multiple demonstrations

One issue of imitation learning is that multiple demonstration trajectories provided by humans are usually, sometimes severely, warped in time. To compute the distribution of ProMP weights the demonstrated trajectories must be first aligned in relation to a common “clock”. In the context of movement primitives, this clock is often referred to as the phase variable. In this paper, all human and robot trajectories collected during the experiments presented in Sect. 4 were aligned by using the method briefly presented in (Maeda et al. 2014) and will be described in detail here.

The method consists of minimizing the cost J defined as the cumulative absolute difference between the demonstrated trajectory to be time-aligned $\varvec{y}_w$ and a trajectory taken as a phase reference $\varvec{y}_{r}$,

$$\begin{aligned} J= \sum _{k=0}^K | \varvec{y}_{r}( k ) - \varvec{y}_{w}( \varvec{t}_w^{j+1}(k) ) |, \end{aligned}$$

(25)

where both trajectories are resampled to have the same number of K steps. The vector $\varvec{t}_w^{j+1}$ is the unwarped time, which is the solution of the iterative update

$$\begin{aligned} \varvec{t}_w^{j+1} = v^j_0 + \varvec{G}^j\varvec{t}_w^{j}, \end{aligned}$$

(26)

where $\varvec{G} = \text {diag}(g(1), ..., g(K))$ and j is the iteration number of the optimization step.

We propose g(k) as a smooth and continuous warping function parameterized by N weights

$$\begin{aligned} {g}(k) = {\psi }_k^T \varvec{v}_{1:N}, \end{aligned}$$

(27)

where ${\psi }_k$ is the Gaussian basis function at time step k. The vector of parameters $\varvec{v}^j = [v_0, \ v_1, ..., \ v_N ]$ is optimized by gradient descent to decrease the cost J defined in (25). The extra parameter $v_0$ is used to shift the time which is useful when the reference and warped trajectories are, in fact, identical but start at different instants. The optimization is usually initialized with $v^j_0=0$ and $\varvec{t}_w^{j} = \varvec{t}_r$.

Dynamic Time Warping (DTW) (Sakoe and Chiba 1978) is a method widely used for solving time-alignment problems. An extension of DTW for the case where the time-alignment must be made on-line given only partial observations of $\varvec{y}_w$ was presented in (Ben Amor et al. 2014). An issue intrinsic to DTW-based algorithms, however, is that several adjacent time steps of the trajectory to be aligned may be attributed to a single time step of the reference trajectory, and vice-versa. For trajectories provided by a dynamical system, this issue leads to discontinuities in the solution and unnatural movements. An extreme example of this problem is shown in Fig. 21(a) where it is observed that parts of the warped trajectory were lost after the DTW alignment.

A heuristic referred to as the slope constraint was proposed in (Sakoe and Chiba 1978) to alleviate this problem by forcing the same index to not be repeated more than a certain threshold. The slope constraint, however, does not completely solve the discontinuity problem and the tuning of the slope constraint is task dependent. By construction, our proposed method enforces that the warping function $\varvec{g}$ is both continuous and smooth. The use of a smooth function not only avoids the tunning of slope constraint but also preserves the overall shape of the trajectory. Figure 21(b) shows the solution of our method for the same input data used in Fig. 21(a).

Fig. 21

Comparison of time-alignment given by classical Dynamic Time Warping without slope constraint (a) and the proposed method using gradient descent (b). In the latter, the thin gray curves represent intermediate solutions of the local optimization while converging