Learn and Transfer Knowledge of Preferred Assistance Strategies in Semi-Autonomous Telemanipulation

Abstract

Enabling robots to provide effective assistance yet still accommodating the operator’s commands for telemanipulation of an object is very challenging because robot’s assistance is not always intuitive for human operators and human behaviors and preferences are sometimes ambiguous for the robot to interpret. Due to the difference in hand structures, some motion assistance from the robot may surprise the operator with counter-intuitive movements, which could introduce more burden to the human to correct the actions and/or reduce the operator’s sense of system control. To address these problems, we developed a novel preference-aware assistance knowledge learning approach. An assistance preference model learns what assistance is preferred by a human, and a stage-wise model updating method ensures the learning stability while dealing with the ambiguity of human preference data. Such a preference-aware assistance knowledge enables a teleoperated robot hand to provide more active yet preferred assistance toward manipulation success. We also developed knowledge transfer methods to transfer the preference knowledge across different robot hand structures to avoid extensive robot-specific training. Experiments to telemanipulate a 3-finger hand and 2-finger hand, respectively, to use, move, and hand over a cup have been conducted. Results demonstrated that the methods enabled the robots to effectively learn the preference knowledge and allowed knowledge transfer between robots with less training effort.

Appendix

The characteristic raw data is broken down into two categories: grasp attributes, and task attributes. Grasp attributes represent the hand kinematics, which include the palm orientation, palm center location, and finger configuration corresponding to the thumb and the index and middle fingers. We denote the set of robot grasp attributes as $\mathcal {R}$ and the set of human grasp attributes as ${\mathscr{H}}$. Task attributes T describe tasks to be done.

1.1 A.1 Intent-Based Strategy

We denote the control variables of the robot as $R_{a}\in \mathcal {R}$, a is the index of variables. A set of R_a produces a probability for each task, which is denoted as P_b(R), b is the index of tasks. An intent-uncertainty-aware human grasp model from previous work [46] is created to refer to the different task inference intents T_b. There are upper and lower bounds for model parameters, U_a and L_a respectively, which the robot must adhere to, such as physical limits of end effector position or joint angles, or force provided. We establish the intent inference, which consists of three principle tasks including Use, Move, and Hand Over. For example, for grasping a cup: Use is using or drinking from the cup, Move is moving the cup to another location, and Hand Over is handing the cup over to another agent. We use intent-uncertainty-aware human grasp model ${\mathscr{M}}$ to infer the intent T_b in Eq. 5:

$$ T_{b} = \mathcal{M}(\mathcal{H}) $$

(5)

The distribution P_b(R) is used to quantify how much each task is satisfied by a given robot pose with features R_a. We use Naive Bayes robot model ${\mathscr{M}}_{r}$ to produce the robot probability vector of satisfying the task P_b(R) in Eqs. 6 to 8, where μ_b is the average value for task b, Σ_b is the covariance matrix for task b, and d is the length of vector R_a.

$$ P_{b}(R)=\mathcal{M}_{r}(R) $$

(6)

$$ P(R_{a}|b)=\frac{1}{\sqrt{det({\varSigma}_{b})(2\pi)^{d}}}e^{-\frac{1}{2}(R_{a}-\mu_{b})^{T}{\varSigma}_{b}^{-1}(R_{a}-\mu_{b})} $$

(7)

$$ P_{b}(R=R_{a})=P(b|R_{a})=\frac{P(R_{a}|b)P(b)}{{{\sum}_{b}^{B}}P(R_{a}|b)P(b)} $$

(8)

Upon developing the target probability vector and the robot probability vector, the intent-based strategy can be constructed based on the intent-based shared control criterion with added constraint, where the objective function is:

$$ \begin{array}{llll} C_{1} = min\left( \frac{1}{2}\sum\limits_{b}(P_{b}(R)-T_{b})^{2}\right)\\ s.t.\quad L_{a}\leq R_{a}\leq U_{a} \quad\forall_{i}\\ norm(R_{a})=1 \quad\forall_{a} \quad needed\, for\, palm\, direction \end{array} $$

(9)

1.2 A.2 Mimic-Based Strategy

If a human operator needs the robot to strictly follow the motion command, unintended errors may occur, but we can still achieve this goal by adding extra constraints to the intent-based strategy. The motion constraints can be explicitly dictated by adding the following set of constraints:

$$ R_{a} = H_{a} \quad \forall_{a} $$

(10)

This will give the operator full control of all features of the robot. The new constraints added to the control diagram ensure the robot follows the human exactly by matching the robot features and human features to mimic the motion. The objective functions are:

$$ \begin{array}{llll} C_{2} = min\left( \frac{1}{2}\sum\limits_{b}(P_{b}(R)-T_{b})^{2}\right)\\ s.t.\quad L_{a}\leq R_{a}\leq U_{a} \quad\forall_{i}\\ norm(R_{a})=1 \quad\forall_{a} \quad needed \, for\, palm\, direction \\ R_{a} = H_{a} \quad \forall_{a} \end{array} $$

(11)

1.3 A.3 Intent-Mimic Hybrid Strategy

We first define λ_a as the KL divergence between the distribution of each feature:

$$ \lambda_{a} = \mathcal{K}\mathcal{L}(\bar{R_{a}}||\bar{H_{a}})=ln\frac{\sigma_{H_{a}}}{\sigma_{R_{a}}}+\frac{\sigma_{R_{a}}^{2}+(\mu_{R_{a}}-\mu_{H_{a}})^{2}}{2\sigma_{H_{a}}^{2}}-\frac{1}{2} $$

(12)

Additionally, the multivariate normal distribution between two populations can be used to determine the overall divergence between hand configurations in Eq. 13:

$$ \begin{array}{@{}rcl@{}} \gamma &=& \mathcal{K}\mathcal{L}(\bar{R}||\bar{H})=\frac{1}{2}\left( trace({\varSigma}_{H}^{-1}{\varSigma}_{R})+(\mu_{H}-\mu_{R})^{T}\right.\\&&\left.{\varSigma}_{H}^{-1}(\mu_{H}-\mu_{R})-k+ln\frac{|{\varSigma}_{H}|}{|{\varSigma}_{R}|}\right) \end{array} $$

(13)

The formulation results in making the mimic constraint from the previous formulation in the objective function to act as an elastic constraint which allows the robot to bend the rules on mimicking the human. The grasp position is generated by minimizing Eq. 14.

$$ \begin{array}{llll} C_{3} = min\left( \frac{1}{2}\sum\limits_{b}(p_{b}(R)-T_{b})^{2}+\frac{1}{\gamma}{\sum\limits_{a}^{A}}\frac{1}{\lambda_{a}}(R_{a}-H_{a})^{2}\right)\\ s.t.\quad L_{a}\leq R_{a}\leq U_{a} \quad\forall_{i}\\ norm(R_{a})=1 \quad\forall_{a} \quad needed\,for\, palm\, direction \end{array} $$

(14)