Skip to main content
Log in

Learn and Transfer Knowledge of Preferred Assistance Strategies in Semi-Autonomous Telemanipulation

  • Short Paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Enabling robots to provide effective assistance yet still accommodating the operator’s commands for telemanipulation of an object is very challenging because robot’s assistance is not always intuitive for human operators and human behaviors and preferences are sometimes ambiguous for the robot to interpret. Due to the difference in hand structures, some motion assistance from the robot may surprise the operator with counter-intuitive movements, which could introduce more burden to the human to correct the actions and/or reduce the operator’s sense of system control. To address these problems, we developed a novel preference-aware assistance knowledge learning approach. An assistance preference model learns what assistance is preferred by a human, and a stage-wise model updating method ensures the learning stability while dealing with the ambiguity of human preference data. Such a preference-aware assistance knowledge enables a teleoperated robot hand to provide more active yet preferred assistance toward manipulation success. We also developed knowledge transfer methods to transfer the preference knowledge across different robot hand structures to avoid extensive robot-specific training. Experiments to telemanipulate a 3-finger hand and 2-finger hand, respectively, to use, move, and hand over a cup have been conducted. Results demonstrated that the methods enabled the robots to effectively learn the preference knowledge and allowed knowledge transfer between robots with less training effort.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Data and Material Availability

Not applicable.

Code Availability

Not applicable.

References

  1. Lichiardopol, S.: Technische Universitat Eindhoven. DCT report 20, 40 (2007)

    Google Scholar 

  2. Colgate, J.E.: . IEEE Trans. Robot. Autom. 9(4), 374 (1993)

    Article  Google Scholar 

  3. Park, A.J., Kazman, R.N.: In: Telemanipulator and Telepresence Technologies, vol. 2351, pp. 119–129. International Society for Optics and Photonics (1995)

  4. Yang, E., Dorneich, M.C.: . Int. J. Soc. Robot. 9(4), 491 (2017)

    Article  Google Scholar 

  5. Rehman, S., Raza, S.J., Stegemann, A.P., Zeeck, K., Din, R., Llewellyn, A., Dio, L., Trznadel, M., Seo, Y.W., Chowriappa, A.J., et al: . Int. J. Surg. 11(9), 841 (2013)

    Article  Google Scholar 

  6. Çavusoglu, M. C., Sherman, A., Tendick, F.: . IEEE Trans. Robot. Autom. 18(4), 641 (2002)

    Article  Google Scholar 

  7. Murphy, R.R., Kravitz, J., Stover, S.L., Shoureshi, R.: . IEEE Robot. Autom. Mag. 16 (2), 91 (2009)

    Article  Google Scholar 

  8. Hauser, K.: . Auton. Robot. 35(4), 241 (2013)

    Article  Google Scholar 

  9. Yang, X, Sreenath, K, Michael, N: In: 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp. 5948–5953 (2017)

  10. Jain, S., Argall, B.: In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 3905–3912 (2018)

  11. Cui, J., Tosunoglu, S., Roberts, R., Moore, C., Repperger, D.W.: In: Proceedings of the Florida Conference on Recent Advances in Robotics. Florida Atlantic University Boca Raton, pp. 1–12 (2003)

  12. Liu, C., Walker, J., Chai, J.Y. (2010)

  13. Breazeal, C., Berlin, M., Brooks, A., Gray, J., Thomaz, A.L.: . Robot. Autonom. Syst. 54(5), 385 (2006)

    Article  Google Scholar 

  14. Griffin, W.B., Findley, R.P., Turner, M.L., Cutkosky, M.R.: In: ASME IMECE 2000 Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems. Citeseer, pp. 1–8 (2000)

  15. Gioioso, G., Salvietti, G., Malvezzi, M., Prattichizzo, D: . IEEE Trans. Robot. 29(4), 825 (2013)

    Article  Google Scholar 

  16. Cui, L., Cupcic, U., Dai, J.S.: . J. Mech. Des. 136(9), 091005 (2014)

    Article  Google Scholar 

  17. Li, S., Ma, X., Liang, H., Görner, M., Ruppel, P., Fang, B., Sun, F., Zhang, J.: In: 2019 International Conference on Robotics and Automation (ICRA). IEEE, pp. 416–422 (2019)

  18. Salvietti, G., Meli, L., Gioioso, G., Malvezzi, M., Prattichizzo, D: . IEEE/ASME Trans. Mechatron. 22(1), 445 (2016)

    Article  Google Scholar 

  19. Bicchi, A., Kumar, V.: In: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), vol. 1, pp. 348–353. IEEE (2000)

  20. Gualtieri, M., Ten Pas, A., Saenko, K., Platt, R.: In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 598–605 (2016)

  21. Calandra, R., Owens, A., Upadhyaya, M., Yuan, W., Lin, J., Adelson, E.H., Levine, S.: arXiv:1710.05512 (2017)

  22. Ekvall, S., Kragic, D.: In: Proceedings 2007 IEEE International Conference on Robotics and Automation. IEEE, pp. 4715–4720 (2007)

  23. Li, Y., Tee, K.P., Chan, W.L., Yan, R., Chua, Y., Limbu, D.K.: . IEEE Trans. Robot. 31(3), 672 (2015)

    Article  Google Scholar 

  24. Webb, J.D., Li, S., Zhang, X.: In: 2016 American Control Conference (ACC). IEEE, pp. 7110–7116 (2016)

  25. Muelling, K., Venkatraman, A., Valois, J.S., Downey, J.E., Weiss, J., Javdani, S., Hebert, M., Schwartz, A.B., Collinger, J.L., Bagnell, J.A.: . Auton. Robot. 41(6), 1401 (2017)

    Article  Google Scholar 

  26. Marayong, P, Li, M, Okamura, AM, Hager, GD: In: 2003 IEEE International Conference on Robotics and Automation (Cat. No. 03CH37422), vol. 2, pp. 1954–1959. IEEE (2003)

  27. Kuhn, S, Gecks, T, Henrich, D: In: 2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp. 485–492. IEEE (2006)

  28. Nikolaidis, S, Zhu, YX, Hsu, D, Srinivasa, S: In: 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 294–302. IEEE (2017)

  29. Tsarouchi, P., Makris, S., Chryssolouris, G: . Int J Comput Integr Manuf 29(8), 916 (2016)

    Article  Google Scholar 

  30. Nikolaidis, S., Forlizzi, J., Hsu, D., Shah, J., Srinivasa, S.: arXiv:1707.02586 (2017)

  31. Jain, A., Sharma, S., Joachims, T., Saxena, A.: . Int J Robot Res 34(10), 1296 (2015)

    Article  Google Scholar 

  32. Munzer, T., Toussaint, M., Lopes, M.: In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 879–885. IEEE (2017)

  33. Akkaladevi, S.C., Plasch, M., Eitzinger, C., Maddukuri, S.C., Rinner, B.: In: International Conference on Intelligent Human Computer Interaction, pp. 3–14. Springer (2016)

  34. Liu, R., Zhang, X.: arXiv:1701.08269 (2017)

  35. Bowman, M., Zhang, X.: arXiv:2003.03677 (2020)

  36. Colasanto, L., Suárez, R., Rosell, J.: . IEEE Trans. Syst. Man. Cybern: Syst 43(2), 390 (2012)

    Article  Google Scholar 

  37. Peer, A., Einenkel, S., Buss, M.: In: RO-MAN 2008-The 17th IEEE International Symposium on Robot and Human Interactive Communication, pp. 465–470. IEEE (2008)

  38. Rohling, R.N., Hollerbach, J.M.: In: [1993] Proceedings IEEE International Conference on Robotics and Automation, pp. 769–775. IEEE (1993)

  39. Li, S., Jiang, J., Ruppel, P., Liang, H., Ma, X., Hendrich, N., Sun, F., Zhang, J.: In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 10,900–10,906. IEEE (2020)

  40. Reed, R., Marks, R.J. II: Neural smithing: supervised learning in feedforward artificial neural networks. Mit Press (1999)

  41. Kullback, S., Leibler, R.A.: . Ann. Math. Stat. 22(1), 79 (1951)

    Article  Google Scholar 

  42. Lu, S., Lu, Z., Zhang, Y.D.: . J. Comput. Sci. 30, 41 (2019)

    Article  Google Scholar 

  43. Hosny, K.M., Kassem, M.A., Foaud, M.M.: . PloS one 14(5), e0217293 (2019)

    Article  Google Scholar 

  44. Glorot, X., Bordes, A., Bengio, Y.: In: Proceedings of the fourteenth international conference on artificial intelligence and statistics (JMLR Workshop and Conference Proceedings), pp. 315–323 (2011)

  45. Dayan, P., Abbott, L.F.: Theoretical neuroscience: Computational and mathematical modeling of neural systems. Computational Neuroscience Series (2001)

  46. Bowman, M., Li, S., Zhang, X.: In: 2019 International Conference on Robotics and Automation (ICRA), pp. 409–415. IEEE (2019)

Download references

Acknowledgments

This material is based on work supported by the US NSF under grant 1652454 and 2114464. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the National Science Foundation.

Funding

This material is based on work supported by the US NSF under grant 1652454. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the National Science Foundation.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the study conception and design. The first manuscript was written by Lingfeng Tao. Dr. Jiucai Zhang and Dr. Xiaoli Zhang provided comments and edits towards the creation of the final manuscript.

Corresponding author

Correspondence to Xiaoli Zhang.

Ethics declarations

Ethics approval and consent to participate

The experiments were approved by the Institutional Review Board (IRB) of Colorado School of Mines. Prior to participating in the study, a short introduction was provided to the participants, including technologies involved, the system setup, and the purpose of the study. Informed consent was obtained from all individual participants included in the study.

Consent for Publication

The participants have consented to the submission of the case report to the journal.

Conflict of Interests

Not applicable.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

The characteristic raw data is broken down into two categories: grasp attributes, and task attributes. Grasp attributes represent the hand kinematics, which include the palm orientation, palm center location, and finger configuration corresponding to the thumb and the index and middle fingers. We denote the set of robot grasp attributes as \(\mathcal {R}\) and the set of human grasp attributes as \({\mathscr{H}}\). Task attributes T describe tasks to be done.

1.1 A.1 Intent-Based Strategy

We denote the control variables of the robot as \(R_{a}\in \mathcal {R}\), a is the index of variables. A set of Ra produces a probability for each task, which is denoted as Pb(R), b is the index of tasks. An intent-uncertainty-aware human grasp model from previous work [46] is created to refer to the different task inference intents Tb. There are upper and lower bounds for model parameters, Ua and La respectively, which the robot must adhere to, such as physical limits of end effector position or joint angles, or force provided. We establish the intent inference, which consists of three principle tasks including Use, Move, and Hand Over. For example, for grasping a cup: Use is using or drinking from the cup, Move is moving the cup to another location, and Hand Over is handing the cup over to another agent. We use intent-uncertainty-aware human grasp model \({\mathscr{M}}\) to infer the intent Tb in Eq. 5:

$$ T_{b} = \mathcal{M}(\mathcal{H}) $$
(5)

The distribution Pb(R) is used to quantify how much each task is satisfied by a given robot pose with features Ra. We use Naive Bayes robot model \({\mathscr{M}}_{r}\) to produce the robot probability vector of satisfying the task Pb(R) in Eqs. 6 to 8, where μb is the average value for task b, Σb is the covariance matrix for task b, and d is the length of vector Ra.

$$ P_{b}(R)=\mathcal{M}_{r}(R) $$
(6)
$$ P(R_{a}|b)=\frac{1}{\sqrt{det({\varSigma}_{b})(2\pi)^{d}}}e^{-\frac{1}{2}(R_{a}-\mu_{b})^{T}{\varSigma}_{b}^{-1}(R_{a}-\mu_{b})} $$
(7)
$$ P_{b}(R=R_{a})=P(b|R_{a})=\frac{P(R_{a}|b)P(b)}{{{\sum}_{b}^{B}}P(R_{a}|b)P(b)} $$
(8)

Upon developing the target probability vector and the robot probability vector, the intent-based strategy can be constructed based on the intent-based shared control criterion with added constraint, where the objective function is:

$$ \begin{array}{llll} C_{1} = min\left( \frac{1}{2}\sum\limits_{b}(P_{b}(R)-T_{b})^{2}\right)\\ s.t.\quad L_{a}\leq R_{a}\leq U_{a} \quad\forall_{i}\\ norm(R_{a})=1 \quad\forall_{a} \quad needed\, for\, palm\, direction \end{array} $$
(9)

1.2 A.2 Mimic-Based Strategy

If a human operator needs the robot to strictly follow the motion command, unintended errors may occur, but we can still achieve this goal by adding extra constraints to the intent-based strategy. The motion constraints can be explicitly dictated by adding the following set of constraints:

$$ R_{a} = H_{a} \quad \forall_{a} $$
(10)

This will give the operator full control of all features of the robot. The new constraints added to the control diagram ensure the robot follows the human exactly by matching the robot features and human features to mimic the motion. The objective functions are:

$$ \begin{array}{llll} C_{2} = min\left( \frac{1}{2}\sum\limits_{b}(P_{b}(R)-T_{b})^{2}\right)\\ s.t.\quad L_{a}\leq R_{a}\leq U_{a} \quad\forall_{i}\\ norm(R_{a})=1 \quad\forall_{a} \quad needed \, for\, palm\, direction \\ R_{a} = H_{a} \quad \forall_{a} \end{array} $$
(11)

1.3 A.3 Intent-Mimic Hybrid Strategy

We first define λa as the KL divergence between the distribution of each feature:

$$ \lambda_{a} = \mathcal{K}\mathcal{L}(\bar{R_{a}}||\bar{H_{a}})=ln\frac{\sigma_{H_{a}}}{\sigma_{R_{a}}}+\frac{\sigma_{R_{a}}^{2}+(\mu_{R_{a}}-\mu_{H_{a}})^{2}}{2\sigma_{H_{a}}^{2}}-\frac{1}{2} $$
(12)

Additionally, the multivariate normal distribution between two populations can be used to determine the overall divergence between hand configurations in Eq. 13:

$$ \begin{array}{@{}rcl@{}} \gamma &=& \mathcal{K}\mathcal{L}(\bar{R}||\bar{H})=\frac{1}{2}\left( trace({\varSigma}_{H}^{-1}{\varSigma}_{R})+(\mu_{H}-\mu_{R})^{T}\right.\\&&\left.{\varSigma}_{H}^{-1}(\mu_{H}-\mu_{R})-k+ln\frac{|{\varSigma}_{H}|}{|{\varSigma}_{R}|}\right) \end{array} $$
(13)

The formulation results in making the mimic constraint from the previous formulation in the objective function to act as an elastic constraint which allows the robot to bend the rules on mimicking the human. The grasp position is generated by minimizing Eq. 14.

$$ \begin{array}{llll} C_{3} = min\left( \frac{1}{2}\sum\limits_{b}(p_{b}(R)-T_{b})^{2}+\frac{1}{\gamma}{\sum\limits_{a}^{A}}\frac{1}{\lambda_{a}}(R_{a}-H_{a})^{2}\right)\\ s.t.\quad L_{a}\leq R_{a}\leq U_{a} \quad\forall_{i}\\ norm(R_{a})=1 \quad\forall_{a} \quad needed\,for\, palm\, direction \end{array} $$
(14)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tao, L., Bowman, M., Zhou, X. et al. Learn and Transfer Knowledge of Preferred Assistance Strategies in Semi-Autonomous Telemanipulation. J Intell Robot Syst 104, 48 (2022). https://doi.org/10.1007/s10846-022-01596-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-022-01596-2

Keywords

Navigation