Human Manipulation Segmentation and Characterization Based on Instantaneous Work

  • Anthony RemazeillesEmail author
  • Irati Rasines
  • Asier Fernandez
  • Joseph McIntyre
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1092)


This paper is related to the observation of human operator manipulating objects for teaching a robot to reproduce the action. Assuming the robotic system is equipped with basic manipulation skills, we focus here on the automatic segmentation of the observed manipulation, for extracting the relevant key frames in which the manipulation is best described. The segmentation method proposed is based on the instantaneous work, and presents the advantage of not depending on the force and pose sensing locations. The proposed approach is experimented with two different manipulation skills, sliding and folding, under different settings.


Teaching by demonstration Manipulation segmentation 



Supported by the Elkartek MALGUROB and the SARAFun project under the European Union’s Horizon 2020 research & innovation programme, grant agreement No. 644938. The authors would like to thank Dr. Pierre Barralon for the fruitfull discussions that led to the segmentation approach presented here.


  1. 1.
    Almeida, D., Karayiannidis, Y.: Folding assembly by means of dual-arm robotic manipulation. In: IEEE ICRA, pp. 3987–3993, May 2016Google Scholar
  2. 2.
    Billard, A., Calinon, S., Dillmann, R., Schaal, S.: Robot Programming by Demonstration, pp. 1371–1394. Springer, Berlin (2008)Google Scholar
  3. 3.
    Calinon, S.: Robot Programming by Demonstration. A Probabilistic Approach. EPFL Press, Lausanne (2009)Google Scholar
  4. 4.
    De Schutter, J., De Laet, T., Rutgeerts, J., Decré, W., Smits, R., Aertbeliën, E., Claes, K., Bruyninckx, H.: Constraint-based task specification and estimation for sensor-based robot systems in the presence of geometric uncertainty. Int. J. Robot. Res. 26, 433–455 (2007)CrossRefGoogle Scholar
  5. 5.
    Kormushev, P., Calinon, S., Caldwell, D.G.: Imitation learning of positional and force skills demonstrated via kinesthetic teaching and haptic input. Adv. Robot. 25(5), 581–603 (2011)CrossRefGoogle Scholar
  6. 6.
    Kovács, E.: Rotation about an arbitrary axis and reflection through an arbitrary plane. In: Annales Mathematicae et Informaticae, vol. 40, pp. 175–186, January 2012Google Scholar
  7. 7.
    Kragic, D., Gustafson, J., Karaoguz, H., Jensfelt, P., Krug, R.: Interactive, collaborative robots: challenges and opportunities. In: IJCAI-18, pp. 18–25 (2018)Google Scholar
  8. 8.
    Murray, R.M., Sastry, S.S., Zexiang, L.: A Mathematical Introduction to Robotic Manipulation, 1st edn. CRC Press, Boca Raton (1994)zbMATHGoogle Scholar
  9. 9.
    Pedersen, M., Nalpantidis, L., Andersen, R., Schou, C., Bogh, S., Kruger, V., Madsen, O.: Robot skills for manufacturing: from concept to industrial deployment. Robot. Comput.-Integr. Manuf. 37, 282–291 (2016)CrossRefGoogle Scholar
  10. 10.
    Perez-D’Arpino, C., Shah, J.A.: C-learn: learning geometric constraints from demonstrations for multi-step manipulation in shared autonomy. In: IEEE ICRA, pp. 4058–4065, May 2017Google Scholar
  11. 11.
    Piperagkas, G.S., Mariolis, I., Ioannidis, D., Tzovaras, D.: Key-frame extraction with semantic graphs in assembly processes. IEEE Robot. Autom. Lett. 2(3), 1264–1271 (2017)CrossRefGoogle Scholar
  12. 12.
    Ureche, A.L.P., Umezawa, K., Nakamura, Y., Billard, A.: Task parameterization using continuous constraints extracted from human demonstrations. IEEE Trans. Rob. 31(6), 1458–1471 (2015)CrossRefGoogle Scholar
  13. 13.
    Villani, V., Pini, F., Leali, F., Secchi, C.: Survey on human-robot collaboration in industrial settings: safety, intuitive interfaces and applications. Mechatronics 55, 248–266 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Anthony Remazeilles
    • 1
    Email author
  • Irati Rasines
    • 1
  • Asier Fernandez
    • 1
  • Joseph McIntyre
    • 2
  1. 1.TECNALIASan SebastianSpain
  2. 2.Ikerbasque Research FoundationBilbaoSpain

Personalised recommendations