The Visual Computer

, Volume 31, Issue 4, pp 423–440 | Cite as

A human-like learning control for digital human models in a physics-based virtual environment

  • Giovanni De MagistrisEmail author
  • Alain Micaelli
  • Paul Evrard
  • Jonathan Savin
Original Article


This paper presents a new learning control framework for digital human models in a physics-based virtual environment. The novelty of our controller is that it combines multi-objective control based on human properties (combined feedforward and feedback controller) with a learning technique based on human learning properties (human-being’s ability to learn novel task dynamics through the minimization of instability, error and effort). This controller performs multiple tasks simultaneously (balance, non-sliding contacts, manipulation) in real time and adapts feedforward force as well as impedance to counter environmental disturbances. It is very useful to deal with unstable manipulations, such as tool-use tasks, and to compensate for perturbations. An interesting property of our controller is that it is implemented in Cartesian space with joint stiffness, damping and torque learning in a multi-objective control framework. The relevance of the proposed control method to model human motor adaptation has been demonstrated by various simulations.


Digital human model Motion control Bio-inspired motor control Virtual reality 


  1. 1.
    Abe, Y., Silva, M.D., Popović, J.: Multiobjective control with frictional contacts. In: Proceedings of ACM SIGGRAPH/EG Symposium on Computer Animation, pp. 249–258. Airela-Ville (2007)Google Scholar
  2. 2.
    Andreiev, N.: A process controller that adapts to signal and process conditions. Control Eng. 38, 38–40 (1977)Google Scholar
  3. 3.
    Astrom, K., Borrison, U., Wittenmark, B.: Theory and application of self-tuning regulators. Automatica 13, 457–476 (1977)CrossRefGoogle Scholar
  4. 4.
    Badler, N.: Virtual humans for animation, ergonomics, and simulation. In: Proceedings of the IEEE Workshop on Non-rigid and Articulates Motion, pp. 28–36 (1997)Google Scholar
  5. 5.
    Bien, Z., Xu, J.: Iterative Learning Control: Analysis, Design. Integration and Applications. Kluwer Academic Publishers, Norwell (1998)CrossRefGoogle Scholar
  6. 6.
    Bradwell, B., Li, B.: A tutorial on motion capture driven character animation. In: Eight IASTED International Conference Visualization, Imaging, and Image Processing, Palma de Mallorca (2008)Google Scholar
  7. 7.
    Bretl, T., Lall, S.: Testing static equilibrium for legged robots. IEEE Trans. Robotics 24, 794–807 (2008)CrossRefGoogle Scholar
  8. 8.
    Buchli, J., Stulp, F., Theodorou, E., Schaal, S.: Learning variable impedance control. Int. J. Robotics 30, 820–833 (2011)CrossRefGoogle Scholar
  9. 9.
    Buchli, J., Theodorou, E., Schaal, S.: Reinforcement learning of full-body humanoid motor skills. In: 10th IEEE-RAS International Conference on Humanoid Robots (Humanoids), pp. 405–410 (2010)Google Scholar
  10. 10.
    Burdet, E., Osu, R., Franklin, D., Milner, T., Kawato, M.: The central nervous system stabilizes unstable dynamics by learning optimal impedance. Nature 414, 446–449 (2001)CrossRefGoogle Scholar
  11. 11.
    Burdet, E., Tee, K.P., Mareels, I., Milner, T.E., Chew, C., Franklin, D.W., Osu, R., Kawato, M.: Stability and motor adaptation in human arm movements. Biol. Cybern. 94, 20–32 (1998)CrossRefGoogle Scholar
  12. 12.
    Castillo, E.: Run-to-run process control: literature review and extensions. J. Qual. Technol. 29, 184–196 (1997)Google Scholar
  13. 13.
    Chaffin, D.: Human motion simulation for vehicle and workplace design. Hum. Factors Ergon. Manuf. 17, 475–484 (2007)CrossRefGoogle Scholar
  14. 14.
    Cheah, C., Wang, D.: Learning impedance control for robotic manipulators. IEEE Trans. Robotics Autom. 14, 452–465 (1998)CrossRefGoogle Scholar
  15. 15.
    Colette, C., Micaelli, A., Andriot, C., Lemerle, P.: Robust balance optimization control of humanoid robots with multiple non coplanar grasps and frictional contacts. In: Proceedings of the IEEE International Conference on Robotics and Automation, Pasadena, pp. 3187–3193 (2008)Google Scholar
  16. 16.
    Commuri, S., Lewis, F.: Adaptive-fuzzy logic control of robot manipulators. In: IEEE International Conference on Robotics and Automation, vol. 3, Minneapolis, pp. 2604–2609 (1996)Google Scholar
  17. 17.
    De Magistris, G.: Dynamic digital human model control design for the assessment of the workstation ergonomics. PhD Thesis, Pierre and Marie Curie University (2013)Google Scholar
  18. 18.
    De Magistris, G., Micaelli, A., Andriot, C., Savin, J., Marsot, J.: Dynamic virtual manikin control design for the assessment of the workstation ergonomy. In: First International Symposium on Digital Human Modeling, Lyon (2011)Google Scholar
  19. 19.
    De Magistris, G., Micaelli, A., Evrard, P., Andriot, C., Savin, J., Gaudez, C., Marsot, J.: Dynamic control of DHM for ergonomic assessments. Int. J. Ind. Ergon. 43, 170–180 (2013)CrossRefGoogle Scholar
  20. 20.
    De Magistris, G., Micaelli, A., Savin, J., Gaudez, C., Marsot, J.: Dynamic digital human model for ergonomic assessment based on human-like behaviour and requiring a reduced set of data for a simulation. In: Second International Digital Human Model Symposium 2013, Ann Arbor (2013)Google Scholar
  21. 21.
    Fitts, P.: The information capacity of the human motor system in controlling the amplitude of movement. J. Exp. Psychol. 47, 381–391 (1954)CrossRefGoogle Scholar
  22. 22.
    Flash, T., Hogan, N.: The coordination of arm movements: an experimentally confirmed mathematical model. J. Neurosci. 7, 1688–1703 (1985)Google Scholar
  23. 23.
    Franklin, D., Burdet, E., Osu, R., Tee, K., Chew, C., Milner, T., Kawato, M.: CNS learns stable, accurate, and efficient movements using a simple algorithm. J. Neurosci. 28, 11165–11173 (2008)CrossRefGoogle Scholar
  24. 24.
    Ganesh, G., Albu-Schaeffer, A., Haruno, M., Kawato, M., Burdet, E.: Biomimetic motor behavior or simultaneous adaptation of force, impedance and trajectory in interaction tasks. In: IEEE International Conference on Robotics and Automation, Anchorage (2010)Google Scholar
  25. 25.
    Gomi, H., Osu, R.: Task-dependent viscoelasticity of human multijoint arm and its spatial characteristics for interaction with environments. J. Neurosci. 18, 8965–8978 (1998)Google Scholar
  26. 26.
    Gribble, P., Ostry, D.: Compensation for interaction torques during single and multijoint limb movement. J. Neurophysiol. 82, 2310–2326 (1999)Google Scholar
  27. 27.
    Grossman, T., Balakrishnan, R.: Pointing at trivariate targets in 3d environments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, pp. 447–454 (2004)Google Scholar
  28. 28.
    Haddadin, S., Albu-Schffer, A., Hirzinger, G.: Requirements for safe robots: measurements, analysis and new insights. Int. J. Robotics Res. 28, 1507–1527 (2009)CrossRefGoogle Scholar
  29. 29.
    Hanavan, E.: A mathematical model of the human body. Wright-Patterson Air Force Base, Report No. AMRLTR-102, pp. 64–102 (1964)Google Scholar
  30. 30.
    Hogan, N.: Impedance control: an approach to manipulation-part I: theory; part II: implementation; part III: applications. Trans. ASME J. Dyn. Syst. Meas. Control 107, 11–24 (1985)Google Scholar
  31. 31.
    Hogan, N.: Mechanical impedance of single- and multiarticular systems. In: Winters, J.M., Woo, S.L.: Multiple Muscle Systems: Biomechanics and Movement Organization. Springer, New York (1990).Google Scholar
  32. 32.
    Hovland, G., Sikka, P., McCarragher, B.: Skill acquisition from human demonstration using a hidden markov model. In: IEEE International Conference on Robotics and Automation, vol. 3, Minneapolis, pp. 2706–2711 (1996)Google Scholar
  33. 33.
    Hsu, L.: Self-oscillating adaptive systems (soas) without limit-cycles. In: Proceedings of the American Control Conference, vol. 13. Albuquerque (1997)Google Scholar
  34. 34.
    Hyman, R.: Stimulus information as a determinant of reaction time. J. Exp. Psychol. 45, 188–196 (1953)CrossRefGoogle Scholar
  35. 35.
    Ioannou, P., Kokotović, P.: Adaptive systems with reduced models. Lecture Notes in Control and Information Sciences, vol. 47 (1983)Google Scholar
  36. 36.
    Jagannathan, S.: Neural Network Control of Nonlinear Discrete-Time Systems. CRC Press Taylor & Francis Group, FL (2006)zbMATHGoogle Scholar
  37. 37.
    Kawato, M., Gomi, H.: A computational model of four regions of the cerebellum based on feedback error learning. Biol. Cybern. 69, 95–103 (1992)CrossRefGoogle Scholar
  38. 38.
    Khatib, O., Sentis, L., Park, J., Warren, J.: Whole-body dynamic behavior and control of human-like robots. Int. J. Hum. Robotics 01, 29–43 (2004)CrossRefGoogle Scholar
  39. 39.
    Kirsch, R., Boskov, D., Rymer, W., Center, R., Center, M., Cleveland, O.: Muscle stiffness during transient and continuous movements of catmuscle: perturbation characteristics and physiological relevance. IEEE Trans. Biomed. Eng. 41, 758–770 (1994)CrossRefGoogle Scholar
  40. 40.
    Lackner, J., Dizio, P.: Rapid adaptation to coriolis force perturbations of arm trajectory. J. Neurophysiol. 72, 299–313 (1994)Google Scholar
  41. 41.
    Li, C., Zhang, D., Zhuang, X.: A survey of repetitive control. In: Proceedings of 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sendai, pp. 1160–1166 (2004)Google Scholar
  42. 42.
    Liu, M., Micaelli, A., Evrard, P., Escande, A., Andriot, C.: Interactive dynamics and balance of a virtual character during manipulation tasks. In: IEEE International Conference on Robotics and Automation, Shanghai, pp. 1676–1682 (2011)Google Scholar
  43. 43.
    L\(\ddot{a}\)mkull, D., Hanson, L., Ortengren, R.: A comparative study of digital human modelling simulation results and their outcomes in reality: a case study within manual assembly of automobiles. Int. J. Ind. Ergon. 39, 428–441 (2009)Google Scholar
  44. 44.
    McIntyre, J., Mussa-Ivaldi, F., Bizzi, E.: The control of stable arm postures in the multi-joint arm. Exp Brain Res 110, 248–264 (1996)Google Scholar
  45. 45.
    Merlhiot, X.: A robust, efficient and time-stepping compatible collision detection method for non-smooth contact between rigid bodies of arbitrary shape. In: Proceedings of the Multibody Dynamics ECCOMAS Thematic Conference (2007)Google Scholar
  46. 46.
    Merlhiot, X.: Extension of a time-stepping compatible contact determination method between rigid bodies to deformable models. In: Proceedings of the Multibody Dynamics ECCOMAS Thematic Conference (2009)Google Scholar
  47. 47.
    Miller, A., Knoop, S., Christensen, H., Allen, P.: Automatic grasp planning using shape primitives. IEEE Int. Conf. Robotics Autom. 2, 1824–1829 (2003)Google Scholar
  48. 48.
    Milner, T., Cloutier, C.: Compensation for mechanically unstable loading in voluntary wrist movement. Exp. Brain Res. 94, 522–532 (1993)CrossRefGoogle Scholar
  49. 49.
    Miyata, N., Kouki, M., Mochimaru, M., Kawachi, K., Kurihara, T.: Hand link modelling and motion generation from motion capture data based on 3d joint kinematics. In: Proceedings SAE International Iowa (2005)Google Scholar
  50. 50.
    Morasso, P.: Spatial control of arm movements. Exp. Brain Res. 42, 223–227 (1981)CrossRefGoogle Scholar
  51. 51.
    Morasso, P., Mussa-Ivaldi, F.: Trajectory formation and handwriting: a computational model. Biol. Cybern. 45, 131–142 (1982)CrossRefGoogle Scholar
  52. 52.
    Morasso, P., Sanguineti, V.: Ankle stiffness alone cannot stabilize upright standing. J. Neurophysiol. 88, 2157–2162 (2002)Google Scholar
  53. 53.
    Occhipinti, E.: Ocra, a concise index for the assessment of exposure to repetitive movements of the upper limbs. Ergonomics 41, 1290–1311 (1998)CrossRefGoogle Scholar
  54. 54.
    Perreault, E., Kirsch, R., Crago, P.: Multijoint dynamics and postural stability of the human arm. Exp. Brain Res. 157, 507–517 (2004)CrossRefGoogle Scholar
  55. 55.
    Porter, J.M., Case, K., Marshall, R., Gyi, D., Sims, R.: Beyond Jack and Jill: designing for individuals using HADRIAN. Int. J. Ind. Ergon. 33, 249–264 (2004)CrossRefGoogle Scholar
  56. 56.
    Prakash, N.: Differential Geometry, An Integrated Approach. TATA McGraw-Hill Publishing Company Limited, New Delhi (1981)zbMATHGoogle Scholar
  57. 57.
    Pratt, J., Torres, A., Dilworth, P., Pratt, G.: Virtual actuator control. In: IEEE International Conference on Intelligent Robots and Systems, pp. 1219–1226 (1996) Google Scholar
  58. 58.
    Salini, J.: Dynamic control for the task/posture coordination of humanoids: toward synthesis of complex activities. Ph.D. thesis, University of Pierre and Marie Curie (2012)Google Scholar
  59. 59.
    Schaub, K., Landau, K., Menges, R., Grossmann, K.: A computer-aided tool for ergonomic workplace design and preventive health care. Hum. Factors Ergon. Manuf. 7, 269–304 (1997)CrossRefGoogle Scholar
  60. 60.
    Sciavicco, L., Siciliano, B.: Modelling and Control of Robot Manipulators. Springer, London (2000)CrossRefzbMATHGoogle Scholar
  61. 61.
    Shadmehr, R., Mussa-Ivaldi, F.: Adaptive representation of dynamics during learning of a motor task. J. Neurosci. 14, 3208–3224 (1997)Google Scholar
  62. 62.
    Si, J., Zhang, N., Tang, R.: Modified fuzzy associative memory scheme using genetic algorithm. In: Proceedings of the 1999 Congress on Evolutionary Computation (CEC), vol.3, pp. 2002–2006 (1999)Google Scholar
  63. 63.
    Slotine, J., Li, W.: Applied Nonlinear Control. Prentice Hall, Englewood Cliffs (1991)zbMATHGoogle Scholar
  64. 64.
    Smith, A.: Does the cerebellum learn strategies for the optimal time-varying control of joint stiffness? Behav. Brain Sci. 19, 399–410 (1996)CrossRefGoogle Scholar
  65. 65.
    Tee, K., Franklin, D., Kawato, M., Milner, T., Burdet, E.: Concurrent adaptation of force and impedance in the redundant muscle system. Biol. Cybern. 102, 31–44 (2010)CrossRefzbMATHGoogle Scholar
  66. 66.
    Todorov, E., Jordan, M.: Smoothness maximization along a predefined path accurately predicts the speed profiles of complex arm movements. J. Neurophysiol. 80, 697–714 (1998)Google Scholar
  67. 67.
    Uno, Y., Kawato, M., Suzuki, R.: Formation and control of optimal trajectory in human multijoint arm movement: minimum torque-change model. Biol. Cybern. 61, 89–101 (1989)CrossRefGoogle Scholar
  68. 68.
    Vignes, R.: Modeling muscle fatigue in digital humans. The University of IOWA. Tech. rep, Center for Computer-Aided Design (2004)Google Scholar
  69. 69.
    VSR Research Group: Technical report for project virtual soldier research. The University of IOWA. Tech. rep, Center for Computer-Aided Design (2004)Google Scholar
  70. 70.
    Wolpert, D., Miall, C., Kawato, M.: Internal models in the cerebellum. Trends Cogn. Sci. 2, 338–347 (1998)CrossRefGoogle Scholar
  71. 71.
    Won, J., Hogan, N.: Stability properties of human reaching movements. Exp. Brain Res. 107, 125–136 (1995)CrossRefGoogle Scholar
  72. 72.
    Yang, C., Ganesh, G., Haddadin, S., Parusel, S., AlbuSchaeffer, A., Burdet, E.: Human like adaptation of force and impedance in stable and unstable interactions. Trans. Robotics 27, 918–930 (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Giovanni De Magistris
    • 1
    Email author
  • Alain Micaelli
    • 1
  • Paul Evrard
    • 1
  • Jonathan Savin
    • 2
  1. 1.CEA, LIST, LSIGif-sur-YvetteFrance
  2. 2.Institut national de recherche et de sécurité (INRS)Vand œuvre-l ès-NancyFrance

Personalised recommendations