Learning to Balance Upright Posture: What can be Learnt Using Adaptive NN Models?

Abstract

Human upright vertical position is unstable. A continuous activation of postural muscles is therefore required to avoid falling down. The problem is complicated by the reduced dimension of the support base (the feet) and by the articulated structure of the human skeleton. Nevertheless, upright posture is a capability, which is learnt in the first year of life. Here, the learning process is investigated by using neural networks models and the reinforcement learning paradigm. After creating a mechanically realistic digital human body, a parametric feed-back controller is defined. It outputs a set of joint torques as a function of orientation and rotational velocity of the body segments. The controller does not have any information either on the model of the human body or on the suitable set of its parameters. It learns the parameters which allow the controller to maintain the vertical position, through trial-and-error (success-fail) by using reinforcement learning. When learning is completed, the kinematics behaviour qualitatively resemble that of real experiments. The amplitude of the oscillations is larger than in the real case; this is due to the lack of any explicit penalization of the oscillations amplitude. The kinematics resembles the real one also when the body has to maintain the vertical upright position facing a tilt or displacement of the support base.