A Real-time Graphic Interface for the Monitoring of the Human Joint Overloadings with Application to Assistive Exoskeletons

  • Marta Lorenzini
  • Wansoo Kim
  • Elena De Momi
  • Arash Ajoudani
Conference paper
Part of the Biosystems & Biorobotics book series (BIOSYSROB, volume 22)


This work presents an intuitive graphic interface to make its users aware of potentially risky body configurations while being exposed to external loads. Employing an algorithm we proposed in a recent work, we estimate the human joint torque overloading caused by an external force. This information is used as an input for the graphical interface to provide the user with an intuitive feedback about the strain on each joint. Hence, the users can be aware of the loading states, react to them accordingly, and minimise the risk of injuries or chronic pain. This graphical interface can help the users learn and achieve more ergonomic configurations during industrial job duties.

1 Introduction

Musculoskeletal disorders (MSD) represents the single largest category of work-related diseases in several industrial countries [1], and impose extremely large costs each year in lost productivity and absence due to sickness. One of the major contributors to MSDs in every industry division is performing repetitive or heavy manufacturing tasks (e.g. lifting, pushing, or pulling on objects), resulted by an excessive and continuous mechanical overloading of body joints [2]. Within this context, an intelligent framework which ensures workers’ well-being in the execution of their daily work is highly needed. To encourage the workers not to assume improper body postures and minimise the risk of work-related injuries, two keys factors must be considered: a method to the real-time monitoring of excessive physical loading on body joints, and intuitive feedback and guidance interfaces to drive them to more ergonomic postures and working conditions.

Extensive literature exists on the assessment of human physical loads associated with lifting or carrying heavy objects in different postures [3], but only off-line procedures are proposed. Alternative research works develop accurate biomechanical models to evaluate the human dynamic behaviour. However, these models may be defined relatively precise with the identification of a large number of parameters [4] or differently, they can be obtained with anthropometric tables, hence, they are not subject-specific and can introduce a large level of uncertainty. These conditions can limit the real-time applicability of such approaches in industrial use-cases. On the other hand, regarding the feedback technologies, different modalities (e.g. visual, auditory and haptic) have been proposed to improve workers’ risk-awareness, but the feedback provided to the users only contains kinematic information [5].

Accordingly, the main objective of this work is to present an intuitive graphic interface which makes humans aware of whole-body kinematic and dynamic information. The dynamic information is expressed in terms of overloadings on the joints caused by an external force, that are estimated by means of a reduced-complexity approach we recently proposed in [6]. Such a feedback system can be combined with assistive technologies like industrial exoskeletons to serve as a support tool in reducing physical loadings while performing heavy tasks. In particular, when using lower-body or upper-body exoskeletons, the loading condition of the rest of the joints can be evaluated by using the proposed interface.

2 Monitoring Method

We recently proposed an algorithm for the real-time estimation of human overloading joint torques1 [6], defined as \(\Delta {\varvec{\tau }}_i\), where i denote the i-th joint. Our method is based on the estimation of the translational displacement of the whole body Centre of Pressure (CoP) in the presence of external forces, calculated from the difference between an estimated one (using an off-line calibrated model) and the measured one (using wearable sensors). If no external interactions of the human with the environment (or object) are in place, the estimation of the CoP vector \(\hat{\mathbf {C}}_{P_{wo}}\) achieved by the human body model, which consider no external load except the body weight, is similar to the measured one \({\mathbf {C}}_{P_{wt}}\). If an external force is applied to the human, the estimated and the measured CoP vectors differ. Accordingly, the overloading joint torque vector is estimated using the contact Jacobians of this CoP displacement along with the difference between the measured vertical ground reaction force (vGRF), which takes into account the effect of external forces, and the estimated one (simply equal to the body weight). A reduce-complexity human model is developed to ensure the real-time applicability of the method and to take specifically into account the body joints which are mainly at risk of injuries. The needed measurements on the human are collected by means of a wearable motion-capture system and foot insole sensors. A complete and detailed explanation of the method can be found in [6].
Fig. 1.

An overview of the graphic interface: in each row the human is represented in three different body configurations, without any tool in the first row and with two different tools in the second and third row, respectively. The level of overloading joint torque is color-coded to denote a high, medium or low value, and illustrated in the main joints of the human body.

3 Visual Feedback Interface

The method we briefly present in Sect. 2 enables the on-line estimation of human joint overloadings due to external forces. To make the human aware of this meaningful information along with his/her current body configuration we take advantage of a graphic interface, shown to the users by means of a dedicated screen. The ROS 3D visualizer RViz is employed for the displaying purpose. Figure 1 illustrates an example of the information that can be provided in real-time to the human: the current body configuration, the level of the computed overloading joint torques, which are color-coded to denote a high (red), medium (orange) or low (green) value and finally, the positions of the measured and the estimated CoP. The three levels of overloading are determined as explained in Table 1. Different tools/objects are represented with a different color/shape in the graphic interface. Depending on their weight, the external force experienced by the human changes and thus the parameters of the overloading joint torques estimation technique can be tuned accordingly. In each row of Fig. 1, three different body configurations are represented, passing from a very risky condition, denoted by high values of overloadings on the joints, to a safe and comfortable one in which the physical effort is minimised, as suggested by the lower values of joints overloadings. Nevertheless, the first row shows the human operating without any tool and thus without the effect of any external forces, therefore, the overloading on the joints is always low. The second and the third row, instead, depict the human operating with two different types of tool, respectively.

Considering that industrial exoskeletons in most cases provide power or support only to the lower limbs or to the upper extremities and in some cases even just to one single-joint, this visual feedback can take into account the body parts which are not covered by the exoskeleton. As long as the human is assisted by the exoskeleton, he/she can view the information provided by the graphic interface, displayed on a screen, and monitor the level of the overloading torques on all his/her own joints. As a result, the human is supported and guided towards more ergonomic and comfortable whole-body configurations by means of a combined contribution of the exoskeleton with the visual feedback.
Table 1.

Stepwise scheme for joint torque overloading level

Overloading level

Control threshold


0\(<\Delta {\varvec{\tau }}_i\le 0.3 {\varvec{\tau }}_{\text {max}_i}\)


0.3\({\varvec{\tau }}_{\text {max}_i}<\Delta {\varvec{\tau }}_i\le 0.6 {\varvec{\tau }}_{\text {max}_i}\)


0.6\({\varvec{\tau }}_{\text {max}_i}<\Delta {\varvec{\tau }}_i\le {\varvec{\tau }}_{\text {max}_i}\)

4 Conclusion

In this work we presented a graphic interface to provide human workers with a visual feedback about the overloadings on their joints while performing a heavy task with an external tool or object. This system can be integrated to assistive technologies such as industrial exoskeletons, making the human aware of the physical loadings exerted on all his/her joints, along with meaningful information regarding the current body configuration, CoP and the object/tool properties. Through this synergistic combination, the workers are assisted and guided while performing heavy or repetitive tasks and the risk of injuries or chronic pain can be reduced. In future works, the proposed visual feedback interface will be combined with robotic assistive systems or considered for rehabilitation purposes.


  1. 1.

    The overloading joint torque refers to the torque induced into the human joint by an external load.


  1. 1.
    Bevan, S.: Fit for work? Musculoskeletal disorders in the European workforce. The Work Foundation (2009)Google Scholar
  2. 2.
    Kumar, S.: Theories of musculoskeletal injury causation. Ergonomics 44(1), 17–47 (2001)CrossRefGoogle Scholar
  3. 3.
    Van Den Bogert, A.J.: Analysis and simulation of mechanical loads on the human musculoskeletal system: a methodological overview. Exerc. Sport. Sci. Rev. 22, 23–23 (1994)Google Scholar
  4. 4.
    Ayusawa, K.: Identifiability and identification of inertial parameters using the underactuated base-link dynamics for legged multibody systems. Int. J. Robot. Res. 33(3), 446–468 (2013)CrossRefGoogle Scholar
  5. 5.
    Azuma, R.: Recent advances in augmented reality. IEEE Comput. Graph. Appl. 21(6), 34–47 (2001)CrossRefGoogle Scholar
  6. 6.
    Kim, W.: A real-time and reduced-complexity approach to the detection and monitoring of static joint overloading in humans. In: International Conference on Rehabilitation Robotics (ICORR), pp. 828–834. IEEE (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Marta Lorenzini
    • 1
    • 2
  • Wansoo Kim
    • 1
  • Elena De Momi
    • 2
  • Arash Ajoudani
    • 1
  1. 1.HRI2 Lab, Department of Advanced RoboticsIstituto Italiano di TecnologiaGenoaItaly
  2. 2.Department of Electronics, Information and BioengineeringPolitecnico di MilanoMilanoItaly

Personalised recommendations