1 Introduction

In the industry, physical exertion is required for productivity to remain at an acceptable level. In most occupations, the manual tasks may not be particularly demanding but they will repeat many times during a work shift. This can cause microtrauma to muscles and soft tissues to accumulate, leading to permanent damage. Injuries caused on the muscles, tendons, and the skeletal system in general due to work-related conditions are collectively called work-related musculoskeletal disorders (WMSDs) and they are prevalent across a variety of occupations (Antwi-Afari et al. 2017).

WMSDs are not clinically different from other musculoskeletal disorders, such as overuse syndrome on the prosthesis users’ population, in the sense that their cause is that there is not enough time for the tissues to recover from repeated motions. Indeed, increased and/or repetitive load on the muscles can cause microtears (Nakama et al. 2007), if the repetitive microtrauma overwhelms the regeneration of the tissue, then permanent damage sets in (Verdon 1996).

In general, musculoskeletal disorders can appear in many different fields (instrument players, prosthesis users, athletes etc.) with prevention being the best approach (Gambrell 2008). Since WMSDs are work-related, a clear and concise assessment of the postures that are repeated and stress the body during work is required.

The ergonomics risk factors that can lead to WMSDs are repetition, force, awkward postures, vibrations, contact stress, static loading (Jaffar et al. 2011). The existence of any or all of those aspects doesn’t necessarily lead to WMSDs if their occurrence is not very frequent. As it was mentioned, it’s the long exposures to these risk factors and the insufficient rest time in-between that can lead to WMSDs. Moreover, it is not apparent which occupations have a higher probability to cause WMSDs. For example, employees in the food services have reported higher discomfort levels than those in the manufacturing industry (Oakman and Chan 2015). Such an unintuitive observation highlights that ergonomics risk factors need to be measured in an objective way.

To evaluate ergonomics risk factors in the workplace various approaches have been tried, but they can be grossly separated in two categories, scoring based systems by an observer and\(\backslash \)or self-reporting. Scoring based systems require a questionnaire to be filed by an observer that evaluates the postures and assign a number. There are many different methods (Valero et al. 2016) with the rapid upper limb assessment (RULA) (McAtamney and Nigel Corlett 1993) being the most widely used. RULA is focused on the joint angles of the torso, neck, shoulders, elbow, and wrist positions of the most common postures, and whether the person is standing up during a task. However, RULA is subjective to the observer’s judgment (Antwi-Afari et al. 2017) and the most common posture might be difficult to identify (Dockrell et al. 2012). Despite that, it is very popular because it does not require specialized equipment or an extremely skilled observer while it does not disrupt the working environment and it’s very fast. Another observational method is the occupational repetitive actions (OCRA) index (Occhipinti 1998) that considers the repetition of each action during a work shift as well as the forces that the human body is exposed to, and the general posture. Like RULA, its main goal is to produce a single number (index) to assess the severity of the ergonomic hazard. Sana et al. (Sana et al. 2019) used both RULA and OCHRA as inputs to a genetic algorithm in order to optimize the workplace and the task rotation that a worker should do in a virtual environment.

Direct posture measurement methods have also been proposed but they have not been adopted for industrial use. These methods use motions sensors to quantify biomechanics properties of the human motion and do not rely on subjective assessments. However, there has not been a consensus regarding the measurements that should be used for WMSDs risk factors’ assessment. Since RULA uses joint angles, the most logical step would be to measure the angles during each posture.

Valero et al. (Valero et al. 2016) tracked the activity of construction workers. To do that, they developed a system that used inertia measurement units (IMUs) to record the relative acceleration of each segment and extract the angles. While their system was fairly robust, their focus was on the lower back and legs and therefore the hand motions were ignored. However, the biggest limitation of the study was that the strain on the body that does not come from awkward postures was not considered. For example, static loads, vibrations, and contact forces are ergonomic risk factors but they do not particularly affect the joint angles. As a results, this approach cannot distinguish between individuals that are under different external loads.

Computer vision (CV) has also been employed for human body tracking. Open source modules, such as OpenPose by Cao et al. (Cao et al. 2017), allow the tracking of multiple individuals and the computation of the person’s kinematic model and joint angles. This is done by using a convolutional neural network (CNN) that is trained to detect the human anatomy. Once the network has been trained, OpenPose can achieve real-time body tracking. Cao et al.’s scope was to develop the algorithm and released it for public use. As such, they did not perform any ergonomics related biomechanics experiment but their tool has great potential. Mehrizi et al. (Mehrizi et al. 2018) proposed a model that could extract the joint angles of twelve healthy males performing symmetric lifting tasks. However, Mehrizi et al. focused on the creation of an accurate measuring apparatus and did not examine any biomechanics regarding ergonomics either. Similarly, Kanazawa et al. (Kanazawa et al. 2017) was able to extract the human body’s pose and shape using computer vision. Like Cao et al. and Mehrizi et al., Kanazawa et al. did not focus on biomechanics analysis. One of the few ergonomics assessment experiments was done by Manghisi et al. (Manghisi et al. 2017). They used a depth camera (Kinect v2, Microsoft Co, Redmond, WA,USA) to detect the human posture and classify the RULA score. Their results showed a “perfect” agreement with RULA scores from human experts. However, their method did not address RULA limitations or attempted to extract more quantifiable data beyond the ones the depth camera’s software was providing. Currently, CV for human tracking is a very active field but the main focus is to refine the algorithms to be more accurate, efficient, and robust. While work has been done in the field of biomechanics using CV, there are very few benefits in using a markerless video based MoCap method when more accurate alternatives can be used. Of course, there are very few doubts that when the CV algorithms mature even more, they will become the dominant MoCap apparatus.

MoCap is not the only method that has been used to monitor human motion for ergonomic assessment. Antwi-Afari et al. (Antwi-Afari et al. 2017) used surface electro-myograph (sEMG) to collect the muscle activation signals. Using sEMGs, it was possible to identify which muscles were affected the most and were fatigued when lifting a weight. However, Antwi-Afari et al. found inconsistent results with the literature between lifting postures. The inconsistencies were attributed to the different experimental protocol that was used. This implies that even though muscles are directly affected by improper postures, if the joint angles are not considered, it is difficult to have a comprehensive evaluation.

The main issue with all those different approaches is that it is not entirely clear what is the primary component that needs to be measured in order to monitor the probability of WMSDs to appear. Observational methods have limitations, but they are not necessarily less accurate than biomechanics-based approaches because the direct measurement methods may not record the most useful property of the posture. As a result, attempts to optimize the workplace environment, such as the one done by Sana et al., by using ambient intelligence methods, hinge entirely on the ergonomic criteria used. This work proposes a method to objectively evaluate the kinematic and kinetic aspect of a worker’s motion in order to build a foundation which can provide richer data to be used for ergonomic evaluation.

Fig. 1
figure 1

Flow chart of the proposed work

Full size image

This paper uses angles and torques of the joints during tasks in an effort to assess the overall behavior of the human body. The concept is to compare each joint’s kinematic and kinetic contribution during the whole performance. While motions do imply forces acting on a body, there are situations where high forces are present to prevent motion (e.g. holding a weight). Similarly, joints can exhibit high mobility without the accumulation of high forces. Because of that, both joint motions and torques is examined during a task. Additionally, an individual trial is compared with the average performance to detect deviations. Using the proposed methodology, it is possible to identify a performance that varies from the average, the different contributions of each joint, and the a direct examination of the individual joint that raises concerns. Figure 1 gives an overview of the steps involved for this work.

2 Methods

2.1 Data collection

Three males performed upper body range of motion (ROM) tasks that were recorded using a suit that had 50 IMU sensors (Nansense inc., Los Angeles, CA, USA) in a lab environment. Additionally, in order to examine more realistic data, MoCap recordings were also done in a TV manufacturing factory (Arçelik A.Ş., Istanbul, Turkey). Four workers were recorded (one female, three males) in the span of two days. The RULA score was assigned following the proper guidelines (McAtamney and Nigel Corlett 1993). The joint angles were calculated by the IMUs and custom software was written in Matlab (Mathworks, Nattick, MA, USA) to calculate the joint torques and carry out complementary calculations. This setup was used instead of a CV algorithm because high accuracy and immunity to occlusions can be guaranteed by using wearable sensors. This is especially true in an industrial environment where people can walk in and out of the field of view of a camera. However, please note that the purpose of this work is to examine a skeleton and its postures. Whether this skeleton was created using CV or IMUs does not affect the methods that are discussed in this paper.

The ROM tasks the participants performed were forearm flexion/extension, forearm rotation, and shoulder flexion/extension. The forearm flexion/extension task required the participants to stand, flex their forearms to a mid-position somewhere in the middle of their ROM, hold them for a couple of seconds, then flex their forearms until they are perpendicular to the ground with their fingers, and finally, extend them to their original position. The forearm rotation had the person standing up with the elbows bend at approximately 90°, the participant was then instructed to rotate their wrist to the limits of their ROM. Lastly, during shoulder flexion/extension, the person is standing up raising their arms in front of them and pausing in three positions. In the first position the arms are below the shoulder level, in the second position, the arms are parallel to the ground, and in the last position, the arms are above the shoulder level. Then the participant will bring their arms down to the neutral position. Figure 2 shows examples of the recordings for the ROM tasks.

Fig. 2
figure 2

The ROM tasks, a is forearm flexion where the elbows bend, b is forearm rotation where each ulna and radius rotate the hands, c is shoulder flexion where the hands are lifted above the head

Full size image

The common characteristic of those tasks is that they are simple and employ mostly a single joint. These motion data were used to allow the examination of RULA scoring and joint torques on simpler motions. Data from an actual TV assembly line were also recorded and analyzed. The more complex dataset was used to show how joint angles and torques can be used in a realistic setting and give more information about the WMSDs risk factors than the RULA scoring.

The industrial tasks under consideration here are a TV packaging task and a TV assembling task. During the packaging task, the worker stacks TVs on a wooden pallet and then wraps them with a plastic membrane. The packaging task is mostly bi-manual since the worker is using both hands simultaneously frequently. For the assembling task, the worker is screwing boards of electronic circuits on the TV’s frame. The right hand is used to operate an electronic screwdriver while the left hand is holding the circuit board. Both tasks were recorded during normal operation of the production line and without giving specific instructions. It proved impossible to have all the workers perform the same tasks because it was necessary to keep the production line intact and avoid having individuals doing tasks that they were not familiar with. As a result the female worker performed 54 assembling trials and one of the male workers performed 22 assembling trials for a total of 76 assembling trial recordings. For the packaging task the three male workers were used because those were the ones who are normally working on packaging the TVs. One worker performed 5 trials, a second worker did 18 and the last one did 17 for a total of 40 packaging trials. Figure 3 shows an example of the two industrial tasks that were considered in this work.

Fig. 3
figure 3

Industrial tasks, a, b show a frame from the assembling task in video and MoCap. c, d show a frame from the packaging task. Note that both tasks involve a lot more motions between the main ones that are shown in this figure

Full size image

In general, the industrial task are characterized as being longer and involving a lot of motions. This makes them challenging to identify the most common posture. It became apparent that even if the dominant motions are identified, examining only them, leaves a relatively large amount of motions that are ignored. As such, an RULA assessment might be inaccurate since it would only examine a portion of the complete work cycle.

2.2 RULA scoring

RULA assigns three scores for each posture. The first score is for the arms, the second is for the neck and torso, and the third one is given on the legs for the sitting or standing position. The RULA score that is assigned is proportional to the bending of a joint. For example, starting with the hands perpendicular to the ground pointing downwards, will get a score of one. If the shoulders start extending, then the score will get higher as the arms move upwards until they reach their highest position and get a score of four. Similarly, all joints get a higher score as they deviate from the anatomical position (McAtamney and Nigel Corlett 1993). In general, as more joints bend, the RULA score will become higher. However, RULA is not particularly concerned with duration or repetitions, in the sense that it only adds one extra point if the duration is for more than one minute or it is repeated more than four times in a minute. Whether the posture is being held for two minutes or five is inconsequential to the RULA scoring. However, the greatest benefit of this system is that the whole process takes only a few seconds for each posture.

2.3 Joint torques calculation and comparison of multiple joints

Calculating the joint torques requires certain steps. First is the creation of a kinematic chain that represents the human upper body. Then, the body can be treated as a multi-body link, similar to a robot manipulator, and calculate the joint torques between each segment. The model has 24 degrees of freedom (DoFs) and encompasses the torso, both arms, and the head. Each joint is modeled as a three DoFs Euclidean coordinate frame.

For this work, a model of the upper human body was created, starting from the pelvis. The model splits in three multi-body chains: the right arm chain, the left arm, and the head chain. The coordinate frame of each joint is described using Lie algebra. For every axis, a twist is established. A screw axis is a 6x1 vector that describes the linear and angular velocity(Selig 2005; Lynch and Park 2017). Equation 1 shows how a single screw axis is defined mathematically.

$$\begin{aligned} {\xi }_{i}=\begin{bmatrix} {\dot{X}} \\ {\dot{\varOmega }}\\ \end{bmatrix} = \begin{bmatrix} {\dot{x}} \\ {\dot{y}} \\ {\dot{z}} \\ \dot{\omega _x} \\ \dot{\omega _y} \\ \dot{\omega _z} \\ \end{bmatrix} \end{aligned}$$
(1)

where \(\xi _i\) is a screw axis of a DoF i with respect to a global frame, \({\dot{X}}\) is the linear velocity of the axis in the three Cartesian coordinates \([{\dot{x}},{\dot{y}},{\dot{z}}]\), and \({\dot{Omega}}\) is the angular velocity \([\dot{\omega _x},\dot{\omega _y},\dot{\omega _z}]\) of the screw axis. Every DoF is described mathematically using a screw axis, and each kinematic chain is a sequence of \(\xi _i\) vectors. In essence, each axis contributes certain linear and angular velocities to the system, by chaining together the appropriate DoFs, an accurate kinematic model is defined. This mathematical framework allows for complex kinematic chains to be created with relative ease because it does not need to abide by any constraint, unlike other methods, such as the widely used in robotics Denavit-Hartenberg parameters that are very strict in their frame assignment (Menychtas 2018).

Once the kinematic chains are constructed, the inertia properties of each body segment are calculated using regression tables (Dumas et al. 2007). Those tables consider a person’s gender and weight to regress the weight of each body part and the inertia properties. Though there are subject specific deviations that this methods does not account for, mire accurate methods would require medical imaging and reliance on simulation (Rossi et al. 2016; Durkin and Dowling 2003), something that was impractical to do for the data recorded. As a result, regression equations were used since they are considered a valid practical approach.

With the kinematic chains created and the inertia properties established, the Newton-Euler algorithm is used to calculate the joint torques (Featherstone 2008). This algorithm is used for multi-body links to calculate the forces and the torques. In general, it is consisted of two propagation loops. The first loop propagates from the base of the kinematic model to its end-effector and calculates the accelerations, velocities, and forces that each joint contributes to the system. The second loop propagates backwards towards the base and calculates the torques each joint generates.

The most important mathematical entity for the whole algorithm to function is the adjoint representation (Selig 2005; Lynch and Park 2017). This \(6-by-6\) matrix maps a vector, such as a screw axis, from one coordinate frame to another. Equation 2 defines the adjoint representation matrix.

$$\begin{aligned} {}_{i-1}{Ad}_{i}=\begin{bmatrix} {R} &{} {\widehat{p}}{R} \\ O_{3\times 3} &{} {R} \end{bmatrix} \end{aligned}$$
(2)

where i is the current frame, \(i-1\) is the previous one, R is the standard \(3\times 3\) rotation matrix between frames, and \({\widehat{p}}\) is the origin point of the current coordinate frame arranged in a \(3-by-3\) skew-symmetric matrix. Equation 2 will map a vector from one frame to another, allowing the projection of velocities, accelerations and forces across each segment of the chain.

For kinetic components, the equivalent adjoint matrix is shown in Eq. 3 and it is used to map forces and torques between frames.

$$\begin{aligned} {}_{i-1}{AdF}_{i}=\begin{bmatrix} {R} &{} O_{3\times 3} \\ {\widehat{p}}{R} &{} {R} \end{bmatrix} \end{aligned}$$
(3)

To reverse the mappings the inverted adjoint matrices can be used. Equations 4 and 5 show the formulas to invert the matrices for kinematics and kinetics.

$$\begin{aligned} {}_{i-1}{Adinv}_{i}= & {} \begin{bmatrix} {R}^T &{} {\widehat{p}}{R}^T \\ O_{3\times 3} &{} {R}^T \end{bmatrix} \end{aligned}$$
(4)
$$\begin{aligned} {}_{i-1}{AdFinv}_{i}= & {} \begin{bmatrix} {R}^T &{} O_{3\times 3} \\ {\widehat{p}}{R}^T &{} {R}^T \end{bmatrix} \end{aligned}$$
(5)

with the superscript T denoting the transpose of a matrix. Without going into details here, The adjoint matrix has the same function as a standard \(4-by-4\) transformation matrix. The only difference is that the adjoint operates on \(6-by-1\) vectors that either contain the linear and angular velocity of the segment that is being investigated, or its torques (angular forces) and linear forces.

The adjoint representations are used extensively in the Newton-Euler algorithm to project velocities, accelerations, forces, and torques across the whole kinematic chain. As such, each body’s dynamics can be calculated separately. This allows the analysis of complex configurations, without making the equations more complicated.

As it was mentioned, the first iteration will start from the base and propagate the velocities, and acceleration, as well as the forces and torques along the kinematic chain. The equations that are used for the forward loop are shown in Eqs. 6, 7, 8.

$$\begin{aligned} v_i= \,& {} {}_{i-1}{Ad}_{i}v_{i-1}+\xi _i{\dot{q}}_i \end{aligned}$$
(6)
$$\begin{aligned} \alpha _1= \,& {} {}_{i-1}{Ad}_{i}g+\xi _i\ddot{q}_i+\widehat{v_i}\xi _i{\dot{q}}_i \end{aligned}$$
(7)
$$\begin{aligned} F_i= \,& {} M_i\alpha _1+\widehat{v_i}^{T}M_iv_i \end{aligned}$$
(8)

where \(v_i\) is the velocity vector of the current segment i, \({\dot{q}}_i\) is the joint angle velocity, \(\alpha _i\) is the acceleration vector, of the current segment i, g is the acceleration due to gravity, \(\ddot{q}_i\), is the acceleration of the joint angle, \({\widehat{v}}\) is the velocity vector arranged in a \(6-by-6\) skew symmetric matrix, \(F_i\) is the \(6-by-1\) force vector, and \(M_i\) is the \(6-by-1\) mass matrix of the segment.

Equations 9 and 10 are used during the backwards loop to account for the reaction forces and calculate the torque on each joint.

$$\begin{aligned} F_{i-1}= \,& {} F_{i-1}+{{}_{i-1}{AdFinv}}^{T}_{i}F_{i} \end{aligned}$$
(9)
$$\begin{aligned} \tau _i= \,& {} \xi _i^TF_i \end{aligned}$$
(10)

where \(\tau _i\) is the torque of the joint. By the end of both loops every joint torque will be calculated based on the current state of the body. In general, the kinetic results have the same patterns as the kinematic components, however, incorporating the gravity vector gives a more complete image regarding the effort that is required. In essence, a stationary posture can require higher effort if it counters gravity despite no motion taking place.

2.4 Extracting the usage percentage of each joint

To compare the motions and torques of all the joints the integrals of the series of angles and torques for each joint was calculated as shown in Eq. 11

$$\begin{aligned} S_i = \int s_i(t) dt \end{aligned}$$
(11)

where \(s_i(t)\) is the signal (either joint angles or torques) of a DoF i and t is time.

Integrating with respect to time will result in one value that is directly related to the overall behavior of the joint for the whole task. In order to make comparison between joints easier, the integrals will be normalized with respect to the highest integral of their respective series. The resulting values are presented as percentages as shown in Eq. 12

$$\begin{aligned} S_i\% = \frac{S_i}{S_{max}} \times 100\% \end{aligned}$$
(12)

where \(S_i\%\) is the percentage that a joint was utilized (either kinematically or kinetically) during the whole task with respect to the rest of the body. The joint with the highest contribution has \( 100\% \) usage and the rest of the joints are scaled accordingly. This gives a perspective regarding the motion of each joint throughout the task both individually and as part of the whole body.

The last problem that needs to be resolved is that heavier segments, such as the torso, have inherently larger torques even when there are no particularly great forces acting on them. This makes the kinetic comparison between each joint problematic since a weight-supporting joint can withstand greater forces. To resolve the issue, the resulting torques are divided over the weight they are under, this includes all the preceding segments as well. For example, the wrist joint will be normalized for the weight of the hand while the elbow joint will be normalized for the weight of the forearm and the hand. The torques-per-kilogram that are found can be compared across different joints and identify the joints that are under excessive strain due to motion rather than the weight they carry.

3 Results

The forearm flexion/extension task required the participants to stand, flex their forearms to a mid-position somewhere in the middle of their ROM, hold them for a couple of seconds, then flex their forearms until they are perpendicular to the ground with their fingers, and finally, extend them to their original position. Even though the mid-position was different for each individual, the RULA score was 2. However, the torques give a much more detailed insight.

Fig. 4a–c show the motions of flexion/extension of the left forearm for each participant. The graphs are dual axes graphs with degrees as their unit on the left vertical axes and Nm on the right. Both joint angles and joint torques are shown. The graphs presented are dual axes graphs with joint angles on the left vertical axis and joint torques on the right. In Fig. 4b, c, the torque graphs is not be aligning with the joint angle graphs, However, this is because of how the axes are scaled, the relevant information is how the torques’ profiles follow the joint angles during this motion. Also, note how the RULA method gives the same score to all those different motion profiles.

Fig. 4
figure 4

Forearm flexion ROM from the three subjects

Full size image

Fig. 5a, b show the forearm rotation of the left arm for two of the participants. The third person exhibited the same behavior so it is not shown to avoid repeating data. In this ROM task, the RULA scoring is 3, which makes it a more ergonomically improper task than the previous. However, the torques remain almost zero for the duration of the trials.

Lastly, Fig. 6a, b show the shoulder flexion and shoulder abduction motions during the shoulder flexion/extension task. They are both shown because during this task, the motion is shared between those to joints, i.e. two axes of rotation of the shoulder are used to raise each arm. The RULA score for this task is 5. In this task, the torques of axis of rotation for the shoulder flexion are lower when the arm goes above the shoulder level as shown in Fig. 5a.

Fig. 5
figure 5

Forearm rotation from the two subjects

Full size image

To sum up the ROM results, during motion the joint torques follow the joint angles most of the times. However, there are cases where the torques are mostly unaffected by motion or they follow a different trajectory. This is related to the orientation of the rotation axis of the joint and the force of gravity. If the motion is not acting against gravity or the body segment ends up perpendicular to the ground (parallel to the force of gravity), then the torques are diminished.

Fig. 6
figure 6

Shoulder extension and shoulder abduction during the same trial

Full size image

The next part is demonstrating two real-life tasks in a TV assembly factory. The first one is packaging and the second one is an assembling task. Both of those tasks were recorded without any particular instructions, i.e. the tasks were performed in a natural manner with motions that may not be related to the task explicitly (Figs. 7, 8).

Figure 9a, b show the elbow flexion of both of the arms during a TV packaging trial. In this task, both arms work in conjunction but they do not necessarily mirror each other. In both of those graphs, the torques follow the joints’ profile for the most part, but there are instances where the joint angles and the corresponding torques have a different profile.

Fig. 7
figure 7

Left elbow flexion during the packaging task

Full size image
Fig. 8
figure 8

Right elbow flexion during the packaging task

Full size image
Fig. 9
figure 9

The elbow motions during the packaging task. Both joint angles and joint torques are shown

Full size image

Figure 10a, b show the left and right elbow flexion during an assembling trial of a single worker. Like the packaging task, there is no mirroring of the motions between the two arms.

Fig. 10
figure 10

The elbow motions during the packaging task. Both joint angles and joint torques are shown

Full size image

Even though Figs. 9a–10b give a lot of information about the trial through its duration, they can be overbearing. More importantly, they only show one joint each. This is where the simplicity of RULA can be seen explicitly. Indeed, it is very impractical to go over all the graphs for all the joints to assess the strain on the body.

To address this limitation, Figs. 11, 12 show the kinematic and the kinetic contribution of each joints during the packaging and the assembling task of the same worker as shown Figs. 9a–10b. It is possible to see which joint angles (DoFs) exhibit more motion and where the torque is higher across each task. After the torque normalization, the torso is not biased to exhibit the higher strain on heavier segments and a more accurate description can be given. The graphs show the arms being much more involved both kinemantically and kinetically which is what it was expected.

Fig. 11
figure 11

Normalized integrals during the packaging task

Full size image
Fig. 12
figure 12

Normalized integrals during the assembling task

Full size image

The packaging task in Fig. 11 shows that both shoulders contribute the same amount kinematically, but the right shoulder has more contribution kinetically. In essence, the right shoulder rotation and abduction/adduction are the DoFs with the highest torques per kilogram during the whole task. Note that this task is bilateral, both hands are used in unison.

The assembling task in Fig. 12 shows that the right forearm flexion/extension and the right wrist flexion/extension perform large motions without particularly high torques. The left shoulder however, has high torques without large motions, which means those DoFs are used to stabilize the left arm while the more distal joints (elbow and wrist) perform some finer motions.

Fig. 13
figure 13

Kinematic and kinetic comparison of a single trial of packaging against the average performance

Full size image

The kinematic and the kinetic contributions of each recorded trial can also be compared with the average contributions of the rest of the trials. Figure 13a, b show such comparisons for the packaging task. Note that the trial being examined in these figures is the same as the one in Fig. 11 and it is not part of the dataset that was used for the average values. Out of the 40 packaging trials, 35 were used to extract the average kinematic and kinetic contributions. In general, the contributions from the trial follow the pattern of the average performance, however, there are exceptions. The torso rotation, the right shoulder abduction/adduction, and the left shoulder abduction/adduction are more mobile than the average as Fig. 13a shows. At the same time, the left shoulder rotation is more stationary than the corresponding average. This could imply that the person used the torso rotation to substitute some motions from the left shoulder. The kinetic contributions’ comparison in Fig. 13b give more information to believe that. The left shoulder rotation and the left shoulder abduction/adduction have lower torques from the average for the whole trial. So it can be assumed, with relative certainty that the person used their torso more to minimize the forces on their shoulder. Of course further analysis and validation is required for a definitive answer.

Fig. 14
figure 14

Kinematic and kinetic comparison of a single trial of assembling against the average performance

Full size image

An assembling trial is compared with the average performance in Fig. 14a, b. As before, the trial that is used was examined in isolation from the rest of the dataset in Fig. 12 and it was not used to extract the average values. Out of the 76 assembling trials, 71 were used to extract the average kinematic and kinetic contributions. A major difference the average performance and the individual’s motions is in the right wrist flexion/extension. This DoF is utilized more than the average both kinematically and kinetically. There are others kinematic discrepancies in the left wrist abduction/adduction and left wrist flexion/extension, but they do not appear to increase the torques of that joint but they are responsible for the increased kinetic contribution of the left shoulder abduction/adduction.

The reason to compare the tasks on their own, such as in Figs. 11, 12, is to identify if there are joints that are used more than necessary. In a sense, the purpose is to identify if the motions and the forces are being distributed evenly across the body. In an ideal scenario all joint will have kinematic and kinetic contributions close to \(100\%\). This would mean that all joints contribute evenly, of course this is not a possible in reality, but it gives a direction on how to examine motions from an ergonomics point of view. Comparing the joints’ contributions of a specific trial with the averages allows the observer to see the individual’s deviation from the normal performance and where exactly those deviations happen.

4 Discussion and conclusions

During the forearm flexion/extension ROM task, the torques follow the same profile as the joint angles (Fig. 4). Even though all three participants got a RULA score of 2 for the task, the motions they performed, and the torques that their joints produced were different. This was expected, but it highlights how much information is being obscured when RULA is used. By inspecting Fig. 4 it is possible to see the repetitions, the positions, and the strain the participants had on their joints. This motion works against gravity and as a result, the joint torques have a similar profile with the joint angles. Even though this was a simple task, it is possible to compare the three participants.

Forearm rotation ROM task in Fig. 5 show the axis of rotation exhibiting large motions. However, the torques in both subjects have almost zero magnitude. This happens because this motion does not work against gravity and therefore only the relatively slow velocity and acceleration are the source of angular forces. The relevance of this case is that there are large motions taking place but the corresponding torques are very low. However, the RULA scoring system gives this task a value of 3, which is higher than the score of the forearm flexion/extension ROM task. This extra one point is given specifically because the hands are rotating. However, even though there are not substantially higher forces acting on the axis of rotation, the RULA scoring treats this task as potentially more dangerous.

Lastly, the shoulder extension ROM tasks uses two motions to be performed. In essence, the task is not a rotation about a single axis but rather a series of different motions that aggregate to a smooth movement of the arms going up to the eye level. Figure 6 shows the same person lifting their arms and how the two DoFs work in tandem. The joint angles are in sync and they increase as the arms go higher. However, the torques of the shoulder extension DoF are reduced past around the angle of 65°. This happens because the lever arm is decreasing for that axis as the arm goes higher which causes the gravity to have less of an impact. The lever arm does not change much for the shoulder abduction resulting in a more consistent torque profile that increases along with the joint angle. The RULA score is 5, making this task the most ergonomically improper ROM task that is examined here. While this is accurate, and the torques are higher than the ones calculated for the two previous tasks, it is challenging to compare the between without having any data. Even though RULA is more efficient, it is not based on any measurement and therefore information can be obscured. Of course, the argument can be made that the proposed approach can lead to a large amount of data that makes it hard to see the bigger picture.

For the real-life scenarios a packaging and an assembling task on a TV production line were used. The elbows’ flexion during packaging in Fig 9a, b are quite dense because it shows natural performance. The only observation that can be made is that the joint torques will follow the joint angles for the majority of the task but not always. Figure 10a, b are a bit simpler because the task is not as complex. The general pattern between angles and torques is exhibited again though in those cases it can be argued that the inclusion of torques does not give any more information than the joint angles because of how difficult it is to interpret them. However, the most important drawback of those graphs is that they are providing details about specific joints but not the bigger picture of each task.

The bar graphs in Figs. 11, 12 solve the issue by showing a “bird’s-eye” view of the tasks. The kinematic contribution for each joint can be compared with its respective kinetic contribution as well as the contributions of the rest of the joints. The use of the normalized integrals give a clear picture of the behavior of each joint while still being anchored to quantitative data. The benefit is that for more demanding cases there is an abundance of data to use but for simpler cases the observer will not be overburden with graphs. Examining a task in this manner allows the ergonomist to identify problematic cases where the majority of the motion and the strain of the torques is accumulated in specific joints.

Finally, Fig. 13a–14b compare the same trials with the average motions and torques of multiple trials. This allows to see how each performance deviates from the average. Of course, the assumption in those graphs is that the average performance is the most optimum way to perform each task. As a result, any deviation is considered problematic. The difference from the previous approach where the individual’s performance was tested in isolation is that specific instructions can be given to the non-optimal performer to converge to the average.

To sum up, RULA has been widely used by the industry due to its simplicity and conciseness. However, it is based on subjective observation and a specific instance of the task with little room to analyze it further. This paper proposed a new approach based on kinematic and kinetic data that allows for a quick review of the task while providing abundant information for the duration of it. The results show that instead of having a relatively nebulous number grading for the severity of the ergonomic hazard, it is possible to examine every joint of the human body and identify where exactly is the problem. By using the normalized integral of the joint angles and the joint torques the nature of the excessive use of the joint (motion or torques) can also be described. This gives insight beyond the scope of RULA and can potentially help in devising strategies to address ergonomics hazards, such as using them as inputs to artificial intelligence algorithms for workspace optimization. There are two possible and complementary ways to utilize the normalized integrals. Comparing the kinematic contributions during a task of each joint with each corresponding kinetic or comparing them with their respective average values from multiple trials of the same task. The first method allows the examination of the task in isolation to identify joints that accumulate the most strain. The second method gives an insight of deviation of an individual with the average, and ideally, optimal performance. In essence, the first method examines how taxing the task itself is on each joint and the implied ergonomics hazards. The second method examines if an individual is performing in a potentially harmful way, even if the task itself is ergonomically safe.

5 Limitations and future work

The biggest limitation of the approach proposed here is that only two tasks were recorded that are not comparable with each other. Indeed, since we are dealing with specialized industrial motions, a pool of recordings is needed to calculate the average manner to do the task. More recordings are required for different tasks to be assessed this way. In general, assuming accurate data can be measured, the methods presented here can give a lot of flexibility to assess a production line in great detail. Future work will be focused in using larger and more diverse datasets to compare different tasks from realistic scenarios as well as refining the methods proposed here based on feedback from ergonomists.