Skip to main content
SpringerLink
Log in

A real-time posture assessment system based on motion capture data for manual maintenance and assembly processes

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

As manufacturing processes of complex products become automated, manual operations still occupy a considerable portion of industrial maintenance and assembly (IMA), especially in the machinery and aerospace fields. Workers often exhibit awkward posture in IMA activities. In these scenarios, posture assessment is critical for improving the well-being of workers because awkward postures can lead to work-related musculoskeletal disorder (WMSD). Although there are several categorized WMSD risk assessment methods, limited evidence suggests that these methods are compliant for modern complex IMA scenarios. In this paper, a posture analysis system for manual operation (PASMO) is presented to monitor working postures and evaluate WMSD risks in IMA processes. The noninvasive depth sensor Kinect v2 and the rapid upper limb analysis (RULA) method are integrated to achieve this purpose. In the PASMO, the RULA is optimized and driven by motion capture (MoCap) data to make evaluating the WMSD risk of working postures more effective and accurate. Industrial and laboratory experiments are designed to verify the effectiveness and system performance of the PASMO. The results show that for the overall body and most joints, the scores obtained by the PASMO substantially agree with those obtained by the ground truth data (p < 0.01, κ = 0.65) under the real industrial environment. Because the experiments are conducted in real IMA scenarios with body occlusion, the results preliminarily prove the effectiveness of the PASMO for WMSD risk assessment in continuous IMA tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Graph 1
Graph 2

Similar content being viewed by others

Availability of data and materials

Not applicable.

Code availability

Not applicable.

References

  1. Souza ML, da Costa CA, de Oliveira Ramos G (2020) da Rosa Righi R A survey on decision-making based on system reliability in the context of Industry 40. J Manuf Syst 56:133–156

    Google Scholar 

  2. Frank AG, Dalenogare LS, Ayala NF (2019) Industry 4.0 technologies: implementation patterns in manufacturing companies. Int J Prod Econ 210:15–26

    Google Scholar 

  3. Peruzzini M, Grandi F, Cavallaro S, Pellicciari M (2021) Using virtual manufacturing to design human-centric factories: an industrial case. Int J Adv Manuf Technol 115(3):873–887

    Google Scholar 

  4. Krugh M, Mears L (2018) A complementary cyber-human systems framework for industry 40 cyber-physical systems. Manuf Lett 15:89–92

    Google Scholar 

  5. Aicha M, Belhadj I, Hammadi M, Aifaoui N (2022) A mathematical formulation for processing time computing in disassembly lines and its optimization. Comput Ind Eng 165:107933

    Google Scholar 

  6. Chang Y-H, Wang Y-C (2010) Significant human risk factors in aircraft maintenance technicians. Saf Sci 48(1):54–62

    Google Scholar 

  7. Asadi H, Yu D, Mott JH (2019) Risk factors for musculoskeletal injuries in airline maintenance, repair & overhaul. Int J Ind Ergon 70:107–115

    Google Scholar 

  8. Sebti R, Boulila A, Hamza S (2020) Ergonomics risk assessment among maintenance operators in a Tunisian railway company: a case study. Hum Factors Ergon Manuf Serv Ind 30(2):124–139

    Google Scholar 

  9. Ciccarelli M, Papetti A, Germani M, Leone A, Rescio G (2022) Human work sustainability tool. J Manuf Syst 62:76–86

    Google Scholar 

  10. WH Organization (2003) Protecting workers’ health series no. 5, preventing musculoskeletal disorders in the workplace. World Health Organization 71(4):47-48

  11. Muylaert K,  Eeckelaert L,  Brueck C,  Lafrenz B,  Jachowicz M, Oleszek B,  Dźwiarek M,  Strawiński T, Makowski  K,  Pietrowski P (2010) Safe maintenance in practice Office for Official Publications of the European Communities

  12. Statistics BoL Nonfatal occupational injuries and illnesses requiring days away from work 2014 2012 US Department of Labor Washington

  13. Diego-Mas JA, Alcaide-Marzal J (2014) Using Kinect sensor in observational methods for assessing postures at work. Appl Ergon 45(4):976–985

    PubMed  Google Scholar 

  14. Bortolini M,  Faccio M,  Gamberi M,  Pilati F (2020) Motion analysis system (MAS) for production and ergonomics assessment in the manufacturing processes Comput Ind Eng 139

  15. Irwin E, Streilein K (2015) Use of field-based motion capture to augment observational data in ergonomic assessment of aircraft maintenance. Procedia Manufacturing 3:4501–4508

    Google Scholar 

  16. Zungu L (2015) A comparative study of the prevalence and risk factors of lower back pain among aircraft technicians in Ethiopian airlines. Occup Health South Afr 21(2):18–23

    Google Scholar 

  17. Louison C, Ferlay F, Keller D, Mestre DR (2017) Operators’ accessibility studies for assembly and maintenance scenarios using virtual reality. Fusion Eng Des 124:610–614

    CAS  Google Scholar 

  18. Bernard F, Zare M, Sagot JC, Paquin R (2019) Integration of human factors into the design process of helicopter maintainability. Hum Factors Ergon Manuf Serv Ind 29(4):350–360

    Google Scholar 

  19. Reiman T (2011) Understanding maintenance work in safety-critical organisations–managing the performance variability. Theor Issues Ergon Sci 12(4):339–366

    Google Scholar 

  20. Manghisi VM, Uva AE, Fiorentino M, Bevilacqua V, Trotta GF, Monno G (2017) Real time RULA assessment using Kinect v2 sensor. Appl Ergon 65:481–491

    PubMed  Google Scholar 

  21. David GC (2005) Ergonomic methods for assessing exposure to risk factors for work-related musculoskeletal disorders. Occup Med 55(3):190–199

    CAS  Google Scholar 

  22. Xu X, McGorry RW, Chou L-S, Lin J-H, Chang C-C (2015) Accuracy of the Microsoft Kinect™ for measuring gait parameters during treadmill walking. Gait Posture 42(2):145–151

    PubMed  Google Scholar 

  23. Micheletti Cremasco M (2019) Risk assessment for musculoskeletal disorders in forestry: a comparison between RULA and REBA in the manual feeding of a wood-chipper. Int J Environ Res Public Health 16(5):793

  24. Karhu O, Kansi P, Kuorinka I (1977) Correcting working postures in industry: a practical method for analysis. Appl Ergon 8(4):199–201

    CAS  PubMed  Google Scholar 

  25. Hignett S, McAtamney L (2000) Rapid entire body assessment (REBA). Appl Ergon 31(2):201–205

    CAS  PubMed  Google Scholar 

  26. Schaub K, Caragnano G, Britzke B, Bruder R (2013) The European assembly worksheet. Theor Issues Ergon Sci 14(6):616–639

    Google Scholar 

  27. Waters TR, Putz-Anderson V, Garg A, Fine LJ (1993) Revised NIOSH equation for the design and evaluation of manual lifting tasks. Ergonomics 36(7):749–776

    CAS  PubMed  Google Scholar 

  28. Fabio G, Margherita P, Luca Z, Marcello P (2019) An automatic procedure based on virtual ergonomic analysis to promote human-centric manufacturing. Proc Manuf 38:488–496

    Google Scholar 

  29. Labbafinejad Y, Imanizade Z, Danesh H (2016) Ergonomic risk factors and their association with lower back and neck pain among pharmaceutical employees in Iran. Workplace Health Saf 64(12):586–595

    PubMed  Google Scholar 

  30. Kee D, Karwowski W (2001) LUBA: an assessment technique for postural loading on the upper body based on joint motion discomfort and maximum holding time. Appl Ergon 32(4):357–366

    CAS  PubMed  Google Scholar 

  31. Bortolini M, Faccio M, Gamberi M, Pilati F (2020) Motion analysis system (MAS) for production and ergonomics assessment in the manufacturing processes. Comput Ind Eng 139:105485

    Google Scholar 

  32. Plantard P, Shum HPH, Le Pierres AS, Multon F (2017) Validation of an ergonomic assessment method using Kinect data in real workplace conditions. Appl Ergon 65:562–569

    PubMed  Google Scholar 

  33. Puthenveetil SC, Daphalapurkar CP, Zhu W, Leu MC, Liu XF, Gilpin-Mcminn JK, Snodgrass SD (2015) Computer-automated ergonomic analysis based on motion capture and assembly simulation. Virtual Real 19(2):119–128

    Google Scholar 

  34. Haggag H, Hossny M, Nahavandi S,  Creighton D (2013) Real time ergonomic assessment for assembly operations using Kinect, in 2013 UKSim 15th International Conference on Computer Modelling and Simulation 495-500

  35. Ho ESL, Chan J, Chan D, Shum HPH, Cheung Y-M, Yuen PC (2016) Improving posture classification accuracy for depth sensor-based human activity monitoring in smart environments. Comput Vis Image Underst 148:97–110

    Google Scholar 

  36. Jiang S,  Liu P,  Gao D,  Xu Y,  Meng X,  Liu Z,  Huang Z,  Xu R (2017) Research on low cost virtual assembly training platform based on somatosensory technology in 2017 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)

  37. Dutta T (2012) Evaluation of the Kinect sensor for 3-D kinematic measurement in the workplace. Appl Ergon 43(4):645–649

    PubMed  Google Scholar 

  38. Bonnechère B (2014) Determination of the precision and accuracy of morphological measurements using the Kinect™ sensor: comparison with standard stereophotogrammetry. Ergonomics 57(4):622–631

  39. Tee KS,  Low E,  Saim H, Zakaria WNW,  Khialdin SBM,  Isa H,  Awad MI, Soon CF (2017) A study on the ergonomic assessment in the workplace

  40. Rybarczyk Y (2017) ePHoRt project: a web-based platform for home motor rehabilitation in Recent advances in information systems and technologies. Springer International Publishing, Cham

  41. Plantard P, Auvinet E, Pierres AS, Multon F (2015) Pose estimation with a Kinect for ergonomic studies: evaluation of the accuracy using a virtual mannequin. Sensors (Basel) 15(1):1785–1803

    ADS  PubMed  Google Scholar 

  42. Abobakr A, Nahavandi D, Hossny M, Iskander J, Attia M, Nahavandi S, Smets M (2019) RGB-D ergonomic assessment system of adopted working postures. Appl Ergon 80:75–88

    PubMed  Google Scholar 

  43. Diego-Mas JA, Poveda-Bautista R, Garzon-Leal D (2017) Using RGB-D sensors and evolutionary algorithms for the optimization of workstation layouts. Appl Ergon 65:530–540

    PubMed  Google Scholar 

  44. Rybarczyk Y, Pérez Medina JL, Leconte L, Jimenes K,  González M,  Esparza D (2019) Implementation and assessment of an intelligent motor tele-rehabilitation platform Electronics 8(1)

  45. Han G, Song W (2013) Motion capture of maintenance personnel based on multi-Kinect in 2013 International Conference on Quality, Reliability, Risk, Maintenance, and Safety Engineering (QR2MSE)

  46. Tourlomousis F, Chang RC (2017) Dimensional metrology of cell-matrix interactions in 3D microscale fibrous substrates. Procedia CIRP 65:32–37

    Google Scholar 

  47. Ibarra Zannatha JM (2013) WA. Comput Methods Programs Biomed 112(2):239–49

  48. Tokuyama Y, Rajapakse RPCJ , Yamabe S, Konno K, Hung Y (2019) A Kinect-based augmented reality game for lower limb exercise in 2019 International Conference on Cyberworlds (CW)

  49. Nahangi M, Czerniawski T, Haas CT, Walbridge S (2019) Pipe radius estimation using Kinect range cameras. Autom Constr 99:197–205

    Google Scholar 

  50. Sugiyama S , Baba K,  Yoshikawa T (2012) Guide robot with personal identification method using dress color information via KINECT in 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO)

  51. Ou Y, Hu J, Wang Z, Fu Y, Wu X, Li X (2015) A real-time human imitation system using Kinect. Int J Soc Robot 7(5):587–600

    Google Scholar 

  52. Xu X, McGorry RW (2015) The validity of the first and second generation Microsoft Kinect for identifying joint center locations during static postures. Appl Ergon 49:47–54

    PubMed  Google Scholar 

  53. Jiang S,  Liu P , Fu D, Xue Y,  Luo W, Wang M (2017)  A low-cost rapid upper limb assessment method in manual assembly line based on somatosensory interaction technology

  54. Xu X, Robertson M, Chen KB, Lin JH, McGorry RW (2017) Using the Microsoft Kinect to assess 3-D shoulder kinematics during computer use. Appl Ergon 65:418–423

    PubMed  Google Scholar 

  55. Nguyen TD,  Kleinsorge M,  Krüger J (2014) ErgoAssist: an assistance system to maintain ergonomie guidelines at workplaces in Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA)

  56. Liao H,  Long X (2013) Study on virtual assembly system based on Kinect somatosensory interaction, in 2013 International Conference on Information Science and Cloud Computing 55–60

  57. Otto M,  Lampen E,  Auris F, Gaisbauer F, Rukzio E (2019) Applicability evaluation of Kinect for EAWS ergonomic assessments, in 52nd Cirp Conference on Manufacturing Systems P Butala E Govekar and R Vrabic Editors 781–784

  58. Buisseret F, Dierick F, Hamzaoui O, Jojczyk L (2018) Ergonomic risk assessment of developing musculoskeletal disorders in workers with the Microsoft Kinect: TRACK TMS. Irbm 39(6):436–439

    Google Scholar 

  59. Mgbemena CE,  Tiwari A,  Xu Y,  Oyekan J, Hutabarat W (2018) Ergonomic assessment tool for real-time risk assessment of seated work postures in Advances in Safety Management and Human Factors 423–434

  60. Guo Z, Zhou D, Liu P, He Z, Lv C (2018) A quantitative assessment method for the space design of products based on ergonomics and virtual simulation. PLoS ONE 13(7):e0200880

    PubMed  PubMed Central  Google Scholar 

  61. Jha A, Chiddarwar SS, Alakshendra V, Andulkar MV (2016) Kinematics-based approach for robot programming via human arm motion. J Braz Soc Mech Sci Eng 39(7):2659–2675

    Google Scholar 

Download references

Funding

This work is supported by China Postdoctoral Science Foundation under Certificate Number 2023M730167.

Author information

Authors and Affiliations

  1. State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China

    Dong Zhou, Chengzhang Chen, Ziyue Guo, Qidi Zhou & Aimin Hao

  2. State Key Defense Science and Technology Laboratory On Reliability and Environmental Engineering, Beihang University, Beijing, China

    Dong Zhou, Chengzhang Chen, Ziyue Guo, Qidi Zhou & Dengwei Song

  3. School of Reliability and Systems Engineering, Beihang University, Beijing, China

    Dong Zhou, Chengzhang Chen & Qidi Zhou

  4. School of Computer Science and Engineering, Beihang University, Beijing, China

    Ziyue Guo & Aimin Hao

  5. School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing, 100191, China

    Dengwei Song

Authors
  1. Dong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chengzhang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ziyue Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qidi Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dengwei Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Aimin Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Zhou Dong and Chen Chengzhang: conceptualization, methodology, and writing—original draft preparation.

Chen Chengzhang: conceptualization and funding acquisition.

Guo Ziyue: revision—revised draft preparation.

Zhou Qidi: validation.

Guo Ziyue: resources.

Chen Chengzhang: visualization.

Song Dengwei: investigation.

Hao Aimin: supervision.

Corresponding author

Correspondence to Ziyue Guo.

Ethics declarations

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

The manuscript has been approved by all authors for publication.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (MP4 74205 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, D., Chen, C., Guo, Z. et al. A real-time posture assessment system based on motion capture data for manual maintenance and assembly processes. Int J Adv Manuf Technol 131, 1397–1411 (2024). https://doi.org/10.1007/s00170-024-13114-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-024-13114-9

Keywords

Search

Navigation