Skip to main content
SpringerLink
Log in

Markerless Motion Capture and Virtual Reality for Real-Time Ergonomic Analysis of Operators in Workstations with Collaborative Robots: a preliminary study

  • Conference paper
  • First Online:
Advances on Mechanics, Design Engineering and Manufacturing IV (JCM 2022)

Part of the book series: Lecture Notes in Mechanical Engineering ((LNME))

Abstract

Collaborative robots (cobots) are designed to directly interact with human beings within a shared workspace. To minimize the risk of musculoskeletal disease for the workers, a physical ergonomic assessment of their interaction is needed. Virtual reality (VR) and motion capture (Mocap) systems can aid designers in building low-hazard collaborative environments. This work presents a framework based on VR and Mocap systems for the ergonomic evaluation of collaborative robotic workstations. Starting from the 3D models of the cobot and workstation components, a virtual environment is built in Unity and ROS is employed to manage the cobot behavior. The physical ergonomics is evaluated by means of RULA methodology, exploiting the body tracking capabilities of the device Kinect Azure, a low-cost markerless Mocap system. The framework has been tested by building a virtual environment for collaborative control of flanges with different diameters. The worker interacts with a six-axis Nyro One to move parts on the workstation. The ergonomic assessment is performed in real-time, and a report is generated for later uses and evaluations. The proposed framework fosters the design of collaborative robotics workstations based on an objective assessment of ergonomics. The results of this research work allow planning future development steps for the emulation of more complex workstations with cobots and the use of augmented reality to evaluate how to modify existing workstations to introduce a cobot.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Choi, S.H., et al.: An integrated mixed reality system for safety-aware human-robot collaboration using deep learning and digital twin generation. Robot. Comput.-Integr. Manuf. 73, 102258 (2022)

    Article  Google Scholar 

  2. Gualtieri, L., Rauch, E., Vidoni, R.: Emerging research fields in safety and ergonomics in industrial collaborative robotics: a systematic literature review. Robot. Comput.-Integr. Manuf. 67, 101998 (2021)

    Article  Google Scholar 

  3. Aliev, K., Antonelli, D.: Proposal of a monitoring system for collaborative robots to predict outages and to assess reliability factors exploiting machine learning. Appl. Sci. 11(4), 1621 (2021)

    Article  Google Scholar 

  4. Lorusso, M., Rossoni, M., Carulli, M., Bordegoni, M., Colombo, G.: A virtual reality application for 3D sketching in conceptual design. Comput.-Aided Des. Appl. 19(2), 256–268 (2022)

    Article  Google Scholar 

  5. Khamaisi, R.K., Prati, E., Peruzzini, M., Raffaeli, R., Pellicciari, M.: UX in AR-supported industrial human-robot collaborative tasks: a systematic review. Appl. Sci. 11(21), 10448 (2021)

    Article  Google Scholar 

  6. Galin, R.R., Meshcheryakov, R.V.: Human-robot interaction efficiency and human-robot collaboration. In: Kravets, A.G. (ed.) Robotics: Industry 4.0 Issues & New Intelligent Control Paradigms. SSDC, vol. 272, pp. 55–63. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-37841-7_5

    Chapter  Google Scholar 

  7. Lorusso, M., Rossoni, M., Colombo, G.: Conceptual modeling in product design within virtual reality environments. Comput.-Aided Des. Appl. 18(2), 383–398 (2021)

    Article  Google Scholar 

  8. Müller, J.: Enabling Technologies for Industry 5.0. Results of a workshop with Europe’s technology leaders (2020)

    Google Scholar 

  9. Salunkhe, O., Stensöta, O., Åkerman, M., Berglund, Å.F., Alveflo, P.A.: Assembly 4.0: Wheel Hub Nut assembly using a Cobot. IFAC-PapersOnLine 52(13), 1632–1637 (2019)

    Google Scholar 

  10. Shafti, A., Ataka, A., Lazpita, B.U., Shiva, A., Wurdemann, H.A., Althoefer, K.: Real-time robot-assisted Ergonomics. In: Proceedings - IEEE International Conference on Robotics and Automation, pp. 1975–1981. IEEE, Montreal, Canada (2019)

    Google Scholar 

  11. Lowe, B.D., Dempsey, P.G., Jones, E.M.: Ergonomics assessment methods used by ergonomics professionals. Appl. Ergon. 81, 102882 (2019)

    Article  Google Scholar 

  12. Kim, W., et al.: Adaptable workstations for human-robot collaboration: a reconfigurable framework for improving worker ergonomics and productivity. In: IEEE Robotics & Automation Magazine, pp. 14–26. IEEE, Montreal, Canada (2019)

    Google Scholar 

  13. Malik, A.A., Brem, A.: Digital twins for collaborative robots: a case study in human-robot interaction. Robot. Comput.-Integr. Manuf. 68, 102092 (2021)

    Article  Google Scholar 

  14. Kim, W., Huang, C., Yun, D., Saakes, D., Xiong, S.: Comparison of joint angle measurements from three types of motion capture systems for ergonomic postural assessment. In: Karwowski, W., Goonetilleke, R.S., Xiong, S., Goossens, R.H.M., Murata, A. (eds.) AHFE 2020. AISC, vol. 1215, pp. 3–11. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-51549-2_1

    Chapter  Google Scholar 

  15. Colim, A., et al.: Towards an Ergonomic assessment framework for industrial assembly workstations - a case study. Appl. Sci. 10(9), 3048 (2020)

    Article  Google Scholar 

  16. Theurel, J., Desbrosses, K.: Occupational exoskeletons: overview of their benefits and limitations in preventing work-related musculOSKELETAL DISorders. IISE Trans. Occup. Ergonomics Human Factors 7(3–4), 264–280 (2019)

    Article  Google Scholar 

  17. Bortolini, M., Faccio, M., Gamberi, M., Pilati, F.: Motion Analysis System (MAS) for production and ergonomics assessment in the manufacturing processes. Comput. Ind. Eng. 139, 105485 (2020)

    Article  Google Scholar 

  18. Li, G., Buckle, P.: Current techniques for assessing physical exposure to work-related musculoskeletal risks, with emphasis on posture-based methods. Ergonomics 42(5), 674–695 (1999)

    Article  Google Scholar 

  19. Balogh, I., et al.: Self-assessed and directly measured occupational physical activities - influence of musculoskeletal complaints, age and gender. Appl. Ergon. 35(1), 49–56 (2004)

    Article  MathSciNet  Google Scholar 

  20. Kowalski, K., Rhodes, R., Naylor, P.J., Tuokko, H., MacDonald, S.: Direct and indirect measurement of physical activity in older adults: a systematic review of the literature. Int. J. Behav. Nutr. Phys. Act. 9, 148 (2012)

    Article  Google Scholar 

  21. Tölgyessy, M., Dekan, M., Chovanec, L.: Skeleton tracking accuracy and precision evaluation of kinect V1, Kinect V2, and the Azure Kinect. Appl. Sci. 11(12), 5766 (2021)

    Article  Google Scholar 

  22. Albert, J.A., Owolabi, V., Gebel, A., Brahms, C.M., Granacher, U., Arnrich, B.: Evaluation of the pose tracking performance of the Azure kinect and kinect v2 for Gait analysis in comparison with a gold standard: a pilot study. Sensors 20(18), 5104 (2020)

    Article  Google Scholar 

  23. Gualtieri, L., Palomba, I., Merati, F.A., Rauch, E., Vidoni, R.: Design of human-centered collaborative assembly workstations for the improvement of operators’ physical ergonomics and production efficiency: a case study. Sustainability 12(9), 3606 (2020)

    Article  Google Scholar 

  24. Wu, Y., Wang, Y., Jung, S., Hoermann, S., Lindeman, R.W.: Towards an articulated avatar in VR: improving body and hand tracking using only depth cameras. Entertainment Comput. 31, 100303 (2019)

    Article  Google Scholar 

  25. Mcatamney, L., Nigel Corlett, E.: RULA: a survey method for the investigation of work-related upper limb disorders. Appl. Ergon. 24(2), 91–99 (1993)

    Article  Google Scholar 

  26. Vignais, N., Miezal, M., Bleser, G., Mura, K., Gorecky, D., Marin, F.: Innovative system for real-time ergonomic feedback in industrial manufacturing. Appl. Ergon. 44(4), 566–574 (2013)

    Article  Google Scholar 

  27. Bao, S., Howard, N., Spielholz, P., Silverstein, B., Polissar, N.: Interrater reliability of posture observations. Hum. Factors 51(3), 292–309 (2009)

    Article  Google Scholar 

  28. Di Marino, C., Tarallo, A., Vitali, A., Regazzoni, D.: Collaborative robotics and ergonomics: a scientific review. In: ASME International Mechanical Engineering Congress and Exposition American Society of Mechanical Engineers ASME Virtual, Online, pp. 1–5 (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Bergamo, Viale G. Marconi 5, 24044, Dalmine, Italy

    Daniel Lanzoni, Andrea Cattaneo, Andrea Vitali, Daniele Regazzoni & Caterina Rizzi

Authors
  1. Daniel Lanzoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Cattaneo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrea Vitali
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniele Regazzoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Caterina Rizzi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrea Vitali .

Editor information

Editors and Affiliations

  1. Department of Engineering, University of Campania "Luigi Vanvitelli'', Aversa, Italy

    Salvatore Gerbino

  2. Department of Industrial Engineering, University of Naples FedericoII, Naples, Italy

    Antonio Lanzotti

  3. Department of Industrial Engineering, University of Naples Federico II, Naples, Italy

    Massimo Martorelli

  4. Ingeniería de Diseño y Fabricación, Universidad de Zaragoza, Zaragoza, Zaragoza, Spain

    Ramón Mirálbes Buil

  5. Univesity of Bergamo, Dalmine, Bergamo, Italy

    Caterina Rizzi

  6. Arts et Métiers Institute, Aix En Provence Cedex 1, France

    Lionel Roucoules

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lanzoni, D., Cattaneo, A., Vitali, A., Regazzoni, D., Rizzi, C. (2023). Markerless Motion Capture and Virtual Reality for Real-Time Ergonomic Analysis of Operators in Workstations with Collaborative Robots: a preliminary study. In: Gerbino, S., Lanzotti, A., Martorelli, M., Mirálbes Buil, R., Rizzi, C., Roucoules, L. (eds) Advances on Mechanics, Design Engineering and Manufacturing IV. JCM 2022. Lecture Notes in Mechanical Engineering. Springer, Cham. https://doi.org/10.1007/978-3-031-15928-2_103

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-15928-2_103

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-15927-5

  • Online ISBN: 978-3-031-15928-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation