Skip to main content
SpringerLink
Log in

A Multi-Modal Feedback Communication Interface for Human Working Posture Adjustments

  • Conference paper
  • First Online:
Human-Friendly Robotics 2022 (HFR 2022)

Part of the book series: Springer Proceedings in Advanced Robotics ((SPAR,volume 26))

Included in the following conference series:

  • 434 Accesses

Abstract

This paper studies non-physical feedback mechanisms to guide human workers toward ergonomic body postures. Specifically, the focus is to solve the tasks that involve no direct physical interaction between the human and the robotic system, therefore tactile guidance by the robot body is not feasible. We propose a multi-modal ergonomic posture guidance system that comprises visual feedback and speech-based audio feedback. We hypothesise that the proposed multi-modal system leads to better performance compared to uni-modal feedback systems when trying to guide users from one pose to another. To test the hypothesis we conducted an experiment that compared conditions with only audio feedback, only visual feedback and multi-modal feedback. In addition, we examined speech-based audio guidance in joint space and in endpoint space. The results showed that the speech-based feedback in joint space came out as the preferred audio feedback due to its ability to allow users to carry out efficient and coordinated inter-joint movements, especially in cases of high redundancy. Furthermore, the proposed multi-modal feedback system was superior compared to the other feedback modalities both in terms of objective measures and subjective measures.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    This format of reporting values corresponds to mean±standard deviation and applies for the rest of the paper.

References

  1. Aldini, S., Lai, Y., Carmichael, M.G., Paul, G., Liu, D.: Real-time estimation of the strength capacity of the upper limb for physical human-robot collaboration. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 4533–4536. IEEE (2021)

    Google Scholar 

  2. Barondess, J., Cullen, M., De Lateur, B., Deyo, R., Donaldson, K., Drury, C., et al.: Musculoskeletal disorders and the workplace: low back and upper extremities. Washington, DC: National Academy of Sciences pp. 1–512 (2001)

    Google Scholar 

  3. Brewster, S.: The Human-Computer Interaction Handbook, vol. 1. CRC Press (2012)

    Google Scholar 

  4. Busch, B., Maeda, G., Mollard, Y., Demangeat, M., Lopes, M.: Postural optimization for an ergonomic human-robot interaction. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2778–2785. IEEE (2017)

    Google Scholar 

  5. Chen, Y., Huang, H., Xu, W., Wallis, R.I., Sundaram, H., Rikakis, T., Ingalls, T., Olson, L., He, J.: The design of a real-time, multimodal biofeedback system for stroke patient rehabilitation. In: Proceedings of the 14th ACM international conference on Multimedia, pp. 763–772 (2006)

    Google Scholar 

  6. Feuerstein, M.: Definition, empirical support, and implications for prevention, evaluation, and rehabilitation of occupational upper extremity disorders. In: Beyond Biomechanics: Psychosocial Aspects of Musculoskeletal Disorders in Office Work, pp. 177–206. Taylor & Francis (1996)

    Google Scholar 

  7. Figueredo, L.F., Aguiar, R.C., Chen, L., Chakrabarty, S., Dogar, M.R., Cohn, A.G.: Human comfortability: Integrating ergonomics and muscular-informed metrics for manipulability analysis during human-robot collaboration. IEEE Robotics and Automation Letters 6(2), 351–358 (2020)

    Article  Google Scholar 

  8. Hart, S.G.: Nasa-task load index (nasa-tlx); 20 years later. In: Proceedings of the human factors and ergonomics society annual meeting, vol. 50, pp. 904–908. Sage publications Sage CA: Los Angeles, CA (2006)

    Google Scholar 

  9. Kim, W., Garate, V.R., Gandarias, J.M., Lorenzini, M., Ajoudani, A.: A directional vibrotactile feedback interface for ergonomic postural adjustment. IEEE Transactions on Haptics 15(1), 200–211 (2021)

    Article  Google Scholar 

  10. Kim, W., Lorenzini, M., Balatti, P., Nguyen, P.D., Pattacini, U., Tikhanoff, V., Peternel, L., Fantacci, C., Natale, L., Metta, G., et al.: Adaptable workstations for human-robot collaboration: A reconfigurable framework for improving worker ergonomics and productivity. IEEE Robotics & Automation Magazine 26(3), 14–26 (2019)

    Article  Google Scholar 

  11. Kumar, S.: Theories of musculoskeletal injury causation. Ergonomics 44(1), 17–47 (2001). https://doi.org/10.1080/00140130120716

    Article  Google Scholar 

  12. van der Laan, J.D., Heino, A., De Waard, D.: A simple procedure for the assessment of acceptance of advanced transport telematics. Transportation Research Part C: Emerging Technologies 5(1), 1–10 (1997)

    Article  Google Scholar 

  13. Lee, J., Cho, E., Kim, M., Yoon, Y., Choi, S.: Preventfhp: Detection and warning system for forward head posture. In: 2014 IEEE Haptics Symposium (HAPTICS), pp. 295–298. IEEE (2014)

    Google Scholar 

  14. Lorenzini, M., Kim, W., De Momi, E., Ajoudani, A.: A real-time graphic interface for the monitoring of the human joint overloadings with application to assistive exoskeletons. In: International Symposium on Wearable Robotics, pp. 281–285. Springer (2018)

    Google Scholar 

  15. Marin, A.G., Shourijeh, M.S., Galibarov, P.E., Damsgaard, M., Fritzsch, L., Stulp, F.: Optimizing contextual ergonomics models in human-robot interaction. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1–9. IEEE (2018)

    Google Scholar 

  16. Maurice, P., Padois, V., Measson, Y., Bidaud, P.: Experimental assessment of the quality of ergonomic indicators for dynamic systems computed using a digital human model. International Journal of Human Factors Modelling and Simulation 5(3), 190–209 (2016)

    Article  Google Scholar 

  17. Peternel, L., Fang, C., Laghi, M., Bicchi, A., Tsagarakis, N., Ajoudani, A.: Human arm posture optimisation in bilateral teleoperation through interface reconfiguration. In: 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), pp. 1102–1108. IEEE (2020)

    Google Scholar 

  18. Peternel, L., Fang, C., Tsagarakis, N., Ajoudani, A.: A selective muscle fatigue management approach to ergonomic human-robot co-manipulation. Robotics and Computer-Integrated Manufacturing 58, 69–79 (2019)

    Article  Google Scholar 

  19. Peternel, L., Kim, W., Babič, J., Ajoudani, A.: Towards ergonomic control of human-robot co-manipulation and handover. In: 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), pp. 55–60. IEEE (2017)

    Google Scholar 

  20. Peternel, L., Schøn, D.T., Fang, C.: Binary and hybrid work-condition maps for interactive exploration of ergonomic human arm postures. Frontiers in Neurorobotics p. 114 (2021)

    Google Scholar 

  21. Portnoy, S., Halaby, O., Dekel-Chen, D., Dierick, F.: Effect of an auditory feedback substitution, tactilo-kinesthetic, or visual feedback on kinematics of pouring water from kettle into cup. Applied ergonomics 51, 44–49 (2015)

    Article  Google Scholar 

  22. Rahal, R., Matarese, G., Gabiccini, M., Artoni, A., Prattichizzo, D., Giordano, P.R., Pacchierotti, C.: Caring about the human operator: haptic shared control for enhanced user comfort in robotic telemanipulation. IEEE transactions on haptics 13(1), 197–203 (2020)

    Article  Google Scholar 

  23. Schneider, E., Irastorza, X.: OSH in figures: Work-related musculoskeletal disorders in the EU - Facts and Figures. Publications Office of the European Union (2010). https://doi.org/10.2802/10952

    Article  Google Scholar 

  24. Shafti, A., Ataka, A., Lazpita, B.U., Shiva, A., Wurdemann, H.A., Althoefer, K.: Real-time robot-assisted ergonomics. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 1975–1981. IEEE (2019)

    Google Scholar 

  25. Sigrist, R., Rauter, G., Riener, R., Wolf, P.: Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychonomic bulletin & review 20(1), 21–53 (2013)

    Article  Google Scholar 

  26. van der Spaa, L., Gienger, M., Bates, T., Kober, J.: Predicting and optimizing ergonomics in physical human-robot cooperation tasks. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 1799–1805. IEEE (2020)

    Google Scholar 

  27. Vianello, L., Gomes, W., Stulp, F., Aubry, A., Maurice, P., Ivaldi, S.: Latent ergonomics maps: Real-time visualization of estimated ergonomics of human movements. Sensors 22(11), 3981 (2022)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Cognitive Robotics, Delft University of Technology, Mekelweg 2, 2628, CD, Delft, The Netherlands

    Kushal Thirani, David A. Abbink & Luka Peternel

Authors
  1. Kushal Thirani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David A. Abbink
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luka Peternel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luka Peternel .

Editor information

Editors and Affiliations

  1. Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands

    Pablo Borja

  2. Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands

    Cosimo Della Santina

  3. Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands

    Luka Peternel

  4. Department of Mechanical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands

    Elena Torta

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Thirani, K., Abbink, D.A., Peternel, L. (2023). A Multi-Modal Feedback Communication Interface for Human Working Posture Adjustments. In: Borja, P., Della Santina, C., Peternel, L., Torta, E. (eds) Human-Friendly Robotics 2022. HFR 2022. Springer Proceedings in Advanced Robotics, vol 26. Springer, Cham. https://doi.org/10.1007/978-3-031-22731-8_2

Download citation

Publish with us

Policies and ethics

Search

Navigation