Skip to main content
SpringerLink
Log in

A real-time webcam-based method for assessing upper-body postures

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper presents a new vision-based method for real-time assessment of upper-body postures of a subject who is sitting in front of a desk studying or operating a computer. Unlike most existing vision-based methods that perform offline assessment from human skeletons extracted from RGB video or depth maps, the proposed method analyses directly single images captured by a webcam in front of the subject without the prone-to-error process of extracting the skeleton data from the images or depth maps. To this end, this paper proposes to assess postures by classifying them into predefined classes, without explicitly measuring the variables required for calculating risk scores. Each class of postures is associated with a configuration of the upper body, and an ergonomics risk score is assigned by following one of the scoring methods, e.g. Rapid Upper Limb Assessment (RULA). A data set of upper-body postures that cover the various scenarios when a subject is sitting in front of a desk as well as some extreme cases when the subject turns away from the desk is collected for evaluating the proposed method quantitatively. The proposed method achieved an on-average accuracy of 99.5% for binary classification (low- vs. high-risk postures), 88.2% for classification of 19 risk levels and 81.5% for classification of 30 risk levels on the data set, and the demo developed based on the method runs in real time on a regular computer.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Agarwal, A., Triggs, B.: 3D human pose from silhouettes by relevance vector regression. In: Computer Vision and Pattern Recognition, Vol. 2 (2004)

  2. Akkas, O., Cheng-Hsien, L.C.H., Hu, Y.H., Yen, T.Y., Radwin, R.G.: Measuring elemental time and duty cycle using automated video processing 344(6188), 1173–1178 (2016)

  3. Auvinen, J., Tammelin, T., Taimela, S., Zitting, P., Karppinen, J.: Neck and shoulder pains in relation to physical activity and sedentary activities in adolescence. Spine 32(9), 1038–1044 (2007)

    Article  Google Scholar 

  4. Bijalwan, V., Kumar, V., Kumari, P., Pascual, J.: KNN based machine learning approach for text and document mining. Int. J. Database Theory Appl. 7(1), 61–70 (2014)

    Article  Google Scholar 

  5. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, ACM, New York, NY, USA, COLT ’92, pp. 144–152 (1992)

  6. Brink, Y., Louw, Q., Grimmer, K., Schreve, K., van der Westhuizen, G., Jordaan, E.: Development of a cost effective three-dimensional posture analysis tool: validity and reliability. BMC Musculoskelet. Disord. 14(1), 335 (2013)

    Article  Google Scholar 

  7. Capecci, M., Ceravolo, M.G., Ferracuti, F., Iarlori, S., Kyrki, V., Monteriù, A., Romeo, L., Verdini, F.: A hidden semi-Markov model based approach for rehabilitation exercise assessment. J. Biomed. Inform. 78, 1–11 (2018)

    Article  Google Scholar 

  8. Caputo, F., DAmato, E., Greco, A., Notaro, I., Spada, S.: Human posture tracking system for industrial process design and assessment. In: International Conference on Intelligent Human Systems Integration, Springer, pp. 450–455 (2018)

  9. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2:27:1–27:27. software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm (2011)

  10. Chen, C., Yang, Y., Nie, F., Odobez, J.M.: 3D human pose recovery from image by efficient visual feature selection. Comput. Vis. Image Underst. 115(3), 290–299 (2011)

    Article  Google Scholar 

  11. Chen, C.H., Hu, Y.H., Yen, T.Y., Radwin, R.G.: Automated video exposure assessment of repetitive hand activity level for a load transfer task. Hum. Factors 55(2), 298–308 (2013)

    Article  Google Scholar 

  12. Chen, C.H., Azari, D., Hu, Y.H., Lindstrom, M.J., Thelen, D., Thomas, Y.Y., Radwin, R.G.: The accuracy of conventional 2D video for quantifying upper limb kinematics in repetitive motion occupational tasks 344(6188), 1173–1178 (2015)

  13. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  14. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, IEEE, Vol. 1, pp. 886–893 (2005)

  15. David, G.C.: Ergonomic methods for assessing exposure to risk factors for work-related musculoskeletal disorders. Occup. Med. 55(3), 190–199 (2005). http://occmed.oxfordjournals.org/content/55/3/190.full.pdf+html

  16. Diego-Mas, J.A., Alcaide-Marzal, J.: Using Kinect sensor in observational methods for assessing postures at work. Appl. Ergon. 45(4), 976–985 (2014)

    Article  Google Scholar 

  17. Ferrario, V.F., Sforza, C., Serrao, G., Grassi, G., Mossi, E.: Active range of motion of the head and cervical spine: a three-dimensional investigation in healthy young adults. J. Orthop. Res. 20(1), 122–129 (2002)

    Article  Google Scholar 

  18. Ferreira, E.A.G., Duarte, M., Maldonado, E.P., Burke, T.N., Marques, A.P.: Postural assessment software (pas/sapo): validation and reliabiliy. Clinics 65, 675–681 (2010)

    Article  Google Scholar 

  19. Graf, M., Guggenbühl, U., Krueger, H.: An assessment of seated activity and postures at five workplaces. Int. J. Ind. Ergon. 15(2), 81–90 (1995)

    Article  Google Scholar 

  20. Greene, R.L., Azari, D.P., Hu, Y.H., Radwin, R.G.: Visualizing stressful aspects of repetitive motion tasks and opportunities for ergonomic improvements using computer vision. Appl. Ergon. 65, 461–472 (2017)

    Article  Google Scholar 

  21. Gu, B., Sheng, V.S., Tay, K.Y., Romano, W., Li, S.: Incremental support vector learning for ordinal regression. IEEE Trans. Neural Netw. Learn. Syst. 26(7), 1403–1416 (2015)

    Article  MathSciNet  Google Scholar 

  22. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  23. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: Computer Vision and Pattern Recognition, IEEE, pp. 1–8 (2007)

  24. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, KQ.: Densely connected convolutional networks. In: Computer Vision and Pattern Recognition, vol. 1, p. 3 (2017)

  25. Karhu, O., Kansi, P., Kuorinka, I.: Correcting working postures in industry: a practical method for analysis. Appl. Ergon. 8(4), 199–201 (1977)

    Article  Google Scholar 

  26. Kratnov, J., Ejglicov, K., Mal, M., Filipov, V.: Prevalence and risk factors of poor posture in school children in the Czech Republic. J. Sch. Health 77(3), 131–137 (2007)

    Article  Google Scholar 

  27. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  28. Lienhart, R., Kuranov, A., Pisarevsky, V.: Empirical analysis of detection cascades of boosted classifiers for rapid object detection. In: Michaelis, B., Krell, G. (eds.) Pattern Recognition. Lecture Notes in Computer Science, vol. 2781, pp. 297–304. Springer, Berlin (2003)

    Chapter  Google Scholar 

  29. Lowe, B., Weir, P., Andrews, D. Observation-based posture assessment: review of current practice and recommendations for improvement. NOISH p. 32 (2014)

  30. Luo, J.H., Lin, C.H.: Pure fpga implementation of an hog based real-time pedestrian detection system. Sensors 18(4), 1174 (2018)

    Article  Google Scholar 

  31. McAtamney, L., Corlett, E.N.: RULA: a survey method for the investigation of work-related upper limb disorders. Appl. Ergon. 24(2), 91–99 (1993)

    Article  Google Scholar 

  32. McVeigh, K.H., Murray, P.M., Heckman, M.G., Rawal, B., Peterson, J.J.: Accuracy and validity of goniometer and visual assessments of angular joint positions of the hand and wrist. J. Hand Surg. 41(4), e21–e35 (2016)

    Article  Google Scholar 

  33. Mehrizi, R., Xu, X., Zhang, S., Pavlovic, V., Metaxas, D., Li, K.: Using a marker-less method for estimating l5/s1 moments during symmetrical lifting. Appl. Ergon. (2017)

  34. Mehrizi, R., Peng, X., Xu, X., Zhang, S., Metaxas, D., Li, K.: A computer vision based method for 3D posture estimation of symmetrical lifting. J. Biomech. (2018)

  35. Mohanaiah, P., Sathyanarayana, P., GuruKumar, L.: Image texture feature extraction using GLCM approach. Int. J. Sci. Res. Publ. 3(5), 1 (2013)

    Google Scholar 

  36. Ong, E.J., Gong, S.: A dynamic human model using hybrid 2D-3D representations in hierarchical PCA space. In: BMVC, Citeseer (1999)

  37. Ortiz-Hernández, L., Tamez-González, S., Martínez-Alcántara, S., Méndez-Ramírez, I.: Computer use increases the risk of musculoskeletal disorders among newspaper office workers. Arch. Med. Res. 34(4), 331–342 (2003)

    Article  Google Scholar 

  38. Park, S., Park, H., Kim, J., Adeli, H.: 3d displacement measurement model for health monitoring of structures using a motion capture system. Measurement 59, 352–362 (2015)

    Article  Google Scholar 

  39. Plantard, P., Shum, H.P., Le Pierres, A.S., Multon, F.: Validation of an ergonomic assessment method using kinect data in real workplace conditions. Appl. Ergon. (2017)

  40. Robertson, M., Amick, B.C., DeRango, K., Rooney, T., Bazzani, L., Harrist, R., Moore, A.: The effects of an office ergonomics training and chair intervention on worker knowledge, behavior and musculoskeletal risk. Appl. Ergon. 40(1), 124–135 (2009)

    Article  Google Scholar 

  41. Ruivo, R.M., Pezarat-Correia, P., Carita, A.I.: Cervical and shoulder postural assessment of adolescents between 15 and 17 years-old and association with upper quadrant pain. Braz. J. Phys. Ther. 18, 364–371 (2014)

    Article  Google Scholar 

  42. Ruivo, R.M., Pezarat-Correia, P., Carita, A.I.: Intrarater and interrater reliability of photographic measurement of upper-body standing posture of adolescents. J. Manipulative Physiol. Ther. 38(1), 74–80 (2015)

    Article  Google Scholar 

  43. Saad, K.R., Colombo, A.S., Joo, S.M.A.: Reliability and validity of the photogrammetry for scoliosis evaluation: a cross-sectional prospective study. J. Manipulative Physiol. Ther. 32(6), 423–430 (2009)

    Article  Google Scholar 

  44. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: inverted residuals and linear bottlenecks. In: Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

  45. Sarsfield, J., Brown, D., Sherkat, N., Langensiepen, C., Lewis, J., Taheri, M., McCollin, C., Barnett, C., Selwood, L., Standen, P., et al.: Clinical assessment of depth sensor based pose estimation algorithms for technology supervised rehabilitation applications. Int. J. Med. Inform. 121, 30–38 (2019)

    Article  Google Scholar 

  46. Schmid, H.: Probabilistic part-of-speech tagging using decision trees. In: New Methods in Language Processing, Routledge, p. 154 (2013)

  47. Seo, J., Han, S., Lee, S., Kim, H.: Computer vision techniques for construction safety and health monitoring. Adv. Eng. Inform. 29(2), 239–251 (2015)

    Article  Google Scholar 

  48. Sonne, M., Villalta, D.L., Andrews, D.M.: Development and evaluation of an office ergonomic risk checklist: Rosa-rapid office strain assessment. Appl. Ergon. 43(1), 98–108 (2012)

    Article  Google Scholar 

  49. United States Bone and Joint Initiative. https://www.boneandjointburden.org/2014-report/xe1/cost-treat-musculoskeletal-diseases (2016)

  50. Vemulapalli, R., Arrate, F., Chellappa, R.: Human action recognition by representing 3D skeletons as points in a lie group. In: Computer Vision and Pattern Recognition, pp. 588–595 (2014)

  51. Vignais, N., Miezal, M., Bleser, G., Mura, K., Gorecky, D., Marin, F.: Innovative system for real-time ergonomic feedback in industrial manufacturing. Appl. Ergon. 44(4), 566–574 (2013)

    Article  Google Scholar 

  52. Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K. Aggregated residual transformations for deep neural networks. In: Computer Vision and Pattern Recognition, pp. 1492–1500 (2017)

  53. Yang, J., Shi, Z., Wu, Z.: Vision-based action recognition of construction workers using dense trajectories. Adv. Eng. Inform. 30(3), 327–336 (2016)

    Article  Google Scholar 

  54. Ye, Q., Han, Z., Jiao, J., Liu, J.: Human detection in images via piecewise linear support vector machines. IEEE Trans. Image Process. 22(2), 778–789 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  55. Zhang, Y., Zhou, W., Yuan, S.: Multifractal analysis and relevance vector machine-based automatic seizure detection in intracranial EEG. Int. J. Neural Syst. 25(06), 1550020 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

Funding was provided by China Scholarship Council (Grant No. 201406320155).

Author information

Authors and Affiliations

  1. Advanced Multimedia Research Lab, University of Wollongong, Northfields Ave, Wollongong, NSW 2522, Australia

    Zewei Ding, Wanqing Li & Philip Ogunbona

  2. The People’s Hospital of Guangxi Zhuang Autonomous Region, Nanning, China

    Ling Qin

Authors
  1. Zewei Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wanqing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Philip Ogunbona
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ling Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zewei Ding.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, Z., Li, W., Ogunbona, P. et al. A real-time webcam-based method for assessing upper-body postures. Machine Vision and Applications 30, 833–850 (2019). https://doi.org/10.1007/s00138-019-01033-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-019-01033-9

Keywords

Search

Navigation