Machine Vision and Applications

, Volume 22, Issue 2, pp 219–234 | Cite as

A vision-based system for automatic hand washing quality assessment

  • David Fernández Llorca
  • Ignacio Parra
  • Miguel Ángel Sotelo
  • Gerard Lacey
Original Paper

Abstract

Hand washing is a critical activity in preventing the spread of infection in health-care environments and food preparation areas. Several guidelines recommended a hand washing protocol consisting of six steps that ensure that all areas of the hands are thoroughly cleaned. In this paper, we describe a novel approach that uses a computer vision system to measure the user’s hands motions to ensure that the hand washing guidelines are followed. A hand washing quality assessment system needs to know if the hands are joined or separated and it has to be robust to different lighting conditions, occlusions, reflections and changes in the color of the sink surface. This work presents three main contributions: a description of a system which delivers robust hands segmentation using a combination of color and motion analysis, a single multi-modal particle filter (PF) in combination with a k-means-based clustering technique to track both hands/arms, and the implementation of a multi-class classification of hand gestures using a support vector machine ensemble. PF performance is discussed and compared with a standard Kalman filter estimator. Finally, the global performance of the system is analyzed and compared with human performance, showing an accuracy close to that of human experts.

Keywords

Hand washing Bi-manual gesture recognition Kalman filter Particle filter Skin detection Tracking Multi-class SVM 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

ESM 1 (AVI 3,451 kb)

References

  1. 1.
    Alonso I.P., Llorca D.F., Sotelo M.A., Bergasa L.M., Revenga P., Nuevo J., Ocaña M., Garrido M.A.: Combination of feature extraction methods for SVM pedestrian detection. IEEE Trans. Intell. Transport. Syst. 8(2), 292–397 (2007)CrossRefGoogle Scholar
  2. 2.
    Boyce J.M., Pittet D.: Guideline for hand hygiene in health-care settings. Morb. Mortal. Wkly. Rep. 51(RR-16), 1–45 (2002)Google Scholar
  3. 3.
    Bradsky G.R., Davis J.W.: Motion segmentation and pose recognition with motion history gradients. Mach. Vis. Appl. 13, 174–178 (2002)CrossRefGoogle Scholar
  4. 4.
    Chen, P., Grecos, C.: A fast skin region detector for colour images. In: Proceedings of IEEE International Conference on Visual Information Engineering (2005)Google Scholar
  5. 5.
    Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm (2001)
  6. 6.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2005)Google Scholar
  7. 7.
    HACCP: A Food Safety Management System based on the Principles of Hazard Analysis and Critical Control Point, Food Safety. Authority of Ireland (2006)Google Scholar
  8. 8.
    Hoey, J.: Tracking using flocks of features, with application to assisted handwashing. In: Proceedings British Machine Vision Conference (2006)Google Scholar
  9. 9.
    Hoey, J., von Bertoldi, A., Poupart, P., Mihailidis, A.: Assisting persons with dementia during handwashing using a partially observable Markov decision process. In: Proceedings of International Conference on Vision Systems (2007)Google Scholar
  10. 10.
    Hsu C.W., Lin C.J.: A comparison of methods for multi-class support vector machines. IEEE Trans. Neural Netw. 13(2), 415–425 (2005)Google Scholar
  11. 11.
    Isard M., Blake A.: CONDENSATION—conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1), 5–28 (1998)CrossRefGoogle Scholar
  12. 12.
    Isard, M., McCormick, J.: BraMBLe: A Bayesian multiple-blob tracker. In: Proceedings of IEEE International Conference on Computer Vision (2001)Google Scholar
  13. 13.
    Kinometrics: Low cost wireless sensors to measure human activity. http://www.kinometrics.com (2007)
  14. 14.
    Lee, H.-J., Chung, J.-H.: Hand gesture recognition using orientation histograms. In: Proceedings of IEEE Region 10 Conference TENCON (1999)Google Scholar
  15. 15.
    Lin, D.-T.: Spatio-temporal hand gesture recognition using neural networks. In: Proceedings of IEEE World Congress on Computational Intelligence (1998)Google Scholar
  16. 16.
    Llorca, D.F., Vilarino, F., Zhou, J., Lacey, G.: A multi-class SVM classifier for automatic hand washing quality assessment. In: Proceedings of British Machine Vision Conference (2007)Google Scholar
  17. 17.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of IUW (1981)Google Scholar
  18. 18.
    Marcel, S., Bernier, O., Viallet, J.-E., Collobert, D.: Hand gesture recognition using input–output hidden Markov models. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (2000)Google Scholar
  19. 19.
    Kolsch, M., Turk, M.: Robust Hand Detection, In. Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (2004)Google Scholar
  20. 20.
    MRSA: Methicillin-resistant Staphylococcus aureus. Guidance for nursing staff, Royal College of Nursing (2005)Google Scholar
  21. 21.
    Pavlovic V., Sharma R., Huang T.: Visual interpretation of hand gestures for human–computer interaction: a review. IEEE Trans. Pattern Anal. Mach. Intell. 19(5), 677–695 (1997)CrossRefGoogle Scholar
  22. 22.
    Qu W., Schonfeld D., Mohamed M.: Real-time distributed multi-object tracking using multiple interactive trackers and a magnetic-inertia potential model. IEEE Trans. Multimedia 5(3), 511–519 (2007)CrossRefGoogle Scholar
  23. 23.
    Sanjeev M., Maskell S., Gordon N., Clapp T.: A tutorial on particle filters for online nonlinear/Non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002)CrossRefGoogle Scholar
  24. 24.
    SARI: Guidelines for hand hygiene in Irish Health Care settings. A strategy for the control of antimicrobial resistance. Infection Control Subcommittee (2004)Google Scholar
  25. 25.
    Starner, T., Pentland, A.: Visual recognition of American sign language using hidden Markov models. In: Proceedings of International Workshop on Automatic Face and Gesture Recognition (1995)Google Scholar
  26. 26.
    Storring, M.: Computer Vision and Human Skin Color. Faculty of Engineering and Sciencem Aalborg University, Niels Jernes Vej 14, 9220 Aalborg, Denmark (2004)Google Scholar
  27. 27.
    Su, M.-C., Huang, H., Lin, C.-H., Huang, C.-L., Lin, C.-D.: Application of neural networks in spatio-temporal hand gesture recognition. In: Proceedings of IEEE World Congress on Computational Intelligence (1998)Google Scholar
  28. 28.
    Boser, B.E., Guyon, I.M, Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: ACM workshop on COLT, pp. 144–152 (1992)Google Scholar
  29. 29.
    Viola P., Jones M.: Robust real-time object detection. Int. J. Comput. Vis. 57(2), 137–154 (2002)CrossRefGoogle Scholar
  30. 30.
    Wu T.F., Lin C.J., Weng R.C.: Probability estimates for multi-class classification by pairwise coupling. J. Mach. Learn. Res. 5, 975–1005 (2004)MathSciNetGoogle Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  • David Fernández Llorca
    • 1
  • Ignacio Parra
    • 1
  • Miguel Ángel Sotelo
    • 1
  • Gerard Lacey
    • 2
  1. 1.Department of ElectronicsUniversity of AlcaláMadridSpain
  2. 2.Department of Computer ScienceTrinity College DublinDublinRepublic of Ireland

Personalised recommendations