Skip to main content

Advertisement

SpringerLink
Log in

Hand Movement-Controlled Image Viewer in an Operating Room by Using Hand Movement Pattern Code

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

Surgeons must intraoperatively view cross-section images under sterilization conditions. Keyboard and computer mouse are sources of contamination. A computer vision algorithm and a hand movement pattern analysis technique have been applied to solve the problem based on surgeon’s behaviors. This paper proposed a new method to control the radiological image viewer in an operating room. A pattern code of hand movement and a grid square guideline are used. Our proposed algorithm comprises three steps: hand tracking, pattern code area identification, and hand movement pattern recognition. First, the system is fed with a sequence of three-dimensional data. A 3D camera captures the whole target body. A skeleton tracking algorithm is used to detect the human body. The left-hand joint in the skeleton data set is tracked. Second, as this algorithm supports one hand movement, a grid square guideline is defined. Hand movements are interpreted from the hand path moving in the grid square area. Finally, the pattern code is defined as a feature vector. By using the feature vector and closest point classifier, the hand movements are recognized by the K-Nearest Neighbors algorithm. To test the performance of the proposed algorithm, data from twenty subjects were used. Seven commands were used to interface with the computer workstation to control the radiological image viewer. The accuracy rate was 95.72%. The repeatability was 1.88. The advantage of this method is that one hand can control the image viewer software from a distance of 1.5 m satisfactorily without contacting computer devices. Our method also does not need big data set to train the system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. W. H. Organization. (2009). WHO guidelines for safe surgery 2009: Safe surgery saves lives. World Health Organization.

    Google Scholar 

  2. Byrne, D. (2006). Adverse impact of surgical site infections in English hospitals. Journal of Hospital Infection, 62(3), 11.

    Article  Google Scholar 

  3. Gawande, A. A., Thomas, E. J., Zinner, M. J., & Brennan, T. A. (1999). The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery, 126(1), 66–75.

    Article  Google Scholar 

  4. Spigelman, A. D., Kable, A. K., & Gibberd, R. W. (2002). Adverse events in surgical patients in Australia. International Journal for Quality in Health Care, 14(4), 269–276.

    Article  Google Scholar 

  5. Bickler, S. W., & Sanno-Duanda, B. (2000). Epidemiology of paediatric surgical admissions to a government referral hospital in the Gambia. Bulletin of the World Health Organization: The International Journal of Public Health, 78, 1330–1336.

    Google Scholar 

  6. Yii, M. K., & Ng, K. J. (2002). Risk-adjusted surgical audit with the POSSUM scoring system in a developing country. Physiological and Operative Severity Score for the enUmeration of Mortality and morbidity. The British Journal of Surgery, 89, 110–113.

    Article  Google Scholar 

  7. Mcconkey, S. J. A. (2002). Case series of acute abdominal surgery in rural Sierra Leone. World Journal of Surgery, 26, 5.

    Article  Google Scholar 

  8. Pronovost, P., Needham, D., Berenholtz, S., Sinopoli, D., Chu, H., Cosgrove, S., Sexton, B., Hyzy, R., Welsh, R., Roth, G., Bander, J., Kepros, J., & Goeschel, C. (2006). An intervention to decrease catheter-related bloodstream infections in the ICU. New England Journal of Medicine, 355(26), 2725–2732.

    Article  Google Scholar 

  9. Patwardhan, N., & Kelkar, U. (2011). Disinfection, sterilization and operation theater guidelines for dermatosurgical practitioners in India. Indian Journal of Dermatology, Venereology, and Leprology, 77(1), 83–93.

    Article  Google Scholar 

  10. Schultz, M., Gill, J., Zubairi, S., Huber, R., & Gordin, F. (2003). Bacterial contamination of computer keyboards in a teaching hospital. Infection Control and Hospital Epidemiology, 24(4), 302–303.

    Article  Google Scholar 

  11. Amer, H., Atia, A., & Tawil, K. (2017). Bacterial contamination of computer keyboards and mice in university and hospital settings. DJ International Journal of Medical Research, 2, 1–7.

    Article  Google Scholar 

  12. Wachs, J. P., Stern, H., Edan, Y., Gillam, M., Handler, J., Feied, C., & Smith, M. (2008). A gesture-based tool for sterile browsing of radiology images. Journal of the American Medical Informatics Association: JAMIA, 15(3), 321–323.

    Article  Google Scholar 

  13. Gavrilovska, L., & Rakovic, V. (2016). Human bond communications: Generic classification and technology enablers. Wireless Personal Communications, 88(1), 5–21.

    Article  Google Scholar 

  14. Cronin, S., & Doherty, G.A.-O. (2019). Touchless computer interfaces in hospitals: A review. Health Information Journal, 25(4), 1325–1342.

    Article  Google Scholar 

  15. Ezzat, A., Kogkas, A., Holt, J., Thakkar, R., Darzi, A., & Mylonas, G. (2021). An eye-tracking based robotic scrub nurse: Proof of concept. Surgical Endoscopy, 35, 5381–5391.

    Article  Google Scholar 

  16. Sharmin, S., Hoque, M. M., Islam, S. M. R., Kader, M. F., & Sarker, I. H. (2021). Development of duplex eye contact framework for human–robot inter communication. IEEE Access, 9, 54435–54456.

    Article  Google Scholar 

  17. Miehle, J., Gerstenlauer, N., Ostler, D., Feußner, H., Minker, W., & Ultes, S. (2018). Expert evaluation of a spoken dialogue system in a clinical operating room. In Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018) (pp. 735–740).

  18. Miehle, J., Ostler, D., Gerstenlauer, N., & Minker, W. (2017). The next step: Intelligent digital assistance for clinical operating rooms. Innovative Surgical Sciences, 2(3), 159–161.

    Article  Google Scholar 

  19. Schulte, A., Suarez-Ibarrola, R., Wegen, D., Pohlmann, P.-F., Petersen, E., & Miernik, A. (2020). Automatic speech recognition in the operating room—An essential contemporary tool or a redundant gadget? A survey evaluation among physicians in form of a qualitative study. Annals of Medicine and Surgery, 59, 81–85.

    Article  Google Scholar 

  20. Hurstel, A., & Bechmann, D. (2019). Approach for intuitive and touchless interaction in the operating room. J-Multidisciplinary Scientific Journal, 2(1), 50–64.

    Article  Google Scholar 

  21. Korayem, M. H., Madihi, M. A., & Vahidifar, V. (2021). Controlling surgical robot arm using leap motion controller with Kalman filter. Measurement, 178, 1–12.

    Article  Google Scholar 

  22. Despinoy, F.A.-O., Zemiti, N., Forestier, G., Sánchez, A., Jannin, P., & Poignet, P. (2018). Evaluation of contactless human–machine interface for robotic surgical training. International Journal of Computer Assisted Radiology and Surgery, 13, 13–24.

    Article  Google Scholar 

  23. Ameur, S., Khalifa, A. B., & Bouhlel, M. S. (2020). Hand-gesture-based touchless exploration of medical images with leap motion controller. In 2020 17th International multi-conference on systems, signals & devices (SSD) (pp. 6–11).

  24. Bockhacker, M., Syrek, H., Elstermann von Elster, M., Schmitt, S., & Roehl, H. (2020). Evaluating usability of a touchless image viewer in the operating room. Applied Clinical Informatics, 11(1), 88–94.

    Article  Google Scholar 

  25. Pauchot, J., Di Tommaso, L., Lounis, A., Benassarou, M., Mathieu, P., Bernot, D., & Aubry, S. (2015). Leap motion gesture control with carestream software in the operating room to control imaging: Installation guide and discussion. Surgical Innovation, 22, 615–620.

    Article  Google Scholar 

  26. Paulo, S. F., Relvas, F., Nicolau, H., Rekik, Y., Machado, V., Botelho, J., Mendes, J. J., Grisoni, L., Jorge, J., & Lopes, D. S. (2019). Touchless interaction with medical images based on 3D hand cursors supported by single-foot input: A case study in dentistry. Journal of Biomedical Informatics, 100, 103316.

    Article  Google Scholar 

  27. Wipfli, R., Dubois-Ferrière, V., Budry, S., Hoffmeyer, P., & Lovis, C. (2016). Gesture-controlled image management for operating room: A randomized crossover study to compare interaction using gestures, mouse, and third person relaying. PLoS ONE, 11(4), e0153596.

    Article  Google Scholar 

  28. Furusawa, K., Liu, J., Tsujinaga, S., Tateyama, T., Iwamoto, Y., & Chen, Y.-W. (2020). Robust hand gesture recognition using multimodal deep learning for touchless visualization of 3D medical images. In Advances in natural computation, fuzzy systems and knowledge discovery (pp. 593–600). Springer.

  29. 3dsensor blog. (2016). Programming for Kinect 4—Kinect App with Skeleton Tracking | 3dsensor blog, Febuary, 2016. http://blog.3dsense.org/programming/programming-for-kinect-4-kinect-app-with-skeleton-tracking-openni-2-0/.

  30. Jalab, H. A., & Omer, H. K. (2015). Human computer interface using hand gesture recognition based on neural network. In 5th National symposium on information technology: Towards new smart world (NSITNSW) (pp. 1–6).

  31. Dardas, N. H., & Petriu, E. M. (2011). Hand gesture detection and recognition using principal component analysis. In 2011 IEEE International conference on computational intelligence for measurement systems and applications (CIMSA) proceedings (pp. 1–6).

  32. Chu, S., & Tanaka, J. (2011). Hand gesture for taking self portrait. In J. A. Jacko (Ed.), Human–computer interaction. Interaction Techniques and Environments (pp. 238–247). Springer.

    Chapter  Google Scholar 

  33. Elakkiya, R., Selvamani, K., Kanimozhi, S., Velumadhava, R., & Kannan, A. (2012). Intelligent system for human computer interface using hand gesture recognition. Procedia Engineering, 38, 3180–3191.

    Article  Google Scholar 

  34. Jacob, M. G., Wachs, J. P., & Packer, R. A. (2013). Hand-gesture-based sterile interface for the operating room using contextual cues for the navigation of radiological images. Journal of the American Medical Informatics Association, 20(e1), e183–e186.

    Article  Google Scholar 

  35. Häggström, M. (2007). File:Computed tomography of human brain—large.png, 1199px-Computed_tomography_of_human_brain_-_large.png.

  36. Wu, B. F., Chen, B. R., & Hsu, C. F. (2021). Design of a facial landmark detection system using a dynamic optical flow approach. IEEE Access, 9, 68737–68745.

    Article  Google Scholar 

  37. Siratanita, S., Chamnongthai, K., & Muneyasu, M. (2021). A method of football-offside detection using multiple cameras for an automatic linesman assistance system. Wireless Personal Communications, 118(3), 1883–1905.

    Article  Google Scholar 

  38. Parvathi, R., & Sankar, M. (2019). An exhaustive multi factor face authentication using neuro-fuzzy approach. Wireless Personal Communications, 109(4), 2353–2375.

    Article  Google Scholar 

  39. Khan, N. S., & Ghani, M. S. (2021). A survey of deep learning based models for human activity recognition. Wireless Personal Communications, 120, 1593–1635.

    Article  Google Scholar 

Download references

Acknowledgements

This research was performed under the financial support of the Faculty of Engineering, Srinakharinwirot University (367/2558).

Author information

Authors and Affiliations

  1. H-Lab, Bangkok, Thailand

    Annop Gobhiran

  2. Lion Intermed Company, Bangkok, Thailand

    Daorutchada Wongjunda

  3. Department of Surgery, HRH Princess Maha Chakri Sirindhorn Medical Center, Faculty of Medicine, Srinakharinwirot University, Nakhon Nayok, Thailand

    Kraiyot Kiatsoontorn

  4. Department of Biomedical Engineering, Faculty of Engineering, Srinakharinwirot University, Nakorn Nayok, Thailand

    Theekapun Charoenpong

Authors
  1. Annop Gobhiran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daorutchada Wongjunda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kraiyot Kiatsoontorn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Theekapun Charoenpong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Theekapun Charoenpong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gobhiran, A., Wongjunda, D., Kiatsoontorn, K. et al. Hand Movement-Controlled Image Viewer in an Operating Room by Using Hand Movement Pattern Code. Wireless Pers Commun 123, 103–121 (2022). https://doi.org/10.1007/s11277-021-09121-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-021-09121-8

Keywords

Search

Navigation