Skip to main content
SpringerLink
Log in

Kinect technology for hand tracking control of surgical robots: technical and surgical skill comparison to current robotic masters

  • New Technology
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Current surgical robots are controlled by a mechanical master located away from the patient, tracking surgeon’s hands by wire and pulleys or mechanical linkage. Contactless hand tracking for surgical robot control is an attractive alternative, because it can be executed with minimal footprint at the patient’s bedside without impairing sterility, while eliminating current disassociation between surgeon and patient. We compared technical and technologic feasibility of contactless hand tracking to the current clinical standard master controllers.

Methods

A hand-tracking system (Kinect™-based 3Gear), a wire-based mechanical master (Mantis Duo), and a clinical mechanical linkage master (da Vinci) were evaluated for technical parameters with strong clinical relevance: system latency, static noise, robot slave tremor, and controller range. Five experienced surgeons performed a skill comparison study, evaluating the three different master controllers for efficiency and accuracy in peg transfer and pointing tasks.

Results

da Vinci had the lowest latency of 89 ms, followed by Mantis with 374 ms and 3Gear with 576 ms. Mantis and da Vinci produced zero static error. 3Gear produced average static error of 0.49 mm. The tremor of the robot used by the 3Gear and Mantis system had a radius of 1.7 mm compared with 0.5 mm for da Vinci. The three master controllers all had similar range. The surgeons took 1.98 times longer to complete the peg transfer task with the 3Gear system compared with Mantis, and 2.72 times longer with Mantis compared with da Vinci (p value 2.1e−9). For the pointer task, surgeons were most accurate with da Vinci with average error of 0.72 mm compared with Mantis’s 1.61 mm and 3Gear’s 2.41 mm (p value 0.00078).

Conclusions

Contactless hand-tracking technology as a surgical master can execute simple surgical tasks. Whereas traditional master controllers outperformed, given that contactless hand-tracking is a first-generation technology, clinical potential is promising and could become a reality with some technical improvements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Tsui C, Kelin R, Garabrant M (2013) Minimally invasive surgery: national trends in adoption and future directions for hospital strategy. Surg Endosc 27:2253–2257

    Article  PubMed  Google Scholar 

  2. Wetter PA, Kavic MS, Levinson CJ, Kelley WE, McDougall EM, Nezhat C (2005) Prevention and management of laparoendoscopic surgical complications, 2nd edn. Society of Laparoscopic Surgeons, Miami

    Google Scholar 

  3. Hartmann F, Schlaefer A (2013) Feasibility of touch-less control of operating room lights. Int J Comput Assist Radiol Surg 8:259–268. doi:10.1007/s11548-012-0778-2

    Article  PubMed  Google Scholar 

  4. Gallo L, Placitelli AP, Ciampi M (2011) Controller-free exploration of medical image data: experiencing the Kinect. IEEE Int Symp Comput Med Syst CBMS 70:1–6. doi:10.1109/CBMS.2011.5999138

    Google Scholar 

  5. Ruppert GC, Reis LO, Amorim PH, de Moraes TF, da Silva JV (2012) Touchless gesture user interface for interactive image visualization in urological surgery. World J Urol 30:687–691. doi:10.1007/s00345-012-0879-0

    Article  PubMed  Google Scholar 

  6. Jacob MG, Wachs JP, Packer RA (2013) Hand-gesture-based sterile interface for the operating room using contextual cues for the navigation of radiological images. J Am Med Inform Assoc. doi:10.1136/amiajnl-2012-001212

    PubMed  Google Scholar 

  7. Van den Bergh M, Carton D, de Nijs R, Mitsou N, Landsiedel C, Kuehnlenz K, Wollherr D (2011) Real-time 3D hand gesture interaction with a robot for understanding directions from humans. IEEE RO-MAN pp 357–362 doi:10.1109/ROMAN.2011.6005195 July 31, 2011

  8. Frati V, Prattichizzo D (2011) Using Kinect for hand tracking and rendering in wearable haptics. IEEE World Haptics Conf. WHC pp 317–321 doi:10.1109/WHC.2011.5945505 June 2011

  9. Ren Z, Yuan J, Zhang Z (2011) Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. IEEE Trans Multimed 15:1110–1120. doi:10.1109/TMM.2013.2246148

    Article  Google Scholar 

  10. Oikonomidis I, Kyriazis N, Argyros A (2011) Efficient model-based 3D tracking of hand articulations using Kinect Proceedings of the British Machine Vision Conference. doi:10.5244/C.25.101, Sep 2011

  11. Jacob MG, Li YT, Wachs JP (2011) A gesture driven robotic scrub nurse. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, pp 2039–2044. doi:10.1109/ICSMC.2011.6083972, Oct 2011

  12. Monnich H, Nicolai P, Beyl T, Raczkowsky J, Worn H (2011) A supervision system for the intuitive usage of a telemanipulated surgical robotic setup. IEEE Int Conf Robot Biomimetics ROBIO, pp 449–454 doi:10.1109/ROBIO.2011.6181327 Dec 2011

  13. Badani KK, Bhandari A, Tewari A, Menon M (2005) Comparison of two-dimensional and three-dimensional suturing: Is there a difference in a robotic surgery setting? J Endourol 19(10):1212–1215. doi:10.1089/end.2005.19.1212

    Article  PubMed  Google Scholar 

  14. Lum MJ, Rosen J, King H, Friedman DC, Lendvay TS, Wright AS, Sinanan MN, Hannaford B (2009) Teleoperation in surgical robotics – network latency effects on surgical performance. Annu Int Conf IEEE Eng Med Biol Soc, Minneapolis, pp 6860–6863 doi:10.1109/IEMBS.2009.5333120, Sept 2009

  15. Schneider CA, Rasband WS, Eliceiri KW (2012) NIH Image to ImageJ: 25 years of image analysis. Nat Methods 9:671–675. doi:10.1038/nmeth.2089

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgments

The authors thank Dr. Aaron Martin, Dr. Timothy Kane, Dr. Miller Hamrick, Dr. Craig Peters, Dr. Nora Lee, and Dr. Peter Kim for performing the surgical tasks.

Disclosures

Mr. Yonjae Kim and Drs. Simon Leonard, Azad Shademan, Axel Krieger, and Peter Kim have no conflicts of interest or financial ties to disclose.

Author information

Authors and Affiliations

  1. Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Medical Center, Room M7777, 111 Michigan Avenue, NW, Washington, DC, 20010, USA

    Yonjae Kim, Simon Leonard, Azad Shademan, Axel Krieger & Peter C. W. Kim

Authors
  1. Yonjae Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Leonard
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Azad Shademan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Axel Krieger
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peter C. W. Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Axel Krieger.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kim, Y., Leonard, S., Shademan, A. et al. Kinect technology for hand tracking control of surgical robots: technical and surgical skill comparison to current robotic masters. Surg Endosc 28, 1993–2000 (2014). https://doi.org/10.1007/s00464-013-3383-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-013-3383-8

Keywords

Search

Navigation