Skip to main content
SpringerLink
Log in

Validation of New Procedures and Training Processes Through Physical Task Analysis

  • Chapter
  • First Online:
Intraoperative Imaging and Image-Guided Therapy

  • 3590 Accesses

Abstract

Image-guided therapy (IGT) procedures offer many possibilities for improved clinical care. However, the complexity and variety of IGT approaches challenge the ability of developers to optimize the development process, so that the best new systems are created. As well, it is difficult to train users of these complex techniques efficiently and effectively. Here we discuss methods to characterize the relative performance of these systems (and their operators) by measuring the movement of the operator and his or her instruments, which provides a rich and explicit data set. We describe three classes of analysis: deterministic, based on recorded positions and kinematics; stochastic, implemented through machine-learning algorithms; and user surveys. A short summary of useful psychosomatic analyses is also given.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Kundel HL. Visual cues in the interpretation of medical images. J Clin Neurophysiol. 1990;7(4):472–83.

    Article  CAS  PubMed  Google Scholar 

  2. Jannin P, Korb W. Assessment of image-guided interventions. Berlin/Heidelberg: Springer; 2008.

    Google Scholar 

  3. Darzi A, Smith S, Taffinder N. Assessing operative skill. Needs to become more objective. Br Med J. 1999;318(7188):887–8.

    Article  CAS  Google Scholar 

  4. Shah SG, Thomas-Gibson S, Brooker JC, et al. Use of video and magnetic endoscope imaging for rating competence at colonoscopy: validation of a measurement tool. Gastrointest Endosc. 2002;56(4):568–73.

    Article  PubMed  Google Scholar 

  5. Fried GM. Lessons from the surgical experience with simulators: incorporation into training and utilization in determining competency. Gastrointest Endosc Clin N Am. 2006;16(3):425–34.

    Article  PubMed  Google Scholar 

  6. Fried GM, Feldman LS, Vassiliou MC, et al. Proving the value of simulation in laparoscopic surgery. Ann Surg. 2004;240(3):518–25; discussion 525–518.

    Article  PubMed  Google Scholar 

  7. Hamdorf JM, Hall JC. Acquiring surgical skills. Br J Surg. 2000;87(1):22–37.

    Article  Google Scholar 

  8. Moorthy K, Munz Y, Sarker SK, Darzi A. Objective assessment of technical skills in surgery. Br Med J. 2003;327(7422):1032–7.

    Article  Google Scholar 

  9. Satava RM, Cuschieri A, Hamdorf J. Metrics for objective assessment. Surg Endosc. 2003;17(2):220–6.

    Article  CAS  PubMed  Google Scholar 

  10. Peters JH, Fried GM, Swanstrom LL, et al. Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery. 2004;135(1):21–7.

    Article  PubMed  Google Scholar 

  11. Gallagher AG, Smith CD, Bowers SP, et al. Psychomotor skills assessment in practicing surgeons experienced in performing advanced laparoscopic procedures. J Am Coll Surg. 2003;197(3):479–88.

    Article  PubMed  Google Scholar 

  12. Vosburgh KG, Stoll J, Noble V, Pohl K, San Jose Estepar R, Takacs B. Image registration assists novice operators in ultrasound assessment of abdominal trauma. Stud Health Technol Inform. 2008;132:532–7.

    PubMed  Google Scholar 

  13. Stylopoulos N, Cotin S, Dawson S, et al. CELTS: a clinically-based computer enhanced laparoscopic training system. Stud Health Technol Inform. 2003;94:336–42.

    PubMed  Google Scholar 

  14. Obstein KL, Patil VD, Jayender J, et al. Evaluation of colonoscopy technical skill levels by use of an objective kinematic-based system. Gastrointest Endosc. 2011;73(2):315–21, 321.e311.

    Article  PubMed Central  PubMed  Google Scholar 

  15. Jayender J, San Jose Estepar R, Vosburgh K. New kinematic metric for quantifying surgical skill for flexible instrument manipulation. Int Conf Inf Process Comput Assist Interv. 2010;6135:81–90.

    Google Scholar 

  16. Vosburgh KG, Stylopoulos N, Estepar RS, Ellis RE, Samset E, Thompson CC. EUS with CT improves efficiency and structure identification over conventional EUS. Gastrointest Endosc. 2007;65(6):866–70.

    Article  PubMed  Google Scholar 

  17. Mackay S, Datta V, Mandalia M, Bassett P, Darzi A. Electromagnetic motion analysis in the assessment of surgical skill: relationship between time and movement. ANZ J Surg. 2002;72(9):632–4.

    Article  PubMed  Google Scholar 

  18. Francis NK, Hanna GB, Cuschieri A. The performance of master surgeons on the advanced Dundee endoscopic psychomotor tester: contrast validity study. Arch Surg. 2002;137(7):841–4.

    Article  PubMed  Google Scholar 

  19. Aggarwal R, Grantcharov T, Moorthy K, et al. An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg. 2007;245(6):992–9.

    Article  PubMed  Google Scholar 

  20. Neary PC, Boyle E, Delaney CP, Senagore AJ, Keane FB, Gallagher AG. Construct validation of a novel hybrid virtual-reality simulator for training and assessing laparoscopic colectomy; results from the first course for experienced senior laparoscopic surgeons. Surg Endosc. 2008;22(10):2301–9.

    Article  PubMed  Google Scholar 

  21. Ferlitsch A, Glauninger P, Gupper A, et al. Evaluation of a virtual endoscopy simulator for training in gastrointestinal endoscopy. Endoscopy. 2002;34(9):698–702.

    Article  CAS  PubMed  Google Scholar 

  22. Moody L, Baber C, Arvanitis TN, Elliott M. Objective metrics for the evaluation of simple surgical skills in real and virtual domains. Presence Teleoperators Virtual Environ. 2003;12(2):207–21.

    Google Scholar 

  23. Acosta E, Temkin B. Haptic laparoscopic skills trainer with practical user evaluation metrics. Stud Health Technol Inform. 2005;111:8–11.

    PubMed  Google Scholar 

  24. Lendvay T, Casale P, Sweet R, Peters C. Initial validation of a virtual-reality robotic simulator. J Robot Surg. 2008;2(3):145–9.

    Article  Google Scholar 

  25. Salvatore S, Massimiliano Z, Zhuohua L, et al. Proceedings of the 2009 IEEE/RSJ international conference on intelligent robots and systems. St. Louis: IEEE Press; 2009.

    Google Scholar 

  26. Wilson M, McGrath J, Vine S, Brewer J, Defriend D, Masters R. Psychomotor control in a virtual laparoscopic surgery training environment: gaze control parameters differentiate novices from experts. Surg Endosc. 2010;24(10):2458–64.

    Article  PubMed Central  PubMed  Google Scholar 

  27. Duffy AJ, Hogle NJ, McCarthy H, et al. Construct validity for the LAPSIM laparoscopic surgical simulator. Surg Endosc. 2005;19(3):401–5.

    Article  CAS  PubMed  Google Scholar 

  28. Cotin S, Stylopoulos N, Ottensmeyer MP, Neumann PF, Rattner D, Dawson S. Metrics for laparoscopic skills trainers: the weakest link! In: Proceedings of the 5th international conference on medical image computing and computer-assisted intervention-part I. Berlin/Heidelberg: Springer; 2002.

    Google Scholar 

  29. Estepar RS, Stylopoulos N, Ellis R, et al. Towards scarless surgery: an endoscopic ultrasound navigation system for transgastric access procedures. Comput Aided Surg. 2007;12(6):311–24.

    Article  PubMed  Google Scholar 

  30. Datta V, Mackay S, Darzi A, Gillies D. Motion analysis in the assessment of surgical skill. Comput Methods Biomech Biomed Engin. 2001;4:515–23.

    Article  Google Scholar 

  31. Verner L, Oleynikov D, Holtmann S, Haider H, Zhukov L. Measurements of the level of surgical expertise using flight path analysis from da Vinci robotic surgical system. Stud Health Technol Inform. 2003;94:373–8.

    PubMed  Google Scholar 

  32. Grober ED, Hamstra SJ, Wanzel KR, et al. Validation of novel and objective measures of microsurgical skill: hand-motion analysis and stereoscopic visual acuity. Microsurgery. 2003;23(4):317–22.

    Article  PubMed  Google Scholar 

  33. Cristancho SM, Hodgson AJ, Panton N, Meneghetti A, Qayumi K. Feasibility of using intraoperatively-acquired quantitative kinematic measures to monitor development of laparoscopic skill. Stud Health Technol Inform. 2007;125:85–90.

    PubMed  Google Scholar 

  34. Megali G, Sinigaglia S, Tonet O, Dario P. Modelling and evaluation of surgical performance using hidden Markov models. IEEE Trans Biomed Eng. 2006;53(10):1911–9.

    Article  PubMed  Google Scholar 

  35. Zhuohua L, Uemura M, Zecca M, et al. Objective evaluation of laparoscopic surgical skills using Waseda bioinstrumentation system WB-3. Paper presented at: robotics and biomimetics (ROBIO), 2010 IEEE International Conference on Tianjin, China; 14–18 Dec 2010.

    Google Scholar 

  36. Taffinder N, Smith S, Mair J, Russell R, Darzi A. Can a computer measure surgical precision? Reliability, validity and feasibility of the ICSAD. Surg Endosc. 1999;13:81.

    Google Scholar 

  37. Smith SG, Torkington J, Brown TJ, Taffinder NJ, Darzi A. Motion analysis. Surg Endosc. 2002;16(4):640–5.

    Article  CAS  PubMed  Google Scholar 

  38. Korman LY, Egorov V, Tsuryupa S, et al. Characterization of forces applied by endoscopists during colonoscopy by using a wireless colonoscopy force monitor. Gastrointest Endosc. 2010;71(2):327–34.

    Article  PubMed Central  PubMed  Google Scholar 

  39. Dohi T, Kikinis R, Yamauchi Y, et al. Surgical skill evaluation by force data for endoscopic sinus surgery training system. In: Medical image computing and computer-assisted intervention — MICCAI 2002, vol. 2488. Berlin/Heidelberg: Springer; 2002. p. 44–51.

    Chapter  Google Scholar 

  40. Yasushi Y, Juli Y, Osamu M, et al. Surgical skill evaluation by force data for endoscopic sinus surgery training system. In: Proceedings of the 5th international conference on medical image computing and computer-assisted intervention-part I. Berlin/Heidelberg: Springer; 2002.

    Google Scholar 

  41. O’Toole III RV, Playter R, Krummel T, et al. Assessing skill and learning in surgeons and medical students using a force feedback surgical simulator. In: Proceedings of the first international conference on medical image computing and computer-assisted intervention. Berlin/New York: Springer; 1998.

    Google Scholar 

  42. McBeth PB, Louw DF, Yang F, Sutherland GR. Quantitative measures of performance in microvascular anastomoses. Comput Aided Surg. 2005;10(3):173–80.

    PubMed  Google Scholar 

  43. Stylopoulos N, Vosburgh KG. Assessing technical skill in surgery and endoscopy: a set of metrics and an algorithm (C-PASS) to assess skills in surgical and endoscopic procedures. Surg Innov. 2007;14(2):113–21.

    Article  PubMed  Google Scholar 

  44. Zheng B, Denk PM, Martinec DV, Gatta P, Whiteford MH, Swanstrom LL. Building an efficient surgical team using a bench model simulation: construct validity of the legacy inanimate system for endoscopic team training (LISETT). Surg Endosc. 2008;22(4):930–7.

    Article  CAS  PubMed  Google Scholar 

  45. Zheng B, Swanstrom LL. Video analysis of anticipatory movements performed by surgeons during laparoscopic procedures. Surg Endosc. 2009;23(7):1494–8.

    Article  PubMed  Google Scholar 

  46. Jayaraman S, Apriasz I, Trejos AL, Bassan H, Patel RV, Schlachta CM. Novel hands-free pointer improves instruction efficiency in laparoscopic surgery. Surg Innov. 2009;16(1):73–7.

    Article  PubMed  Google Scholar 

  47. Pearl JP, Marks JM. The future of teaching surgical endoscopy. Surg Innov. 2006;13(4):280–2.

    Article  PubMed  Google Scholar 

  48. Malik A, Mellinger JD, Hazey JW, Dunkin BJ, MacFadyen Jr BV. Endoluminal and transluminal surgery: current status and future possibilities. Surg Endosc. 2006;20(8):1179–92.

    Article  CAS  PubMed  Google Scholar 

  49. Shirai Y, Yoshida T, Shiraishi R, et al. Prospective randomized study on the use of a computer-based endoscopic simulator for training in esophagogastroduodenoscopy. J Gastroenterol Hepatol. 2008;23(7 Pt 1):1046–50.

    Article  PubMed  Google Scholar 

  50. Cohen J, Cohen SA, Vora KC, et al. Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest Endosc. 2006;64(3):361–8.

    Article  PubMed  Google Scholar 

  51. Harewood GC, Petersen BT, Ott BJ. Prospective assessment of the impact of feedback on colonoscopy performance. Aliment Pharmacol Ther. 2006;24(2):313–8.

    Article  CAS  PubMed  Google Scholar 

  52. Stocco L, Salcudean SE, Sassani F. Fast constrained global minimax optimization of robot parameters. Robotica. 1998;16(6):595–605.

    Article  Google Scholar 

  53. Spencer F. Teaching and measuring surgical techniques: the technical evaluation of competence. Bull Am Coll Surg. 1978;63(3):9–12.

    Google Scholar 

  54. Rosen J, Solazzo M, Hannaford B, Sinanan M. Task decomposition of laparoscopic surgery for objective evaluation of surgical residents’ learning curve using hidden Markov model. Comput Aided Surg. 2002;7(1):49–61.

    Article  PubMed  Google Scholar 

  55. Leong JJ, Nicolaou M, Atallah L, Mylonas GP, Darzi AW, Yang GZ. HMM assessment of quality of movement trajectory in laparoscopic surgery. Comput Aided Surg. 2007;12(6):335–46.

    PubMed  Google Scholar 

  56. Leong JJ, Nicolaou M, Atallah L, Mylonas GP, Darzi AW, Yang GZ. HMM assessment of quality of movement trajectory in laparoscopic surgery. Med Image Comput Comput Assist Interv. 2006;9(Pt 1):752–9.

    PubMed  Google Scholar 

  57. Lin HC, Shafran I, Murphy TE, Okamura AM, Yuh DD, Hager GD. Automatic detection and segmentation of robot-assisted surgical motions. Med Image Comput Comput Assist Interv. 2005;8(Pt 1):802–10.

    PubMed  Google Scholar 

  58. Lin HC, Shafran I, Yuh D, Hager GD. Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. Comput Aided Surg. 2006;11(5):220–30.

    PubMed  Google Scholar 

  59. Reiley CE, Hager GD. Task versus subtask surgical skill evaluation of robotic minimally invasive surgery. Med Image Comput Comput Assist Interv. 2009;12(Pt 1):435–42.

    PubMed  Google Scholar 

  60. Chen J, Yeasin M, Sharma R. Visual modelling and evaluation of surgical skill. Pattern Anal Appl. 2003;6(1):1–11.

    Article  Google Scholar 

  61. Ahmadi SA, Sielhorst T, Stauder R, Horn M, Feussner H, Navab N. Recovery of surgical workflow without explicit models. Med Image Comput Comput Assist Interv. 2006;9(Pt 1):420–8.

    PubMed  Google Scholar 

  62. Kanav K, Narayanan CK, Vineeth NB, Sethuraman P, Marshall S, John F. Measuring movement expertise in surgical tasks. In: Proceedings of the 14th annual ACM international conference on multimedia. Santa Barbara: ACM; 2006.

    Google Scholar 

  63. Speidel S, Zentek T, Sudra G, et al. Recognition of surgical skills using hidden Markov models. Paper presented at: Proc. SPIE 7261. Florida; 2009.

    Google Scholar 

  64. Mackel T, Rosen J, Pugh C. Application of hidden markov modeling to objective medical skill evaluation. Stud Health Technol Inform. 2007;125:316–8.

    PubMed  Google Scholar 

  65. Sewell C, Morris D, Blevins NH, et al. Providing metrics and performance feedback in a surgical simulator. Comput Aided Surg. 2008;13(2):63–81.

    PubMed  Google Scholar 

  66. Rabiner LR. A tutorial on hidden Markov models and selected applications in speech recognition. Proc of the IEEE. 1989;77(2):257–86.

    Article  Google Scholar 

  67. Hartigan J, Wong M. A K-means clustering algorithm. JR Sta Soc Ser C. 1979;28:100–8.

    Google Scholar 

  68. Lentz GM, Mandel LS, Lee D, Gardella C, Melville J, Goff BA. Testing surgical skills of obstetric and gynecologic residents in a bench laboratory setting: validity and reliability. Am J Obstet Gynecol. 2001;184(7):1462–8; discussion 1468–1470.

    Article  CAS  PubMed  Google Scholar 

  69. Sedlack RE. The mayo colonoscopy skills assessment tool: validation of a unique instrument to assess colonoscopy skills in trainees. Gastrointest Endosc. 2010;72(6):1125–33.

    Article  PubMed  Google Scholar 

  70. Cremers SL, Ciolino JB, Ferrufino-Ponce ZK, Henderson BA. Objective assessment of skills in intraocular surgery (OASIS). Ophthalmology. 2005;112(7):1236–41.

    Article  PubMed  Google Scholar 

  71. Cremers SL, Lora AN, Ferrufino-Ponce ZK. Global rating assessment of skills in intraocular surgery (GRASIS). Ophthalmology. 2005;112(10):1655–60.

    Article  PubMed  Google Scholar 

  72. de Leval MR, Carthey J, Wright DJ, Farewell VT, Reason JT. Human factors and cardiac surgery: a multicenter study. J Thorac Cardiovasc Surg. 2000;119(4 Pt 1):661–72.

    Article  PubMed  Google Scholar 

  73. Schaeffer H, Helmreich R. The operating room management attitudes questionnaire. Texas: Austin; 1993.

    Google Scholar 

  74. Helmreich RL, Schaefer HG. Team performance in the operating room. Bogner, Marilyn Sue (Ed); Hillsdale, NJ, England. 1994.

    Google Scholar 

  75. Subramonian K, DeSylva S, Bishai P, Thompson P, Muir G. Acquiring surgical skills: a comparative study of open versus laparoscopic surgery. Eur Urol. 2004;45(3):346–51.

    Article  PubMed  Google Scholar 

  76. Rosser Jr JC, Rosser LE, Savalgi RS. Objective evaluation of a laparoscopic surgical skill program for residents and senior surgeons. Arch Surg. 1998;133(6):657–61.

    Article  PubMed  Google Scholar 

  77. Goff BA, Lentz GM, Lee D, Houmard B, Mandel LS. Development of an objective structured assessment of technical skills for obstetric and gynecology residents. Obstet Gynecol. 2000;96(1):146–50.

    Article  CAS  PubMed  Google Scholar 

  78. Vassiliou MC, Feldman LS, Andrew CG, et al. A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg. 2005;190(1):107–13.

    Article  PubMed  Google Scholar 

  79. Hart SG, Staveland LE. Development of NASA-TLX (task load index): results of empirical and theoretical research. Human Mental Workload. 1988;1:139–83.

    Article  Google Scholar 

Download references

Acknowledgement

J. Jayender and Kirby G. Vosburgh were supported by the NIH under grants 2R42 CA 115112 and U41 RR019703. Dr. Vosburgh was also supported by the Center for Integration of Medicine and Innovative Technology (CIMIT).

Author information

Authors and Affiliations

  1. National Center for Image Guided Therapy, Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA

    Jagadeesan Jayender PhD & Kirby G. Vosburgh PhD

Authors
  1. Jagadeesan Jayender PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kirby G. Vosburgh PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jagadeesan Jayender PhD .

Editor information

Editors and Affiliations

  1. National Center for Image Guided Therapy, Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, USA

    Ferenc A. Jolesz

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media New York

About this chapter

Cite this chapter

Jayender, J., Vosburgh, K.G. (2014). Validation of New Procedures and Training Processes Through Physical Task Analysis. In: Jolesz, F. (eds) Intraoperative Imaging and Image-Guided Therapy. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7657-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-7657-3_8

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-7656-6

  • Online ISBN: 978-1-4614-7657-3

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation