Surgical model-view-controller simulation software framework for local and collaborative applications
Abstract
Purpose
Surgical simulations require haptic interactions and collaboration in a shared virtual environment. A software framework for decoupled surgical simulation based on a multi-controller and multi-viewer model-view-controller (MVC) pattern was developed and tested.
Methods
A software framework for multimodal virtual environments was designed, supporting both visual interactions and haptic feedback while providing developers with an integration tool for heterogeneous architectures maintaining high performance, simplicity of implementation, and straightforward extension. The framework uses decoupled simulation with updates of over 1,000 Hz for haptics and accommodates networked simulation with delays of over 1,000 ms without performance penalty.
Results
The simulation software framework was implemented and was used to support the design of virtual reality-based surgery simulation systems. The framework supports the high level of complexity of such applications and the fast response required for interaction with haptics. The efficacy of the framework was tested by implementation of a minimally invasive surgery simulator.
Conclusion
A decoupled simulation approach can be implemented as a framework to handle simultaneous processes of the system at the various frame rates each process requires. The framework was successfully used to develop collaborative virtual environments (VEs) involving geographically distributed users connected through a network, with the results comparable to VEs for local users.
Keywords
Surgical simulation Virtual environments Physics-based simulation User interfaces—Haptic I/O Software frameworkPreview
Unable to display preview. Download preview PDF.
Supplementary material
ESM 1 (MPG 28,332 kb)
References
- 1.Adl-Tabatabai A-R, Gross T, Lueh G-Y (1996) Code reuse in an optimizing compiler. OOPSLA ‘96. In: Proceedings of the 11th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, ACM, pp 51–68Google Scholar
- 2.Allard J, Cotin S, Faure F, Bensoussan P-J, Poyer F, Duriez C, Delingette H, Grisoni L (2007) SOFA an open source framework for medical simulation. Medicine meets virtual reality (MMVR’15)Google Scholar
- 3.Ayache N, Delingette H (2003) Surgery simulation and soft tissue modeling, international symposium, IS4TM 2003, Juan-Les-Pins, France, June 12–13, SpringerGoogle Scholar
- 4.Basdogan C, De S, Kim J, Muniyandi M, Kim H, Srinivasan MA (2004) Haptics in minimally invasive surgical simulation and training. IEEE Comput Graph Appl 24(2): 56–64PubMedCrossRefGoogle Scholar
- 5.Boeing A, Bräunl T (2007) Evaluation of real-time physics simulation systems. ‘GRAPHITE ‘07. In: Proceedings of the 5th international conference on computer graphics and interactive techniques in Australia and Southeast Asia’, ACM, New York, NY, USA, pp 281–288Google Scholar
- 6.Buttolo P, Oboe R, Hannaford B (2000) Architectures for shared haptic virtual environments. Compt Graph 21(4): 478–483Google Scholar
- 7.De S, Lim Y-J, Muniyandi M, Srinivasan MA (2006) Physically realistic virtual surgery using the point-associated finite field (PAFF) approach. Presence, 2006, 15, 294–308. Presence: teleoperators and virtual environments, 1997, 6, 147–159Google Scholar
- 8.Deo D, De S, A physics-driven neural networks-based surgery simulation system (PhyNNeSS) for multimodal interactive virtual environments involving nonlinear deformation objects. Submitted to PresenceGoogle Scholar
- 9.El-Far NR, Nourian S, Jilin Zhou, Hamam A, Xiaojun Shen, Georganas ND (2005) A cataract tele-surgery training application in a hapto-visual collaborative environment running over the CANARIE photonic network, Haptic audio visual environments and their applications. IEEE Int Workshop, 4:1–2Google Scholar
- 10.Gamma E, Helm R, Johnson R, Vlissides J (1995) Design patterns: elements of reusable object-oriented software. Addison-Wesley Professional, MassachusettsGoogle Scholar
- 11.Gottschalk S, Lin MC, Manocha D (1996) OBBTree: a hierarchical structure for rapid interference detection. SIGGRAPH ‘96. In: Proceedings of the 23rd annual conference on computer graphics and interactive techniques, ACM Press, pp 171–180Google Scholar
- 12.Gunn C, Hutchins M, Adcock M, Hawkins R (2004) Surgical training using haptics over long internet distances. Stud health technol inform 98. Published by IOS Press, pp 121–123Google Scholar
- 13.Hermann E, Raffin B, Faure F (2009) Interactive Physical Simulation on Multicore Architectures. In: Eurographics workshop on parallel and graphics and visualization, EGPGV’09. Munich, Germany, March 2009Google Scholar
- 14.Jeffrey H, Gerald MF, Lee LS, Nathaniel JS, Lelan FS, Bruce S, Kaaren H The SAGES FLS Committee: (2004) Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery 135(1): 21–27CrossRefGoogle Scholar
- 15.Lin MC, Manocha D (2000) Fast proximity queries with swept sphere volume. In: Proceedings of international conference on robotics and automation, pp 3719–3726, April 2000Google Scholar
- 16.Liu A, Tendick F, Cleary K, Kaufmann C (2003) A survey of surgical simulation: applications, technology, and education. Presence 12(6): 599–614CrossRefGoogle Scholar
- 17.Teschner M, Kimmerle S, Heidelberger B, Zachmann G, Raghupathi L, Fuhrmann A, Cani M-P, Faure F, Magnenat-Thalmann N, Strasser W, Volino P (2005) Collision detection for deformable objects. Comput Graph Forum 24(1): 61–81CrossRefGoogle Scholar
- 18.Maciel A, Halic T, Lu Z, Nedel LP, De S (2009) Using the PhysX engine for physics-based virtual surgery with force feedback. Int J Med Robot 5: 341–353PubMedGoogle Scholar
- 19.Maciel A, De S (2008) An efficient dynamic point algorithm for line-based collision detection in real time virtual environments involving haptics. Comput Animat Virtual Worlds 19(2): 151–163CrossRefGoogle Scholar
- 20.Maciel A, Liu Y, Ahn W, Singh TP, Dunnican W, De S (2008) Development of the VBLaST TM: a virtual basic laparoscopic skill trainer. Int J Med Robot 4(2): 131–138PubMedGoogle Scholar
- 21.Maciel A, Boulic R, Thalmann D (2007) Efficient collision detection within deforming spherical sliding contact. IEEE transactions on visualization and computer graphics. IEEE Comput Soci 13: 518–529Google Scholar
- 22.Marks S, Windsor J, Wünsche B (2007) Evaluation of game engines for simulated surgical training, GRAPHITE ‘07: In: Proceedings of the 5th international conference on computer graphics and interactive techniques in Australia and Southeast Asia, ACM, New York, pp 273–280Google Scholar
- 23.Ponder M, Papagiannakis G, Molet T, Magnenat-Thalmann N, Thalmann D (2003) VHD++ development framework: towards extendible, component based VR/AR simulation engine featuring advanced virtual character technologies. In: Proceedings computer graphics international. IEEE Computer Society, 2003Google Scholar
- 24.Pree W (1994) Meta patterns—A means for capturing the essentials of reusable object-oriented design, ECOOP ‘94. In: Proceedings of the 8th European conference on object-oriented programming, Springer, pp 150–162. (frozen spots/hot spots)Google Scholar
- 25.Rihele D (2000) Framework design: a role modeling approach. ETH Zürich, ZürichGoogle Scholar
- 26.Ruffaldi E, Frisoli A, Bergamasco M, Gottlieb C, Tecchia F (2006) A haptic toolkit for the development of immersive and web-enabled games. In: VRST ‘06: Proceedings of the ACM symposium on virtual reality software and technology. pp 320–323. ACM, New YorkGoogle Scholar
- 27.Saupin G, Duriez C, Cotin S (2008) Contact model for haptic medical simulations. ISBMS, Switzerland, pp 157–165Google Scholar
- 28.Schroeder W, Martin K, Lorensen B (1997) The visualization toolkit: an object-oriented approach to 3-D graphics. Prentice Hall, New JerseyGoogle Scholar
- 29.Seugling A, Rölin M (2006) Evaluation of physics engines and implementation of a physics module in a 3d-authoring tool. Umea University, SwedenGoogle Scholar
- 30.Shaw C, Green M, Liang J, Sun Y (1993) Decoupled simulation in virtual reality with the MR toolkit. ACM transactions information system vol 11, no. 3. ACM, New York, pp 287–317Google Scholar
- 31.Vlissides JM, Linton MA (1990) Unidraw: a framework for building domain-specific graphical editors. ACM Trans Inf Syst 8: 237–268CrossRefGoogle Scholar
- 32.Yeongho K, Jeong-Ho C, Jongki L, Myeng KK, Nam KK, Jin SY, Yong OK (2001) Collaborative surgical simulation over the internet. Int Comput IEEE 5(3): 65–73CrossRefGoogle Scholar
- 33.Zhou J, Shen X, Georganas ND (2004) Haptic tele-surgery simulation, haptic, audio and visual environments and their applications. HAVE, 2004. In: Proceedings the 3rd IEEE international workshop on, pp 99–104, 2–3, October 2004Google Scholar
- 34.Zhou J, Shen X, Saddik AE, Georganas ND (2007) Design and implementation of haptic tele-mentoring over the internet. In: Proceedings of the 11th IEEE international symposium on distributed simulation and real-time applications (October 22–26, 2007)Google Scholar
- 35.Zhu B, Gu L, Peng X, Zhou Z (2010) A point-based simulation framework for minimally invasive surgery. In: Székely et al. (ed), Biomedical simulation. 5th international symposium, ISBMS 2010. Springer, LNCSGoogle Scholar