In recent years, minimally invasive arthroscopic surgery has replaced a number of conventional open orthopedic surgery procedures on joints. While this achieves a number of advantages for the patient, the surgeons have to learn very different skills, since the surgery is performed with special miniature pencil-like tools and cameras inserted through little incisions while observing the surgical field on video monitor. Therefore, virtual reality simulation becomes an alternative to traditional surgical training based on hundreds years old apprentice–master model that involves either real patients or increasingly difficult to procure cadavers. Normally, 3D simulation of the virtual surgical field requires significant efforts from the software developers but yet remains not always photorealistic. In contrast to this, for photorealistic visualization and haptic interaction with the surgical field we propose to use real arthroscopic images augmented with 3D object models. The proposed technique allows for feeling the joint cavity displayed on video monitor as real 3D objects rather than their images while various surgical procedures, such as menisectomy, are simulated in real time. In the preprocessing stage of the proposed approach, the arthroscopic images are stitched into panoramas and augmented with implicitly defined object models representing deformable menisci. In the simulation loop, depth information from the mixed scene is used for haptic rendering. The scene depth map and visual display are reevaluated only when the scene is modified.
This is a preview of subscription content, log in to check access.
Buy single article
Instant unlimited access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Seymour, N.E., Gallagher, A.G., Roman, S.A., O’Brien, M.K., Bansal, V.K., Andersen, D.K., Satava, R.M.: Virtual reality training improves operating room performance. Ann. Surg. 236(4), 458–464 (2002)
Rasool, S., Sourin, A.: Tangible arthroscopic images. In: Proc. of Comput. Graph. Int. CGI (2012, in press)
Mabrey, J., Reinig, K., Cannon, W.: Virtual reality in orthopaedics: is it a reality? Clin. Orthop. Relat. Res. 468, 2586–2591 (2010). doi:10.1007/s11999-010-1426-1
Ackerman, M.J.: The visible human project: a resource for anatomical visualization. In: Proc. of the IEEE Int. Conf. Inf. Technol. Appl. in Biomedicine, 1977. ITAB ’97, pp. 29–31 (1997)
Heng, P.A., Chun-Yiu, C., Tien-Tsin, W., Yangsheng, X., Yim-Pan, C., Kai-Ming, C., Shiu-Kit, T.: A virtual-reality training system for knee arthroscopic surgery. IEEE Trans. Inf. Technol. Biomed. 8, 217–227 (2002)
Megali, G., Tonet, O., Mazzoni, M., Dario, P., Vascellari, A., Marcacci, M.: A new tool for surgical training in knee arthroscopy. In: Proc. of Med. Image Comput. and Comput.-Assist. Interv.—MICCAI 2002. Lecture Notes in Computer Science, vol. 2489, pp. 170–177. Springer, Berlin (2002)
Bayona, S., Espadero, J., Pastor, L., Fernandez-Arroyo, J.M.: A low-cost arthroscopy surgery training system. In: Proc. of the IASTED Int. Conf. on Vis., Imaging and Image Processing VIIP, Bemnalmadena, Spain, September 2003
Muller, W., Bockholt, U.: The virtual reality arthroscopy training simulator. In: Proc. of Med. Meets Virtual Reality (MMVR)-Art, Science, Technology: Healthcare (1998)
Gibson, S., Samosky, J., Mor, A.: Simulating arthroscopic knee surgery using volumetric object representations, real-time volume rendering and haptic feedback. In: Proc. of CVRMed-MRCAS, pp. 369–378 (1997)
Mabrey, J.D., Gillogly, S.D., Kasser, J.R., Sweeney, H.J., Zarins, B., Mevis, H., Garrett, W.E. Jr., Poss, R., Cannon, W.D.: Virtual reality simulation of arthroscopy of the knee. Arthroscopy 18 (2002)
Sherman, K.P., Ward, J.W., Wills, D.P.M., Mohsen, A.M.M.A.: A portable virtual environment knee arthroscopy training system with objective scoring. In: Proc. of Med. Meets Virtual Reality (MMVR)—The Convergence of Phys. & Inf. Technol.: Options for a New Era in Healthcare, pp. 335–336 (1999)
(GMV Innovative Solutions). insightArthroVR: virtual reality arthroscopy trainer, 2010. www.gmv.com/insight/. Accessed: 12 May 2010
(Simbionix Corporation). Arthro mentor, 2012. http://simbionix.com/simulators/arthro-mentor/. Accessed: 16 May 2012
(VirtaMed AG). Virtamed arthroscopy simulator, 2012. http://www.virtamed.com/products/arthros/. Accessed: 16 May 2012
Rasool, S., Sourin, A.: Tangible images. In: SIGGRAPH Asia 2011 Sketches, SA ’11, pp. 41:1–41:2. ACM, New York (2011)
Rasool, S., Sourin, A.: Haptic interaction with 2d images. In: Proc. of the 10th Int. Conf. on Virtual Reality Continuum and Its Appl. in Industry, VRCAI ’11, pp. 13–22. ACM, New York (2011)
This project is supported by the Singapore National Research Foundation Interactive Digital Media R&D Program, under research Grant NRF2008IDM-IDM 004-002 “Visual and Haptic Rendering in Co-Space”, by the Ministry of Education of Singapore Grant MOE 2011-T2-1-006 “Collaborative Haptic Modeling for Orthopedic Surgery Training in Cyberspace” and by the Russian Foundation for Basic Research grant 12-07-00157-a “Virtual Modeling of Minimally Invasive Surgery Operations.” The authors are very grateful to Fareed Kagda, M.D. for providing medical data and collaboration on developing the surgery training system.
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Rasool, S., Sourin, A. Image-driven virtual simulation of arthroscopy. Vis Comput 29, 333–344 (2013). https://doi.org/10.1007/s00371-012-0736-6
- Image-driven haptic interaction
- Virtual surgery simulation
- Tangible images
- Augmented reality