The Visual Computer

, Volume 29, Issue 5, pp 333–344 | Cite as

Image-driven virtual simulation of arthroscopy

  • Shahzad RasoolEmail author
  • Alexei Sourin
Original Article


In recent years, minimally invasive arthroscopic surgery has replaced a number of conventional open orthopedic surgery procedures on joints. While this achieves a number of advantages for the patient, the surgeons have to learn very different skills, since the surgery is performed with special miniature pencil-like tools and cameras inserted through little incisions while observing the surgical field on video monitor. Therefore, virtual reality simulation becomes an alternative to traditional surgical training based on hundreds years old apprentice–master model that involves either real patients or increasingly difficult to procure cadavers. Normally, 3D simulation of the virtual surgical field requires significant efforts from the software developers but yet remains not always photorealistic. In contrast to this, for photorealistic visualization and haptic interaction with the surgical field we propose to use real arthroscopic images augmented with 3D object models. The proposed technique allows for feeling the joint cavity displayed on video monitor as real 3D objects rather than their images while various surgical procedures, such as menisectomy, are simulated in real time. In the preprocessing stage of the proposed approach, the arthroscopic images are stitched into panoramas and augmented with implicitly defined object models representing deformable menisci. In the simulation loop, depth information from the mixed scene is used for haptic rendering. The scene depth map and visual display are reevaluated only when the scene is modified.


Image-driven haptic interaction Virtual surgery simulation Tangible images Augmented reality 



This project is supported by the Singapore National Research Foundation Interactive Digital Media R&D Program, under research Grant NRF2008IDM-IDM 004-002 “Visual and Haptic Rendering in Co-Space”, by the Ministry of Education of Singapore Grant MOE 2011-T2-1-006 “Collaborative Haptic Modeling for Orthopedic Surgery Training in Cyberspace” and by the Russian Foundation for Basic Research grant 12-07-00157-a “Virtual Modeling of Minimally Invasive Surgery Operations.” The authors are very grateful to Fareed Kagda, M.D. for providing medical data and collaboration on developing the surgery training system.

Supplementary material

(MP4 12.9 MB)


  1. 1.
    Seymour, N.E., Gallagher, A.G., Roman, S.A., O’Brien, M.K., Bansal, V.K., Andersen, D.K., Satava, R.M.: Virtual reality training improves operating room performance. Ann. Surg. 236(4), 458–464 (2002) CrossRefGoogle Scholar
  2. 2.
    Rasool, S., Sourin, A.: Tangible arthroscopic images. In: Proc. of Comput. Graph. Int. CGI (2012, in press) Google Scholar
  3. 3.
    Mabrey, J., Reinig, K., Cannon, W.: Virtual reality in orthopaedics: is it a reality? Clin. Orthop. Relat. Res. 468, 2586–2591 (2010). doi: 10.1007/s11999-010-1426-1 CrossRefGoogle Scholar
  4. 4.
    Ackerman, M.J.: The visible human project: a resource for anatomical visualization. In: Proc. of the IEEE Int. Conf. Inf. Technol. Appl. in Biomedicine, 1977. ITAB ’97, pp. 29–31 (1997) Google Scholar
  5. 5.
    Heng, P.A., Chun-Yiu, C., Tien-Tsin, W., Yangsheng, X., Yim-Pan, C., Kai-Ming, C., Shiu-Kit, T.: A virtual-reality training system for knee arthroscopic surgery. IEEE Trans. Inf. Technol. Biomed. 8, 217–227 (2002) CrossRefGoogle Scholar
  6. 6.
    Megali, G., Tonet, O., Mazzoni, M., Dario, P., Vascellari, A., Marcacci, M.: A new tool for surgical training in knee arthroscopy. In: Proc. of Med. Image Comput. and Comput.-Assist. Interv.—MICCAI 2002. Lecture Notes in Computer Science, vol. 2489, pp. 170–177. Springer, Berlin (2002) CrossRefGoogle Scholar
  7. 7.
    Bayona, S., Espadero, J., Pastor, L., Fernandez-Arroyo, J.M.: A low-cost arthroscopy surgery training system. In: Proc. of the IASTED Int. Conf. on Vis., Imaging and Image Processing VIIP, Bemnalmadena, Spain, September 2003 Google Scholar
  8. 8.
    Muller, W., Bockholt, U.: The virtual reality arthroscopy training simulator. In: Proc. of Med. Meets Virtual Reality (MMVR)-Art, Science, Technology: Healthcare (1998) Google Scholar
  9. 9.
    Gibson, S., Samosky, J., Mor, A.: Simulating arthroscopic knee surgery using volumetric object representations, real-time volume rendering and haptic feedback. In: Proc. of CVRMed-MRCAS, pp. 369–378 (1997) Google Scholar
  10. 10.
    Mabrey, J.D., Gillogly, S.D., Kasser, J.R., Sweeney, H.J., Zarins, B., Mevis, H., Garrett, W.E. Jr., Poss, R., Cannon, W.D.: Virtual reality simulation of arthroscopy of the knee. Arthroscopy 18 (2002) Google Scholar
  11. 11.
    Sherman, K.P., Ward, J.W., Wills, D.P.M., Mohsen, A.M.M.A.: A portable virtual environment knee arthroscopy training system with objective scoring. In: Proc. of Med. Meets Virtual Reality (MMVR)—The Convergence of Phys. & Inf. Technol.: Options for a New Era in Healthcare, pp. 335–336 (1999) Google Scholar
  12. 12.
    (GMV Innovative Solutions). insightArthroVR: virtual reality arthroscopy trainer, 2010. Accessed: 12 May 2010
  13. 13.
    (Simbionix Corporation). Arthro mentor, 2012. Accessed: 16 May 2012
  14. 14.
    (VirtaMed AG). Virtamed arthroscopy simulator, 2012. Accessed: 16 May 2012
  15. 15.
    Rasool, S., Sourin, A.: Tangible images. In: SIGGRAPH Asia 2011 Sketches, SA ’11, pp. 41:1–41:2. ACM, New York (2011) Google Scholar
  16. 16.
    Rasool, S., Sourin, A.: Haptic interaction with 2d images. In: Proc. of the 10th Int. Conf. on Virtual Reality Continuum and Its Appl. in Industry, VRCAI ’11, pp. 13–22. ACM, New York (2011) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  1. 1.Nanyang Technological UniversitySingaporeSingapore

Personalised recommendations