Horizon Stabilized—Dynamic View Expansion for Robotic Assisted Surgery (HS-DVE)

  • Alexander Warren
  • Peter Mountney
  • David Noonan
  • Guang-Zhong Yang
Original Article



New surgical approaches based on natural orifice transluminal surgery (NOTES) have the potential to further decrease morbidity and hospital stay. However, a number of key challenges have been identified preventing its clinical adoption, including inadequate instrument design and spatial disorientation. Furthermore, retroflexion, missing fixed anatomical references, and limited field-of-view are key factors contributing to disorientation in NOTES.


A hybrid approach of integrated orientation sensing and real-time vision processing is proposed to restore orientation cues for improved surgical navigation. The distal tip of an articulated robotic endoscope is equipped with an inertial measurement unit (IMU) enabling video images to be reoriented and stabilized with respect to the horizon. This is performed by measuring the direction of gravity in relation to the cameras. Dynamic view expansion is used to increase the field-of-view of the endoscope. The method registers past video images to the current image and creates an enlarged visualization of the anatomy through simultaneous localization and mapping (SLAM).


The clinical potential of the system is demonstrated on a NOTES appendectomy procedure performed on the NOSsE phantom. This involves an articulated robotic endoscope navigating to visualize the appendix while retroflexed. The horizon stabilization is additionally evaluated quantitatively against known ground truth.


The combination of horizon stabilization and dynamic view expansion presents a realistic approach for reintroducing orientation and navigation cues during NOTES. The platform allows real-time implementation, which is an important prerequisite for further clinical evaluation.


Simultaneous localization and mapping SLAM IMU Horizon stabilization NOTES Disorientation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Rattner D, Kalloo A (2006) Asge/sages working group on natural orifice translumenal endoscopic surgery. Surg Endosc 20: 329–333. doi: 10.1007/s00464-005-3006-0 PubMedCrossRefGoogle Scholar
  2. 2.
    Noonan D, Payne C, Shang J, Sauvage V, Newton R, Elson D, Darzi A, Yang G-Z (2010) Force adaptive multi-spectral imaging with an articulated robotic endoscope. In: Medical image computing and computer-assisted intervention (MICCAI), vol 6363 of Lecture notes in computer science. Springer, Berlin, pp 245–252Google Scholar
  3. 3.
    Koppel D, Wang Y-F, Lee H (2005) Robust and real-time image stabilization and rectification. In: Proceedings of 7th IEEE Workshop on application of computer vision (WACV/MOTION 2005), vol 1, pp 350–355Google Scholar
  4. 4.
    Moll M, Koninckx T, Gool LJ, Koninckx PR (2009) Unrotating images in laparoscopy with an application for 30° laparoscopes. In: Magjarevic R, Sloten J, Verdonck P, Nyssen M, Haueisen J (eds) 4th European conference of the international federation for medical and biological engineering, vol 22 of IFMBE proceedings. Springer, Berlin, pp 966–969Google Scholar
  5. 5.
    Joshi N, Kang SB, Zitnick CL, Szeliski R (2010) Image deblurring using inertial measurement sensors. ACM Trans Graph 29(30): 1–30Google Scholar
  6. 6.
    Höller K, Penne J, Schneider A, Jahn J, Guttiérrez Boronat J, Wittenberg T, Feußner H, Hornegger J (2009) Endoscopic orientation correction. In: Medical image computing and computer-assisted intervention (MICCAI 2009), vol 5761 of Lecture notes in computer dcience. Springer, Berlin, pp 459–466Google Scholar
  7. 7.
    Lerotic M, Chung A, Clark J, Valibeik S, Yang G-Z (2008) Dynamic view expansion for enhanced navigation in natural orifice transluminal endoscopic surgery. In: Metaxas D, Axel L, Fichtinger G, Székely G (eds) Medical image computing and computer-assisted intervention (MICCAI 2008), vol 5242 of Lecture notes in computer science. Springer, Berlin, pp 467–475Google Scholar
  8. 8.
    Mountney P, Yang G-Z (2009) Dynamic view expansion for minimally invasive surgery using simultaneous localization and mapping. In: Engineering in medicine and biology society, 2009. EMBC 2009. Annual international conference of the IEEE. pp 1184 –1187Google Scholar
  9. 9.
    Mountney P, Stoyanov D, Yang G-Z (2010) Three-dimensional tissue deformation recovery and tracking. Signal Process Mag IEEE 27(4): 14–24CrossRefGoogle Scholar
  10. 10.
    Miranda-Luna R, Daul C, Blondel W, Hernandez-Mier Y, Wolf D, Guillemin F (2008) Mosaicing of bladder endoscopic image sequences: distortion calibration and registration algorithm. IEEE Trans Biomed Eng 55(2): 541–553PubMedCrossRefGoogle Scholar
  11. 11.
    Seibel E, Carroll R, Dominitz J, Johnston R, Melville C, Lee C, Seitz S, Kimmey M (2008) Tethered capsule endoscopy, a low-cost and high-performance alternative technology for the screening of esophageal cancer and barrett’s esophagus. IEEE Trans Biomed Eng 55(3): 1032–1042PubMedCrossRefGoogle Scholar
  12. 12.
    Igarashi T, Suzuki H, Naya Y (2009) Computer-based endoscopic image-processing technology for endourology and laparoscopic surgery. Int J Urol 16(6): 533–543PubMedCrossRefGoogle Scholar
  13. 13.
    Mountney P, Stoyanov D, Davison A, Yang G-Z (2006) Simultaneous stereoscope localization and soft-tissue mapping for minimal invasive surgery. In: Larsen R, Nielsen M, Sporring J (eds) Medical image computing and computer-assisted intervention (MICCAI), vol. 4190 of Lecture notes in computer science. Springer, Berlin, pp 347–354Google Scholar
  14. 14.
    Stoyanov D (2007) Camera calibration tools— Feb 2007
  15. 15.
    Shi J, Tomasi C (1994) Good features to track. In: IEEE conference on computer vision and pattern recognition (CVPR’94). pp 593–600Google Scholar
  16. 16.
    Clark J, Sodergren M, Noonan D, Darzi A, Yang G-Z (2009) The natural orifice simulated surgical environment (nosse): exploring the challenges of notes without the animal model. J Laparoendosc Adv Surg Tech 19: 211–214CrossRefGoogle Scholar

Copyright information

© CARS 2011

Authors and Affiliations

  • Alexander Warren
    • 1
  • Peter Mountney
    • 1
  • David Noonan
    • 1
  • Guang-Zhong Yang
    • 1
  1. 1.The Hamlyn Centre for Robotic SurgeryImperial College LondonLondonUK

Personalised recommendations