Fusion of Inertial Measurements and Vision Feedback for Microsurgery
A microsurgery system that achieves real-time enhanced micrometer scale positioning accuracy by fusing visual information from a high speed monovision camera mounted on an optical surgical microscope and acceleration measurements from an intelligent handheld instrument, ITrem2, is presented. The high speed camera captures images of the tool tip of ITrem2 to track its position in real-time. The focus value of the tool tip in the acquired image is used to locate the tool tip along the principal axis of the objective lens of the microscope and edge based geometric template matching gives the position in pixel coordinates. ITrem2 utilizes four dual-axis miniature digital MEMS accelerometers to sense and update the motion information. The system has a first in, first out (FIFO) queue to track the recent history of the slow non-drifting position estimation from the vision system and acceleration readings from the inertial sensors together with their respective time stamps. In the proposed method, real-time visual servoing of micrometer scale motion is achieved by taking into account the dynamic behavior of the vision feedback and incorporating synchronized fusion of these complementary sensors.
Keywordssensor fusion visual servoing microsurgery
Unable to display preview. Download preview PDF.
- 1.Rehbinder, H., Ghosh, B.K.: Multi-rate fusion of visual and inertial data. In: International Conference on Multisensor Fusion and Integration for Intelligent Systems, MFI 2001, pp. 97–102 (2001)Google Scholar
- 2.Armesto, L., Chroust, S., Vincze, M., Tornero, J.: Multi-rate fusion with vision and inertial sensors. In: Robotics and Automation, Proceedings of ICRA 2004, April 26 - May 1, vol. 1, pp. 193–199 (2004)Google Scholar
- 4.Huster, A., Frew, E.W., Rock, S.M.: Relative position estimation for AUVs by fusing bearing and inertial rate sensor measurements. In: OCEANS 2002 MTS/IEEE, October 29-31, vol. 3, pp. 1863–1870 (2002)Google Scholar
- 9.Karras, G.C., Loizou, S.G., Kyriakopoulos, K.J.: A visual-servoing scheme for semi-autonomous operation of an underwater robotic vehicle using an IMU and a Laser Vision System. In: Robotics and Automation, ICRA 2010, May 3-7, pp. 5262–5267 (2010)Google Scholar
- 10.Zhou, Y., Nelson, B.J., Vikramaditya, B.: Fusing force and vision feedback for micromanipulation. In: Proceedings of 1998 IEEE International Conference on Robotics and Automation, May 16-20, vol. 2, pp. 1220–1225 (1998)Google Scholar
- 11.Yu Sun, S., Duthaler, S., Nelson, B.J.: Autofocusing Algorithm Selection in Computer Microscopy. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, August 2-6, pp. 70–76 (2005)Google Scholar
- 12.Wu, Q., Merchant, F.A., Castleman, K.R.: Microscope Image Processing. Academic Press (2008)Google Scholar