Abstract
In this work, several robust vision modules are developed and implemented for fully automated micromanipulation. These are autofocusing, object and end-effector detection, real-time tracking and optical system calibration modules. An image based visual servoing architecture and a path planning algorithm are also proposed based on the developed vision modules. Experimental results are provided to assess the performance of the proposed visual servoing approach in positioning and trajectory tracking tasks. Proposed path planning algorithm in conjunction with visual servoing imply successful micromanipulation tasks.
Similar content being viewed by others
References
Boissenin M., Wedekind J., Selvan A.N., Amavasai B.P., Caparrelli F., Travis J.R.: Computer vision methods for optical microscopes. Image Vis Comput 25(7), 1107–1116 (2006)
Amavasai B.P., Caparrelli F., Selvan A., Boissenin M., Travis J.R., Meikle S.: Machine vision methods for autonomous micro-robotic systems. Kybernetes Int J Syst Cybern 34(9–10), 1421–1439 (2005)
Kim, B., Kang, H., Kim, D.H., Park, G.T., Park, J.O.: Flexible microassembly system based on hybrid manipulation scheme. In: IEEE/RSJ Int. Conf. on intelligent robots and systems, Nevada, pp. 2061–2066 (2003)
Enikov E.T., Minkov L.L., Clark S.: Microassembly experiments with transparent electrostatic gripper under optical and vision-based control. IEEE Trans. Ind. Electron. 52(4), 1005–1012 (2005)
Yang G., Gaines J.A., Nelson B.J.: A flexible experimental workcell for efficient and reliable wafer-level 3D micro-assembly. IEEE Int. Conf. Robot. Automat 1, 133–138 (2001)
Yang G., Gaines J.A., Nelson B.J.: Optomechatronic design of microassembly systems for manufacturing hybrid microsystems. IEEE Trans. Ind. Electron. 52(4), 1013–1023 (2005)
Nelson, B.J., Zhou, Y., Vikramaditya, B.: Sensor-based microassembly of hybrid MEMS device. In: IEEE Control Systems, pp. 35–45 (1998)
Beyeler, F., Bell, D.J., Nelson, B.J., Sun, Y., Neild, A., Oberti, S., Dual, J.: Design of a micro-gripper and an ultrasonic manipulator for handling micron sized objects. In: Int. Conf. on Intelligent Robots and Systems, Beijing, China, pp. 772–777 (2006)
Dechev N., Cleghorn W.L., Mills J.K.: Microassembly of 3-D MEMS structures utilizing a MEMS microgripper with a robotic manipulator. IEEE Int. Conf. Robot. Automat. 3, 3193–3199 (2003)
Popa, D.O., Kang, B.J., Sin, J., Zou, J.: Reconfigurable micro-assembly system for photonics applications. In: IEEE Int. Conf. on Robotics and Automation, pp. 1495–1500, Washington (2006)
Sun Y., Nelson B.J.: Microrobotic cell injection. Int. Conf. Robot. Automat. 1, 620–625 (2001)
Sun Y.W., Kenneth R.P., Bischof J.C., Nelson B.J.: Mechanical property characterization of mouse zona pellucida. IEEE Trans. Nanobiosci. 2(4), 279–286 (2003)
Li, X., Zong, G., Bi, S.: Development of global vision system for biological automatic micromanipulation system. In: Int. Conf. on Robotics and Automation, Seoul, pp. 127–132 (2001)
Kim D.H., Hwang C.N., Sun Y., Lee S.H., Kim B., Nelson B.J.: Mechanical analysis of chorion softening in prehatching stages of zebrafish embryos. IEEE Trans. Nanobiosci. 5(2), 89–94 (2006)
Wang, W.H., Liu, X.Y., Sun, Y.: Autonomous Zebrafish Embryo injection using a microrobotic system. In: Int. Conf. on Automation Science and Engineering, pp. 363–368 (2007)
Kasaya, T., Miyazaki, H., Saito, S., Sato, T.: Micro object handling under SEM by vision-based automatic control. In: IEEE Int. Conf. Robotics and Automation, Detroit, pp. 2189–2196 (1999)
Sitti M.: Controlled pushing of nanoparticles: modeling and experiments. IEEE/ASME Trans. Mechatron. 5(2), 199–221 (2000)
Krotkov E., Martin J.P.: Range from Focus. IEEE Int. Conf. Robot. Automat. 3, 1093–1098 (1986)
Sun, Y., Duthaler, S., Nelson, B.J.: Autofocusing algorithm selection in computer microscopy. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pp. 70–76 (2005)
Xiong, Y., Schafer, S.A.: Depth from Focusing and Defocusing. In: IEEE Computer Society Conf. on computer vision and pattern recognition, pp. 68–73 (1993)
Ma Y., Soatto S., Kosecka J., Sastry S.S.: An Invitation to 3-D Vision, from Images to Geometric Models. Springer, Berlin (2006)
Forsyth D.A., Ponce J.: Computer Vision, a Modern Approach. Prentice-Hall, Englewood Cliffs (2003)
Bilen, H., Hocaoglu, M., Unel, M., Gozuacik, D.: Novel parameter estimation schemes in microsystems. In: IEEE Int. Conf. on Robotics and Automation, pp. 2394–2399 (2009)
Tsai R.Y.: A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Automat. 3, 323–344 (1987)
Zhang, Z.Y.: Flexible camera calibration by viewing a plane from unknown orientations. In: IEEE Int. Conf. on Computer Vision, pp. 666–673 (1999)
Zhou Y., Nelson B.J.: Calibration of a parametric model of an optical microscope. Optical Eng. 38, 1989–1995 (1999)
Ammi, M., Fremont, V., Ferreira, A.: Flexible microscope calibration using virtual pattern for 3-D telemicromanipulation. In: IEEE Trans. on Robotics and Automation, pp. 3888–3893 (2005)
Chaumette F., Hutchinson S.: Visual servo control I. Basic approaches. IEEE Robot Automat Mag. 13(4), 82–90 (2006)
Wang J., Cho H.: Micropeg and hole alignment using image moments based visual servoing method. IEEE Trans. Ind. Electron. 55(3), 1286–1294 (2008)
Motai Y., Kosaka A.: Hand-eye calibration applied to viewpoint selection for robotic vision. IEEE Trans. Ind. Electron. 55(10), 3731–3741 (2008)
Choi K.J., Lee Y.H., Moon J.W., Park C.K, Harashima F.: Development of an automatic stencil inspection system using modified hough transform and fuzzy logic. IEEE Trans. Ind. Electron. 54(1), 604–611 (2007)
Malis E.: Visual servoing invariant to changes in camera intrinsic parameters. IEEE Trans Robot Automat 20, 72–81 (2004)
Gangloff, J., Mathelin, M., Abba, G.: 6 DOF high speed dynamic visual servoing using GPC controllers. In: IEEE Int. Conf. on Robotics and Automation, pp. 2008–2013 (1998)
Nelson B.J., Papanikolopoulos N.P., Khosla P.K.: Robotic visual servoing and robotic assembly tasks. IEEE Robot. Automat. Mag. 3(2), 23–31 (1996)
Hashimoto, K., Ebine, T., Sakamoto, K., Kimura, H.: Full 3D visual tracking with nonlinear model-based control. In: American Control Conference, San Francisco, pp. 3180–3184 (1993)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bilen, H., Hocaoglu, M.A., Unel, M. et al. Developing robust vision modules for microsystems applications. Machine Vision and Applications 23, 25–42 (2012). https://doi.org/10.1007/s00138-010-0267-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-010-0267-y