Machine Vision and Applications

, Volume 23, Issue 1, pp 25–42 | Cite as

Developing robust vision modules for microsystems applications

  • Hakan Bilen
  • Muhammet A. Hocaoglu
  • Mustafa Unel
  • Asif Sabanovic
Original Paper


In this work, several robust vision modules are developed and implemented for fully automated micromanipulation. These are autofocusing, object and end-effector detection, real-time tracking and optical system calibration modules. An image based visual servoing architecture and a path planning algorithm are also proposed based on the developed vision modules. Experimental results are provided to assess the performance of the proposed visual servoing approach in positioning and trajectory tracking tasks. Proposed path planning algorithm in conjunction with visual servoing imply successful micromanipulation tasks.


Microsystems Micromanipulation Visual feedback Robust detection Normalized cross correlation Tracking Visual servoing Path planning 


  1. 1.
    Boissenin M., Wedekind J., Selvan A.N., Amavasai B.P., Caparrelli F., Travis J.R.: Computer vision methods for optical microscopes. Image Vis Comput 25(7), 1107–1116 (2006)CrossRefGoogle Scholar
  2. 2.
    Amavasai B.P., Caparrelli F., Selvan A., Boissenin M., Travis J.R., Meikle S.: Machine vision methods for autonomous micro-robotic systems. Kybernetes Int J Syst Cybern 34(9–10), 1421–1439 (2005)CrossRefGoogle Scholar
  3. 3.
    Kim, B., Kang, H., Kim, D.H., Park, G.T., Park, J.O.: Flexible microassembly system based on hybrid manipulation scheme. In: IEEE/RSJ Int. Conf. on intelligent robots and systems, Nevada, pp. 2061–2066 (2003)Google Scholar
  4. 4.
    Enikov E.T., Minkov L.L., Clark S.: Microassembly experiments with transparent electrostatic gripper under optical and vision-based control. IEEE Trans. Ind. Electron. 52(4), 1005–1012 (2005)CrossRefGoogle Scholar
  5. 5.
    Yang G., Gaines J.A., Nelson B.J.: A flexible experimental workcell for efficient and reliable wafer-level 3D micro-assembly. IEEE Int. Conf. Robot. Automat 1, 133–138 (2001)Google Scholar
  6. 6.
    Yang G., Gaines J.A., Nelson B.J.: Optomechatronic design of microassembly systems for manufacturing hybrid microsystems. IEEE Trans. Ind. Electron. 52(4), 1013–1023 (2005)CrossRefGoogle Scholar
  7. 7.
    Nelson, B.J., Zhou, Y., Vikramaditya, B.: Sensor-based microassembly of hybrid MEMS device. In: IEEE Control Systems, pp. 35–45 (1998)Google Scholar
  8. 8.
    Beyeler, F., Bell, D.J., Nelson, B.J., Sun, Y., Neild, A., Oberti, S., Dual, J.: Design of a micro-gripper and an ultrasonic manipulator for handling micron sized objects. In: Int. Conf. on Intelligent Robots and Systems, Beijing, China, pp. 772–777 (2006)Google Scholar
  9. 9.
    Dechev N., Cleghorn W.L., Mills J.K.: Microassembly of 3-D MEMS structures utilizing a MEMS microgripper with a robotic manipulator. IEEE Int. Conf. Robot. Automat. 3, 3193–3199 (2003)Google Scholar
  10. 10.
    Popa, D.O., Kang, B.J., Sin, J., Zou, J.: Reconfigurable micro-assembly system for photonics applications. In: IEEE Int. Conf. on Robotics and Automation, pp. 1495–1500, Washington (2006)Google Scholar
  11. 11.
    Sun Y., Nelson B.J.: Microrobotic cell injection. Int. Conf. Robot. Automat. 1, 620–625 (2001)Google Scholar
  12. 12.
    Sun Y.W., Kenneth R.P., Bischof J.C., Nelson B.J.: Mechanical property characterization of mouse zona pellucida. IEEE Trans. Nanobiosci. 2(4), 279–286 (2003)CrossRefGoogle Scholar
  13. 13.
    Li, X., Zong, G., Bi, S.: Development of global vision system for biological automatic micromanipulation system. In: Int. Conf. on Robotics and Automation, Seoul, pp. 127–132 (2001)Google Scholar
  14. 14.
    Kim D.H., Hwang C.N., Sun Y., Lee S.H., Kim B., Nelson B.J.: Mechanical analysis of chorion softening in prehatching stages of zebrafish embryos. IEEE Trans. Nanobiosci. 5(2), 89–94 (2006)CrossRefGoogle Scholar
  15. 15.
    Wang, W.H., Liu, X.Y., Sun, Y.: Autonomous Zebrafish Embryo injection using a microrobotic system. In: Int. Conf. on Automation Science and Engineering, pp. 363–368 (2007)Google Scholar
  16. 16.
    Kasaya, T., Miyazaki, H., Saito, S., Sato, T.: Micro object handling under SEM by vision-based automatic control. In: IEEE Int. Conf. Robotics and Automation, Detroit, pp. 2189–2196 (1999)Google Scholar
  17. 17.
    Sitti M.: Controlled pushing of nanoparticles: modeling and experiments. IEEE/ASME Trans. Mechatron. 5(2), 199–221 (2000)CrossRefGoogle Scholar
  18. 18.
    Krotkov E., Martin J.P.: Range from Focus. IEEE Int. Conf. Robot. Automat. 3, 1093–1098 (1986)Google Scholar
  19. 19.
    Sun, Y., Duthaler, S., Nelson, B.J.: Autofocusing algorithm selection in computer microscopy. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pp. 70–76 (2005)Google Scholar
  20. 20.
    Xiong, Y., Schafer, S.A.: Depth from Focusing and Defocusing. In: IEEE Computer Society Conf. on computer vision and pattern recognition, pp. 68–73 (1993)Google Scholar
  21. 21.
    Ma Y., Soatto S., Kosecka J., Sastry S.S.: An Invitation to 3-D Vision, from Images to Geometric Models. Springer, Berlin (2006)Google Scholar
  22. 22.
    Forsyth D.A., Ponce J.: Computer Vision, a Modern Approach. Prentice-Hall, Englewood Cliffs (2003)Google Scholar
  23. 23.
    Bilen, H., Hocaoglu, M., Unel, M., Gozuacik, D.: Novel parameter estimation schemes in microsystems. In: IEEE Int. Conf. on Robotics and Automation, pp. 2394–2399 (2009)Google Scholar
  24. 24.
    Tsai R.Y.: A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Automat. 3, 323–344 (1987)CrossRefGoogle Scholar
  25. 25.
    Zhang, Z.Y.: Flexible camera calibration by viewing a plane from unknown orientations. In: IEEE Int. Conf. on Computer Vision, pp. 666–673 (1999)Google Scholar
  26. 26.
    Zhou Y., Nelson B.J.: Calibration of a parametric model of an optical microscope. Optical Eng. 38, 1989–1995 (1999)CrossRefGoogle Scholar
  27. 27.
    Ammi, M., Fremont, V., Ferreira, A.: Flexible microscope calibration using virtual pattern for 3-D telemicromanipulation. In: IEEE Trans. on Robotics and Automation, pp. 3888–3893 (2005)Google Scholar
  28. 28.
    Chaumette F., Hutchinson S.: Visual servo control I. Basic approaches. IEEE Robot Automat Mag. 13(4), 82–90 (2006)CrossRefGoogle Scholar
  29. 29.
    Wang J., Cho H.: Micropeg and hole alignment using image moments based visual servoing method. IEEE Trans. Ind. Electron. 55(3), 1286–1294 (2008)CrossRefGoogle Scholar
  30. 30.
    Motai Y., Kosaka A.: Hand-eye calibration applied to viewpoint selection for robotic vision. IEEE Trans. Ind. Electron. 55(10), 3731–3741 (2008)CrossRefGoogle Scholar
  31. 31.
    Choi K.J., Lee Y.H., Moon J.W., Park C.K, Harashima F.: Development of an automatic stencil inspection system using modified hough transform and fuzzy logic. IEEE Trans. Ind. Electron. 54(1), 604–611 (2007)CrossRefGoogle Scholar
  32. 32.
    Malis E.: Visual servoing invariant to changes in camera intrinsic parameters. IEEE Trans Robot Automat 20, 72–81 (2004)CrossRefGoogle Scholar
  33. 33.
    Gangloff, J., Mathelin, M., Abba, G.: 6 DOF high speed dynamic visual servoing using GPC controllers. In: IEEE Int. Conf. on Robotics and Automation, pp. 2008–2013 (1998)Google Scholar
  34. 34.
    Nelson B.J., Papanikolopoulos N.P., Khosla P.K.: Robotic visual servoing and robotic assembly tasks. IEEE Robot. Automat. Mag. 3(2), 23–31 (1996)CrossRefGoogle Scholar
  35. 35.
    Hashimoto, K., Ebine, T., Sakamoto, K., Kimura, H.: Full 3D visual tracking with nonlinear model-based control. In: American Control Conference, San Francisco, pp. 3180–3184 (1993)Google Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • Hakan Bilen
    • 1
  • Muhammet A. Hocaoglu
    • 1
  • Mustafa Unel
    • 1
  • Asif Sabanovic
    • 1
  1. 1.Faculty of Engineering and Natural SciencesSabanci UniversityIstanbulTurkey

Personalised recommendations