Skip to main content
Log in

Developing robust vision modules for microsystems applications

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In this work, several robust vision modules are developed and implemented for fully automated micromanipulation. These are autofocusing, object and end-effector detection, real-time tracking and optical system calibration modules. An image based visual servoing architecture and a path planning algorithm are also proposed based on the developed vision modules. Experimental results are provided to assess the performance of the proposed visual servoing approach in positioning and trajectory tracking tasks. Proposed path planning algorithm in conjunction with visual servoing imply successful micromanipulation tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Boissenin M., Wedekind J., Selvan A.N., Amavasai B.P., Caparrelli F., Travis J.R.: Computer vision methods for optical microscopes. Image Vis Comput 25(7), 1107–1116 (2006)

    Article  Google Scholar 

  2. Amavasai B.P., Caparrelli F., Selvan A., Boissenin M., Travis J.R., Meikle S.: Machine vision methods for autonomous micro-robotic systems. Kybernetes Int J Syst Cybern 34(9–10), 1421–1439 (2005)

    Article  Google Scholar 

  3. Kim, B., Kang, H., Kim, D.H., Park, G.T., Park, J.O.: Flexible microassembly system based on hybrid manipulation scheme. In: IEEE/RSJ Int. Conf. on intelligent robots and systems, Nevada, pp. 2061–2066 (2003)

  4. Enikov E.T., Minkov L.L., Clark S.: Microassembly experiments with transparent electrostatic gripper under optical and vision-based control. IEEE Trans. Ind. Electron. 52(4), 1005–1012 (2005)

    Article  Google Scholar 

  5. Yang G., Gaines J.A., Nelson B.J.: A flexible experimental workcell for efficient and reliable wafer-level 3D micro-assembly. IEEE Int. Conf. Robot. Automat 1, 133–138 (2001)

    Google Scholar 

  6. Yang G., Gaines J.A., Nelson B.J.: Optomechatronic design of microassembly systems for manufacturing hybrid microsystems. IEEE Trans. Ind. Electron. 52(4), 1013–1023 (2005)

    Article  Google Scholar 

  7. Nelson, B.J., Zhou, Y., Vikramaditya, B.: Sensor-based microassembly of hybrid MEMS device. In: IEEE Control Systems, pp. 35–45 (1998)

  8. Beyeler, F., Bell, D.J., Nelson, B.J., Sun, Y., Neild, A., Oberti, S., Dual, J.: Design of a micro-gripper and an ultrasonic manipulator for handling micron sized objects. In: Int. Conf. on Intelligent Robots and Systems, Beijing, China, pp. 772–777 (2006)

  9. Dechev N., Cleghorn W.L., Mills J.K.: Microassembly of 3-D MEMS structures utilizing a MEMS microgripper with a robotic manipulator. IEEE Int. Conf. Robot. Automat. 3, 3193–3199 (2003)

    Google Scholar 

  10. Popa, D.O., Kang, B.J., Sin, J., Zou, J.: Reconfigurable micro-assembly system for photonics applications. In: IEEE Int. Conf. on Robotics and Automation, pp. 1495–1500, Washington (2006)

  11. Sun Y., Nelson B.J.: Microrobotic cell injection. Int. Conf. Robot. Automat. 1, 620–625 (2001)

    Google Scholar 

  12. Sun Y.W., Kenneth R.P., Bischof J.C., Nelson B.J.: Mechanical property characterization of mouse zona pellucida. IEEE Trans. Nanobiosci. 2(4), 279–286 (2003)

    Article  Google Scholar 

  13. Li, X., Zong, G., Bi, S.: Development of global vision system for biological automatic micromanipulation system. In: Int. Conf. on Robotics and Automation, Seoul, pp. 127–132 (2001)

  14. Kim D.H., Hwang C.N., Sun Y., Lee S.H., Kim B., Nelson B.J.: Mechanical analysis of chorion softening in prehatching stages of zebrafish embryos. IEEE Trans. Nanobiosci. 5(2), 89–94 (2006)

    Article  Google Scholar 

  15. Wang, W.H., Liu, X.Y., Sun, Y.: Autonomous Zebrafish Embryo injection using a microrobotic system. In: Int. Conf. on Automation Science and Engineering, pp. 363–368 (2007)

  16. Kasaya, T., Miyazaki, H., Saito, S., Sato, T.: Micro object handling under SEM by vision-based automatic control. In: IEEE Int. Conf. Robotics and Automation, Detroit, pp. 2189–2196 (1999)

  17. Sitti M.: Controlled pushing of nanoparticles: modeling and experiments. IEEE/ASME Trans. Mechatron. 5(2), 199–221 (2000)

    Article  Google Scholar 

  18. Krotkov E., Martin J.P.: Range from Focus. IEEE Int. Conf. Robot. Automat. 3, 1093–1098 (1986)

    Google Scholar 

  19. Sun, Y., Duthaler, S., Nelson, B.J.: Autofocusing algorithm selection in computer microscopy. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pp. 70–76 (2005)

  20. Xiong, Y., Schafer, S.A.: Depth from Focusing and Defocusing. In: IEEE Computer Society Conf. on computer vision and pattern recognition, pp. 68–73 (1993)

  21. Ma Y., Soatto S., Kosecka J., Sastry S.S.: An Invitation to 3-D Vision, from Images to Geometric Models. Springer, Berlin (2006)

    Google Scholar 

  22. Forsyth D.A., Ponce J.: Computer Vision, a Modern Approach. Prentice-Hall, Englewood Cliffs (2003)

    Google Scholar 

  23. Bilen, H., Hocaoglu, M., Unel, M., Gozuacik, D.: Novel parameter estimation schemes in microsystems. In: IEEE Int. Conf. on Robotics and Automation, pp. 2394–2399 (2009)

  24. Tsai R.Y.: A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Automat. 3, 323–344 (1987)

    Article  Google Scholar 

  25. Zhang, Z.Y.: Flexible camera calibration by viewing a plane from unknown orientations. In: IEEE Int. Conf. on Computer Vision, pp. 666–673 (1999)

  26. Zhou Y., Nelson B.J.: Calibration of a parametric model of an optical microscope. Optical Eng. 38, 1989–1995 (1999)

    Article  Google Scholar 

  27. Ammi, M., Fremont, V., Ferreira, A.: Flexible microscope calibration using virtual pattern for 3-D telemicromanipulation. In: IEEE Trans. on Robotics and Automation, pp. 3888–3893 (2005)

  28. Chaumette F., Hutchinson S.: Visual servo control I. Basic approaches. IEEE Robot Automat Mag. 13(4), 82–90 (2006)

    Article  Google Scholar 

  29. Wang J., Cho H.: Micropeg and hole alignment using image moments based visual servoing method. IEEE Trans. Ind. Electron. 55(3), 1286–1294 (2008)

    Article  Google Scholar 

  30. Motai Y., Kosaka A.: Hand-eye calibration applied to viewpoint selection for robotic vision. IEEE Trans. Ind. Electron. 55(10), 3731–3741 (2008)

    Article  Google Scholar 

  31. Choi K.J., Lee Y.H., Moon J.W., Park C.K, Harashima F.: Development of an automatic stencil inspection system using modified hough transform and fuzzy logic. IEEE Trans. Ind. Electron. 54(1), 604–611 (2007)

    Article  Google Scholar 

  32. Malis E.: Visual servoing invariant to changes in camera intrinsic parameters. IEEE Trans Robot Automat 20, 72–81 (2004)

    Article  Google Scholar 

  33. Gangloff, J., Mathelin, M., Abba, G.: 6 DOF high speed dynamic visual servoing using GPC controllers. In: IEEE Int. Conf. on Robotics and Automation, pp. 2008–2013 (1998)

  34. Nelson B.J., Papanikolopoulos N.P., Khosla P.K.: Robotic visual servoing and robotic assembly tasks. IEEE Robot. Automat. Mag. 3(2), 23–31 (1996)

    Article  Google Scholar 

  35. Hashimoto, K., Ebine, T., Sakamoto, K., Kimura, H.: Full 3D visual tracking with nonlinear model-based control. In: American Control Conference, San Francisco, pp. 3180–3184 (1993)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mustafa Unel.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bilen, H., Hocaoglu, M.A., Unel, M. et al. Developing robust vision modules for microsystems applications. Machine Vision and Applications 23, 25–42 (2012). https://doi.org/10.1007/s00138-010-0267-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-010-0267-y

Keywords

Navigation