Human-assisted virtual reality for a magnetic-haptic micromanipulation platform

Abstract

This paper deals with the development of a virtual reality interface (VRI) for a magnetic-haptic micromanipulation platform (MHMP) (Mehrtash et al. in IEEE/ASME Trans Mechatron 16(3):459–469, 2011). Our previously developed MHMP has shown a great deal of promise in non-contact micromanipulations. This micromanipulation platform concerns the integration of magnetic actuation technology and a bilateral macro–micro teleoperation. The MHMP has two separate stations: one magnetic microrobotic station and one haptic. The magnetic microrobotic station manipulates micro-sized objects based on the commands from the haptic station. The haptic station uses bilateral communication with the magnetic microrobotic station to allow a human operator the feeling of a micro-domain environment. In this paper, we report a VRI that enables human operators to improve their skills in using the MHMP, before carrying out an actual dexterous task. The VRI is made up of three main components: a haptic station, a simulation engine, and a display unit. The haptic station provides the operator with the force/torque information from virtual or remote environments, and is also used to recognize the operator’s hand motion command. Dynamical computation and control system modeling have been carried out on the simulation engine. Based on the real-time computation, this engine, as the heart of the system, provides force applied to the operator’s hand and the microrobot’s position for the haptic station and the display unit, respectively. The display unit employs 3D computer graphics to demonstrate the micromanipulation tasks and environments. The VRI is also developed in such a way that it can be separately used in parallel with the MHMP for the 3D visualization of a real task by providing multiple virtual viewports. This paper introduces the configuration of the proposed VRI, and reports the result of a preliminary experiment using micromanipulation investigation for validation.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. Ammi M, Ferreira A (2004) Path planning of an AFM-based nanomanipulator using virtual force reflection. In: Proceedings of the international conference on intelligent robots and systems, 2004 (IROS 2004). 2004 IEEE/RSJ, vol 1, pp 577–582

  2. Burdea GC (1999) Invited review: the synergy between virtual reality and robotics. IEEE Trans Robot Autom 15(3):400–410

    Article  Google Scholar 

  3. CPOL (2009) Solving ordinary differential equations in C++. http://www.codeproject.com/KB/recipes/odeint.aspx

  4. Earnshaw S (1842) On the nature of the molecular forces which regulate the constitution of the luminiferous ether. Trans Camb Philos Soc 7:97–112

    Google Scholar 

  5. Ferreira A, Cassier C, Hirai S (2004) Automatic microassembly system assisted by vision servoing and virtual reality. IEEE/ASME Trans Mechatron 9(2):321–333

    Article  Google Scholar 

  6. Hamdi M, Ferreira A (2004) Microassembly planning using physical-based models in virtual environment. In: Proceedings of the international conference on intelligent robots and systems, 2004 (IROS 2004). 2004 IEEE/RSJ, vol 4, pp 3369–3374

  7. Hosseini S, Mehrtash M, Khamesee MB (2011) Design, fabrication and control of a magnetic capsule-robot for the human esophagus. Microsyst Technol 17:1145–1152

    Article  Google Scholar 

  8. Khamesee MB, Kato N, Nomura Y, Nakamura T (2002) Design and control of a microrobotic system using magnetic levitation. IEEE/ASME Trans Mechatron 7(1):1–14

    Article  Google Scholar 

  9. Mehrtash M, Khamesee MB (2011) Design and implementation of lqg\ltr controller for a magnetic telemanipulation system-performance evaluation and energy saving. Microsyst Technol 17:1135–1143

    Article  Google Scholar 

  10. Mehrtash M, Shameli E, Khamesee MB (2010) Magnetic telemanipulation device with mass uncertainty: Modeling, simulation and testing. Int J Appl Electromagn Mech 34(4):211–223

    Google Scholar 

  11. Mehrtash M, Tsuda N, Khamesee MB (2011) Bilateral macro/micro teleoperation using magnetic levitation. IEEE/ASME Trans Mechatron 16(3):459–469

    Article  Google Scholar 

  12. Probst M, Hurzeler C, Borer R, Nelson B (2007) Virtual reality for microassembly. In: Society of photo-optical instrumentation engineers (SPIE) conference series, vol 6718

  13. Stratton JA (2007) Electromagnetic theory. Wiley, Hoboken

    Google Scholar 

  14. Wernecke J (1994) The inventor mentor: programming object-oriented 3D graphics with open inventor, release 2. Addison-Wesley, Boston

  15. Zhou Q, Kallio P, Koivo HN (2000) Virtual environment for operations in the microworld. In: Nelson BJ, Brequet J-M (eds) Microrobotics and microassembly II, SPIE, USA, pp 56–64

Download references

Acknowledgments

The authors gratefully acknowledge the Canada Foundation for Innovation (CFI) and the Natural Science and Engineering Research Council of Canada (NSERC) for financial support.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Mir Behrad Khamesee.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Mehrtash, M., Khamesee, M.B., Tarao, S. et al. Human-assisted virtual reality for a magnetic-haptic micromanipulation platform. Microsyst Technol 18, 1407–1415 (2012). https://doi.org/10.1007/s00542-012-1560-7

Download citation

Keywords

  • Permanent Magnet
  • External Magnetic Field
  • Magnetic Force
  • Haptic Device
  • Simulation Engine