Journal of Micro-Bio Robotics

, Volume 10, Issue 1–4, pp 37–53 | Cite as

Intuitive control of self-propelled microjets with haptic feedback

  • Claudio Pacchierotti
  • Veronika Magdanz
  • Mariana Medina-Sánchez
  • Oliver G. Schmidt
  • Domenico Prattichizzo
  • Sarthak Misra
Research Paper

Abstract

Self-propelled microrobots have recently shown promising results in several scenarios at the microscale, such as targeted drug delivery and micromanipulation of cells. However, none of the steering systems available in the literature enable humans to intuitively and effectively control these microrobots in the remote environment, which is a desirable feature. In this paper we present an innovative teleoperation system with force reflection that enables a human operator to intuitively control the positioning of a self-propelled microjet. A particle-filter-based visual tracking algorithm tracks at runtime the position of the microjet in the remote environment. A 6-degrees-of-freedom haptic interface then provides the human operator with compelling haptic feedback about the interaction between the controlled microjet and the environment, as well as enabling the operator to intuitively control the target position of the microjet. Finally, a wireless magnetic control system regulates the orientation of the microjet to reach the target point. The viability of the proposed approach is demonstrated through two experimentsz enrolling twenty-eight subjects. In both experiments providing haptic feedback significantly improved the performance and the perceived realism of the considered tasks.

Keywords

Microtechnology Haptics Teleoperation Actuators Robotics 

Notes

Acknowledgments

The authors thank Alonso Sanchez for his help in setting up the tracking and control systems, and Frank van den Brink and Momen Abayazid for their help in making the video.

References

  1. 1.
    Solovev AA, Mei Y, Urena EB, Huang G, Schmidt OG (2009). Small 5(14):1688CrossRefGoogle Scholar
  2. 2.
    Khalil ISM, Magdanz V, Sanchez S, Schmidt OG, S M (2013). Applied Physics Letters 103 (17):172–404CrossRefGoogle Scholar
  3. 3.
    Woods SP, Constandinou TG (2011) Proc. international conference of the ieee engineering in medicine and biology society, pp 7372–7375Google Scholar
  4. 4.
    Sanchez S, Solovev AA, Schulze S, Schmidt OG (2011). Chem Commun 47(2):698CrossRefGoogle Scholar
  5. 5.
    Solovev AA, Sanchez S, Pumera M, Mei YF, Schmidt OG (2010). Adv Funct Mater 20(15):2430CrossRefGoogle Scholar
  6. 6.
    Zhang L, Petit T, Peyer KE, Nelson BJ (2012) Nanomedicine: Nanotechnology. Biol Med 8(7):1074Google Scholar
  7. 7.
    Balasubramanian S, Kagan D, Jack Hu CM, Campuzano S, Lobo-Castañon MJ, Lim N, Kang DY, Zimmerman M, Zhang L, Wang J (2011). Angew Chem Int Ed 50(18):4161CrossRefGoogle Scholar
  8. 8.
    Kagan D, Campuzano S, Balasubramanian S, Kuralay F, Flechsig GU, Wang J (2011). Nano Lett 11(5):2083CrossRefGoogle Scholar
  9. 9.
    Soler L, Magdanz V, Fomin VM, Sanchez SO, Schmidt OG (2013). ACS Nano 7:9611CrossRefGoogle Scholar
  10. 10.
    Xi W, Solovev AA, Ananth AN, Gracias DH, Sanchez S, Schmidt OG (2013). Nanoscale 5:1294CrossRefGoogle Scholar
  11. 11.
    Matteucci M, Casella M, Bedoni M, Donetti E, Fanetti M, De Angelis F, Gramatica F, Di Fabrizio E (2008). Microelectron Eng 85(5):1066CrossRefGoogle Scholar
  12. 12.
    Solovev AA, Xi W, Gracias DH, Harazim SM, Deneke SM, Sanchez S, Schmidt OG (2012). ACS Nano 6:1751CrossRefGoogle Scholar
  13. 13.
    Guix M, Orozco J, Garcia M, Gao W, Sattayasamitsathit S, Merkoci A, Wang J (2012). ACS Nano 6:4445CrossRefGoogle Scholar
  14. 14.
    Kuralay F, Sattayasamitsathit S, Gao W, Uygun A, Katzenberg A, Wang J (2012). J Am Chem Soc 134:15217CrossRefGoogle Scholar
  15. 15.
    Orozco J, Corts A, Cheng G, Sattayasamitsathit S, Gao W, Feng X, Shen Y, Wang J (2013). J Am Chem Soc 135:5336CrossRefGoogle Scholar
  16. 16.
    Wu Z, Wu Y, He W, Lin X, Sun J, He Q (2013). Angew Chem Int Ed 52:7000CrossRefGoogle Scholar
  17. 17.
    Nelson BJ, Kaliakatsos IK, Abbott JJ (2010). Annu Rev Biomed Eng 12:55CrossRefGoogle Scholar
  18. 18.
    Fournier-Bidoz S, Arsenault AC, Manners I, Ozin GA (2005). Chem Commun 4:441CrossRefGoogle Scholar
  19. 19.
    Nelson IK B, Abbott J (2010). Annu Rev Biomed Eng 12:55CrossRefGoogle Scholar
  20. 20.
    Paxton WF, Sundararajan S, Mallouk TE, Sen A (2006). Angew Chem Int Ed 45(33):5420CrossRefGoogle Scholar
  21. 21.
    Golestanian R, Liverpool TB, Ajdari A (2005), vol 94, pp 220–801Google Scholar
  22. 22.
    Catchmark JM, Subramanian S, Sen A (2005). Small 1(2): 202CrossRefGoogle Scholar
  23. 23.
    Sanchez S, Ananth AN, Fomin VM, Viehrig M, Schmidt OG (2011). J Am Chem Soc 133(38):14860CrossRefGoogle Scholar
  24. 24.
    Khalil ISM, Magdanz V, Sanchez S, Schmidt OG, Misra S (2014). PLoS One 9(2):e83–053CrossRefGoogle Scholar
  25. 25.
    Sanchez A, Magdanz V, Schmidt OG, Misra S (2014) Proc. 5th IEEE RAS & EMBS international conference on biomedical robotics and biomechatronicsGoogle Scholar
  26. 26.
    Khalil ISM, Magdanz V, Sanchez S, Schmidt OG, Misra S (2014). IEEE Trans Robot 30(1):49CrossRefGoogle Scholar
  27. 27.
    Troccaz J, Delnondedieu Y (1996). Mechatronics 6(4):399CrossRefGoogle Scholar
  28. 28.
    Pacchierotti C, Abayazid M, Misra S, Prattichizzo D (2014). IEEE Tran Haptics 7(4):551CrossRefGoogle Scholar
  29. 29.
    Jakopec M, Baena FR y, Harris S J, Gomes P, Cobb J, Davies BL (2003). IEEE Trans Robot Autom 19(5):902CrossRefGoogle Scholar
  30. 30.
    Hashtrudi-Zaad K, Salcudean SE (2002). IEEE Trans Robot Autom 18(1):108CrossRefGoogle Scholar
  31. 31.
    Okamura AM (2004). Ind Robot An Int J 31(6):499CrossRefGoogle Scholar
  32. 32.
    Okamura AM (2009). Curr Opin Urol 19(1)Google Scholar
  33. 33.
    Westebring-Van Der Putten EP, Goossens RHM, Jakimowicz JJ, Dankelman J (2008). Minim Invasive Ther Allied Technol 17(1):3CrossRefGoogle Scholar
  34. 34.
    van der Meijden OAJ, Schijven MP (2009). Surg Endosc 23(6):1180CrossRefGoogle Scholar
  35. 35.
    Wedmid A, Llukani E, Lee DI (2011). BJU Int 108:1028CrossRefGoogle Scholar
  36. 36.
    Prattichizzo D, Pacchierotti C, Cenci S, Minamizawa K, Rosati G (2010) Haptics: Generating and Perceiving Tangible SensationsGoogle Scholar
  37. 37.
    Prattichizzo D, Pacchierotti C, Rosati G (2012). IEEE Trans Haptics 5(4):289CrossRefGoogle Scholar
  38. 38.
    Massimino MJ, Sheridan TB (1994). Human Factors: The Journal of the Human Factors and Ergonomics Society 36(1):145Google Scholar
  39. 39.
    Pacchierotti C, Chinello F, Malvezzi M, Meli L, Prattichizzo D (2012) Haptics: perception, devices, mobility, and communicationGoogle Scholar
  40. 40.
    Wagner CR, Howe RD, Stylopoulos N (2002) Proc. international symposium on haptic interfaces for virtual environment and teleoperator systemsGoogle Scholar
  41. 41.
    Meli L, Pacchierotti C, Prattichizzo D (2014). IEEE Trans Biomed Eng 61(4):1318CrossRefGoogle Scholar
  42. 42.
    Salcudean SE, Ku S, Bell G (1997) Proc. first joint conference on computer vision, virtual reality and robotics in medicine and medial robotics and computer-assisted surgeryGoogle Scholar
  43. 43.
    Kazi A (2001). Presence: Teleoperators & Virtual Environments 10(5):495CrossRefGoogle Scholar
  44. 44.
    Moody L, Baber C, Arvanitis TN, et al. (2002). Studies in health technology and informatics 85:304Google Scholar
  45. 45.
    Kennedy CW, Hu T, Desai JP, Wechsler AS, Kresh JY (2002). Cardiovasc Eng 2(1):15CrossRefGoogle Scholar
  46. 46.
    Pillarisetti A, Pekarev M, Brooks AD, Desai JP (2007). IEEE Trans Autom Sci Eng 4(3):322CrossRefGoogle Scholar
  47. 47.
    Ando N, Korondi P, Hashimoto H (2001). IEEE/ASME Trans Mechatron 6(4):417CrossRefGoogle Scholar
  48. 48.
    Mehrtash M, Tsuda N, Khamesee MB (2011). IEEE/ASME Trans Mechatron 16(3):459CrossRefGoogle Scholar
  49. 49.
    Bolopion A, Régnier S (2013). IEEE Trans Autom Sci Eng 10(3):496CrossRefGoogle Scholar
  50. 50.
    Ghanbari A, Horan B, Nahavandi S, Chen X, Wang W (2014). IEEE Syst J 8(2):371CrossRefGoogle Scholar
  51. 51.
    van der Schaft A L2-gain and passivity techniques in nonlinear control (Springer Verlag, 2000)Google Scholar
  52. 52.
    Niemeyer G, Slotine JJE (2004). Int J Robot Res 23(9):873CrossRefGoogle Scholar
  53. 53.
    Ryu J, Kwon D, Hannaford B (2004). IEEE Trans Robot Autom 20(2):365CrossRefGoogle Scholar
  54. 54.
    Kim J, Ryu J (2010). Int J Robot Res 29(6):666CrossRefGoogle Scholar
  55. 55.
    Franken M, Stramigioli S, Misra S, Secchi C, Macchelli A (2011). IEEE Trans Robot 27(4):741CrossRefGoogle Scholar
  56. 56.
    Schoonmaker RE, Cao CG Proc. IEEE International Conference on Systems, Man and Cybernetics, vol. 3 (2006), vol. 3, 2464–2469Google Scholar
  57. 57.
    Kitagawa M, Dokko D, Okamura AM, Yuh DD (2005). J Thorac Cardiovasc Surg 129(1):151CrossRefGoogle Scholar
  58. 58.
    Ramos A, Pacchierotti C, Prattichizzo D Proc. IEEE World Haptics Conference (WHC) (2013), 473–478Google Scholar
  59. 59.
    Zilles CB (1995) Proc. IEEE/RSJ international conference of intelligent robots and systemsGoogle Scholar
  60. 60.
    Haralock RM, Shapiro LG (1991) Computer and robot vision, Addison-Wesley Longman Publishing Co., Inc.Google Scholar
  61. 61.
    Arulampalam MS, Maskell S, Gordon N, Clapp T (2002). IEEE Trans Signal Process 50(2):174CrossRefGoogle Scholar
  62. 62.
    Baker S, Matthews I (2004). Int J Comput Vis 56(3):221CrossRefGoogle Scholar
  63. 63.
    Berg B (2014) University of TwenteGoogle Scholar
  64. 64.
    Utkin VI, Chang H (2002). Math Probl Eng 8:4MathSciNetCrossRefGoogle Scholar
  65. 65.
    Bahaj AS, James PAB, Moeschler FD (1996). IEEE Trans Magn 32:5133CrossRefGoogle Scholar
  66. 66.
    Fomin VM, Hippler M, Magdanz V, Soler L, Sanchez S, Schmidt OG (2014). IEEE Trans Robot 30(1):40CrossRefGoogle Scholar
  67. 67.
    Solovev AA, Smith EJ, Bof’Bufon CC, Sanchez S, Schmidt OG (2011). Angew Chem Int Ed 50 (46):10875CrossRefGoogle Scholar
  68. 68.
    Magdanz V, Stoychev G, Ionov L, Sanchez S, Schmidt OG, et al. (2014). Angew Chem Int Ed 53 (10):2673CrossRefGoogle Scholar
  69. 69.
    Xia Y, Whitesides GM (1998). Annu Rev Mater Sci 28(1):153CrossRefGoogle Scholar
  70. 70.
    Cholewiak RW, Collins AA The psychology of touch, laurence erlbaum associates (1991), 13–60Google Scholar
  71. 71.
    Kaczmarek KA, Webster JG, Rita P.B. y., Tompkins W J (1991) IEEE Trans Biomed Eng 38(1):1Google Scholar
  72. 72.
    Diller E, Giltinan J, Sitti M (2013). Int J Robot Res 32(5): 614CrossRefGoogle Scholar
  73. 73.
    Diller E, Floyd S, Pawashe C, Sitti M (2012). IEEE Trans Robot 28(1):172CrossRefGoogle Scholar
  74. 74.
    Moody L, Baber C, Arvanitis TN, et al. (2002) Studies in health technology and informatics:304–310Google Scholar
  75. 75.
    Pacchierotti C, Prattichizzo D, Kuchenbecker KJ (2015) IEEE Trans Biomed EngGoogle Scholar
  76. 76.
    Shapiro SS, Wilk MB (1965) BiometrikaGoogle Scholar
  77. 77.
    Mauchly JW (1940). Ann Math Stat 11(2):204MathSciNetCrossRefGoogle Scholar
  78. 78.
    Gueorguieva R, Krystal JH (2004). Arch Gen Psychiatr 61(3):310CrossRefGoogle Scholar
  79. 79.
    Dunn OJ (1961). J Am Stat Assoc 56(293):52MATHMathSciNetCrossRefGoogle Scholar
  80. 80.
    Friedman M (1937). J Am Stat Assoc 32(200):675CrossRefGoogle Scholar
  81. 81.
    Wilcoxon F (1945) Biometrics bulletinGoogle Scholar
  82. 82.
    Abbot JJ, Marayong P, Okamura AM (2007). Robot Res 28(1):49CrossRefGoogle Scholar
  83. 83.
    Gao W, Dong R, Thamphiwatana S, Li J, Gao W, Zhang L, Wang J (2015). ACS Nano 9 (1):117CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Department of Advanced RoboticsIstituto Italiano di TecnologiaGenovaItaly
  2. 2.Institute for Integrative NanosciencesIFW DresdenDresdenGermany
  3. 3.Material Systems for NanoelectronicsUniversity of Technology ChemnitzChemnitzGermany
  4. 4.Department of Information Engineering and MathematicsUniversity of SienaSienaItaly
  5. 5.Surgical Robotics Laboratory, Department of Biomechanical EngineeringMIRA - Institute for Biomedical Technology and Technical Medicine, University of TwenteEnschedeThe Netherlands
  6. 6.Department of Biomedical EngineeringUniversity of Groningen and University Medical Centre GroningenGroningenThe Netherlands

Personalised recommendations