Skip to main content

Advertisement

SpringerLink
Log in

Intuitive control of self-propelled microjets with haptic feedback

  • Research Paper
  • Published:
Journal of Micro-Bio Robotics Aims and scope Submit manuscript

Abstract

Self-propelled microrobots have recently shown promising results in several scenarios at the microscale, such as targeted drug delivery and micromanipulation of cells. However, none of the steering systems available in the literature enable humans to intuitively and effectively control these microrobots in the remote environment, which is a desirable feature. In this paper we present an innovative teleoperation system with force reflection that enables a human operator to intuitively control the positioning of a self-propelled microjet. A particle-filter-based visual tracking algorithm tracks at runtime the position of the microjet in the remote environment. A 6-degrees-of-freedom haptic interface then provides the human operator with compelling haptic feedback about the interaction between the controlled microjet and the environment, as well as enabling the operator to intuitively control the target position of the microjet. Finally, a wireless magnetic control system regulates the orientation of the microjet to reach the target point. The viability of the proposed approach is demonstrated through two experimentsz enrolling twenty-eight subjects. In both experiments providing haptic feedback significantly improved the performance and the perceived realism of the considered tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Solovev AA, Mei Y, Urena EB, Huang G, Schmidt OG (2009). Small 5(14):1688

    Article  Google Scholar 

  2. Khalil ISM, Magdanz V, Sanchez S, Schmidt OG, S M (2013). Applied Physics Letters 103 (17):172–404

    Article  Google Scholar 

  3. Woods SP, Constandinou TG (2011) Proc. international conference of the ieee engineering in medicine and biology society, pp 7372–7375

  4. Sanchez S, Solovev AA, Schulze S, Schmidt OG (2011). Chem Commun 47(2):698

    Article  Google Scholar 

  5. Solovev AA, Sanchez S, Pumera M, Mei YF, Schmidt OG (2010). Adv Funct Mater 20(15):2430

    Article  Google Scholar 

  6. Zhang L, Petit T, Peyer KE, Nelson BJ (2012) Nanomedicine: Nanotechnology. Biol Med 8(7):1074

    Google Scholar 

  7. Balasubramanian S, Kagan D, Jack Hu CM, Campuzano S, Lobo-Castañon MJ, Lim N, Kang DY, Zimmerman M, Zhang L, Wang J (2011). Angew Chem Int Ed 50(18):4161

    Article  Google Scholar 

  8. Kagan D, Campuzano S, Balasubramanian S, Kuralay F, Flechsig GU, Wang J (2011). Nano Lett 11(5):2083

    Article  Google Scholar 

  9. Soler L, Magdanz V, Fomin VM, Sanchez SO, Schmidt OG (2013). ACS Nano 7:9611

    Article  Google Scholar 

  10. Xi W, Solovev AA, Ananth AN, Gracias DH, Sanchez S, Schmidt OG (2013). Nanoscale 5:1294

    Article  Google Scholar 

  11. Matteucci M, Casella M, Bedoni M, Donetti E, Fanetti M, De Angelis F, Gramatica F, Di Fabrizio E (2008). Microelectron Eng 85(5):1066

    Article  Google Scholar 

  12. Solovev AA, Xi W, Gracias DH, Harazim SM, Deneke SM, Sanchez S, Schmidt OG (2012). ACS Nano 6:1751

    Article  Google Scholar 

  13. Guix M, Orozco J, Garcia M, Gao W, Sattayasamitsathit S, Merkoci A, Wang J (2012). ACS Nano 6:4445

    Article  Google Scholar 

  14. Kuralay F, Sattayasamitsathit S, Gao W, Uygun A, Katzenberg A, Wang J (2012). J Am Chem Soc 134:15217

    Article  Google Scholar 

  15. Orozco J, Corts A, Cheng G, Sattayasamitsathit S, Gao W, Feng X, Shen Y, Wang J (2013). J Am Chem Soc 135:5336

    Article  Google Scholar 

  16. Wu Z, Wu Y, He W, Lin X, Sun J, He Q (2013). Angew Chem Int Ed 52:7000

    Article  Google Scholar 

  17. Nelson BJ, Kaliakatsos IK, Abbott JJ (2010). Annu Rev Biomed Eng 12:55

    Article  Google Scholar 

  18. Fournier-Bidoz S, Arsenault AC, Manners I, Ozin GA (2005). Chem Commun 4:441

    Article  Google Scholar 

  19. Nelson IK B, Abbott J (2010). Annu Rev Biomed Eng 12:55

    Article  Google Scholar 

  20. Paxton WF, Sundararajan S, Mallouk TE, Sen A (2006). Angew Chem Int Ed 45(33):5420

    Article  Google Scholar 

  21. Golestanian R, Liverpool TB, Ajdari A (2005), vol 94, pp 220–801

  22. Catchmark JM, Subramanian S, Sen A (2005). Small 1(2): 202

    Article  Google Scholar 

  23. Sanchez S, Ananth AN, Fomin VM, Viehrig M, Schmidt OG (2011). J Am Chem Soc 133(38):14860

    Article  Google Scholar 

  24. Khalil ISM, Magdanz V, Sanchez S, Schmidt OG, Misra S (2014). PLoS One 9(2):e83–053

    Article  Google Scholar 

  25. Sanchez A, Magdanz V, Schmidt OG, Misra S (2014) Proc. 5th IEEE RAS & EMBS international conference on biomedical robotics and biomechatronics

  26. Khalil ISM, Magdanz V, Sanchez S, Schmidt OG, Misra S (2014). IEEE Trans Robot 30(1):49

    Article  Google Scholar 

  27. Troccaz J, Delnondedieu Y (1996). Mechatronics 6(4):399

    Article  Google Scholar 

  28. Pacchierotti C, Abayazid M, Misra S, Prattichizzo D (2014). IEEE Tran Haptics 7(4):551

    Article  Google Scholar 

  29. Jakopec M, Baena FR y, Harris S J, Gomes P, Cobb J, Davies BL (2003). IEEE Trans Robot Autom 19(5):902

    Article  Google Scholar 

  30. Hashtrudi-Zaad K, Salcudean SE (2002). IEEE Trans Robot Autom 18(1):108

    Article  Google Scholar 

  31. Okamura AM (2004). Ind Robot An Int J 31(6):499

    Article  Google Scholar 

  32. Okamura AM (2009). Curr Opin Urol 19(1)

  33. Westebring-Van Der Putten EP, Goossens RHM, Jakimowicz JJ, Dankelman J (2008). Minim Invasive Ther Allied Technol 17(1):3

    Article  Google Scholar 

  34. van der Meijden OAJ, Schijven MP (2009). Surg Endosc 23(6):1180

    Article  Google Scholar 

  35. Wedmid A, Llukani E, Lee DI (2011). BJU Int 108:1028

    Article  Google Scholar 

  36. Prattichizzo D, Pacchierotti C, Cenci S, Minamizawa K, Rosati G (2010) Haptics: Generating and Perceiving Tangible Sensations

  37. Prattichizzo D, Pacchierotti C, Rosati G (2012). IEEE Trans Haptics 5(4):289

    Article  Google Scholar 

  38. Massimino MJ, Sheridan TB (1994). Human Factors: The Journal of the Human Factors and Ergonomics Society 36(1):145

    Google Scholar 

  39. Pacchierotti C, Chinello F, Malvezzi M, Meli L, Prattichizzo D (2012) Haptics: perception, devices, mobility, and communication

  40. Wagner CR, Howe RD, Stylopoulos N (2002) Proc. international symposium on haptic interfaces for virtual environment and teleoperator systems

  41. Meli L, Pacchierotti C, Prattichizzo D (2014). IEEE Trans Biomed Eng 61(4):1318

    Article  Google Scholar 

  42. Salcudean SE, Ku S, Bell G (1997) Proc. first joint conference on computer vision, virtual reality and robotics in medicine and medial robotics and computer-assisted surgery

  43. Kazi A (2001). Presence: Teleoperators & Virtual Environments 10(5):495

    Article  Google Scholar 

  44. Moody L, Baber C, Arvanitis TN, et al. (2002). Studies in health technology and informatics 85:304

    Google Scholar 

  45. Kennedy CW, Hu T, Desai JP, Wechsler AS, Kresh JY (2002). Cardiovasc Eng 2(1):15

    Article  Google Scholar 

  46. Pillarisetti A, Pekarev M, Brooks AD, Desai JP (2007). IEEE Trans Autom Sci Eng 4(3):322

    Article  Google Scholar 

  47. Ando N, Korondi P, Hashimoto H (2001). IEEE/ASME Trans Mechatron 6(4):417

    Article  Google Scholar 

  48. Mehrtash M, Tsuda N, Khamesee MB (2011). IEEE/ASME Trans Mechatron 16(3):459

    Article  Google Scholar 

  49. Bolopion A, Régnier S (2013). IEEE Trans Autom Sci Eng 10(3):496

    Article  Google Scholar 

  50. Ghanbari A, Horan B, Nahavandi S, Chen X, Wang W (2014). IEEE Syst J 8(2):371

    Article  Google Scholar 

  51. van der Schaft A L2-gain and passivity techniques in nonlinear control (Springer Verlag, 2000)

  52. Niemeyer G, Slotine JJE (2004). Int J Robot Res 23(9):873

    Article  Google Scholar 

  53. Ryu J, Kwon D, Hannaford B (2004). IEEE Trans Robot Autom 20(2):365

    Article  Google Scholar 

  54. Kim J, Ryu J (2010). Int J Robot Res 29(6):666

    Article  Google Scholar 

  55. Franken M, Stramigioli S, Misra S, Secchi C, Macchelli A (2011). IEEE Trans Robot 27(4):741

    Article  Google Scholar 

  56. Schoonmaker RE, Cao CG Proc. IEEE International Conference on Systems, Man and Cybernetics, vol. 3 (2006), vol. 3, 2464–2469

  57. Kitagawa M, Dokko D, Okamura AM, Yuh DD (2005). J Thorac Cardiovasc Surg 129(1):151

    Article  Google Scholar 

  58. Ramos A, Pacchierotti C, Prattichizzo D Proc. IEEE World Haptics Conference (WHC) (2013), 473–478

  59. Zilles CB (1995) Proc. IEEE/RSJ international conference of intelligent robots and systems

  60. Haralock RM, Shapiro LG (1991) Computer and robot vision, Addison-Wesley Longman Publishing Co., Inc.

  61. Arulampalam MS, Maskell S, Gordon N, Clapp T (2002). IEEE Trans Signal Process 50(2):174

    Article  Google Scholar 

  62. Baker S, Matthews I (2004). Int J Comput Vis 56(3):221

    Article  Google Scholar 

  63. Berg B (2014) University of Twente

  64. Utkin VI, Chang H (2002). Math Probl Eng 8:4

    Article  MathSciNet  Google Scholar 

  65. Bahaj AS, James PAB, Moeschler FD (1996). IEEE Trans Magn 32:5133

    Article  Google Scholar 

  66. Fomin VM, Hippler M, Magdanz V, Soler L, Sanchez S, Schmidt OG (2014). IEEE Trans Robot 30(1):40

    Article  Google Scholar 

  67. Solovev AA, Smith EJ, Bof’Bufon CC, Sanchez S, Schmidt OG (2011). Angew Chem Int Ed 50 (46):10875

    Article  Google Scholar 

  68. Magdanz V, Stoychev G, Ionov L, Sanchez S, Schmidt OG, et al. (2014). Angew Chem Int Ed 53 (10):2673

    Article  Google Scholar 

  69. Xia Y, Whitesides GM (1998). Annu Rev Mater Sci 28(1):153

    Article  Google Scholar 

  70. Cholewiak RW, Collins AA The psychology of touch, laurence erlbaum associates (1991), 13–60

  71. Kaczmarek KA, Webster JG, Rita P.B. y., Tompkins W J (1991) IEEE Trans Biomed Eng 38(1):1

  72. Diller E, Giltinan J, Sitti M (2013). Int J Robot Res 32(5): 614

    Article  Google Scholar 

  73. Diller E, Floyd S, Pawashe C, Sitti M (2012). IEEE Trans Robot 28(1):172

    Article  Google Scholar 

  74. Moody L, Baber C, Arvanitis TN, et al. (2002) Studies in health technology and informatics:304–310

  75. Pacchierotti C, Prattichizzo D, Kuchenbecker KJ (2015) IEEE Trans Biomed Eng

  76. Shapiro SS, Wilk MB (1965) Biometrika

  77. Mauchly JW (1940). Ann Math Stat 11(2):204

    Article  MathSciNet  Google Scholar 

  78. Gueorguieva R, Krystal JH (2004). Arch Gen Psychiatr 61(3):310

    Article  Google Scholar 

  79. Dunn OJ (1961). J Am Stat Assoc 56(293):52

    Article  MATH  MathSciNet  Google Scholar 

  80. Friedman M (1937). J Am Stat Assoc 32(200):675

    Article  Google Scholar 

  81. Wilcoxon F (1945) Biometrics bulletin

  82. Abbot JJ, Marayong P, Okamura AM (2007). Robot Res 28(1):49

    Article  Google Scholar 

  83. Gao W, Dong R, Thamphiwatana S, Li J, Gao W, Zhang L, Wang J (2015). ACS Nano 9 (1):117

    Article  Google Scholar 

Download references

Acknowledgments

The authors thank Alonso Sanchez for his help in setting up the tracking and control systems, and Frank van den Brink and Momen Abayazid for their help in making the video.

Author information

Authors and Affiliations

  1. Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genova, Italy

    Claudio Pacchierotti & Domenico Prattichizzo

  2. Institute for Integrative Nanosciences, IFW Dresden, Dresden, Germany

    Veronika Magdanz, Mariana Medina-Sánchez & Oliver G. Schmidt

  3. Material Systems for Nanoelectronics, University of Technology Chemnitz, Chemnitz, Germany

    Oliver G. Schmidt

  4. Department of Information Engineering and Mathematics, University of Siena, Siena, Italy

    Domenico Prattichizzo

  5. Surgical Robotics Laboratory, Department of Biomechanical Engineering, MIRA - Institute for Biomedical Technology and Technical Medicine, University of Twente, Enschede, The Netherlands

    Sarthak Misra

  6. Department of Biomedical Engineering, University of Groningen and University Medical Centre Groningen, Groningen, The Netherlands

    Sarthak Misra

Authors
  1. Claudio Pacchierotti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Veronika Magdanz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mariana Medina-Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Oliver G. Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Domenico Prattichizzo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sarthak Misra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claudio Pacchierotti.

Additional information

This project (ROBOTAR) has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 Research and Innovation programme (Grant Agreement #638428). The research leading to these results has also received funding from the European Union Seventh Framework Programme FP7/2007-2013 under grant agreement #601165 of project “WEARHAP - WEARable HAPtics for humans and robots”.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pacchierotti, C., Magdanz, V., Medina-Sánchez, M. et al. Intuitive control of self-propelled microjets with haptic feedback. J Micro-Bio Robot 10, 37–53 (2015). https://doi.org/10.1007/s12213-015-0082-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12213-015-0082-7

Keywords

Search

Navigation