Skip to main content
SpringerLink
Log in

Simulating the obstacle avoidance behavior day and night based on the visible-infrared MoS2/Ge heterojunction field-effect phototransistor

  • Research Article
  • Published:
Nano Research Aims and scope Submit manuscript

Abstract

The contradiction between the high number of visually handicapped people and the scarcity of guide dogs has stimulated the demand for electronic guide dogs (EGDs). Here, we demonstrate an EGD by leveraging piezoresistors on a MoS2/Ge heterostructure for simultaneous pressure-sensing and optical-sensing functions. The device has excellent gating capability and exhibits large positive and negative photoresponses under visible (532 nm, 182 A/W) and infrared (1550 nm, 37 A/W) illumination. These characteristics allow the device to efficiently classify different obstacles at all times of day using pressure and light signals. The device reaches nearly 100% accuracy after 48 training sessions when used to classify frequent scenes. The device adopts passive and active detection modes during the day and night, respectively, which improves the battery life of the EGD. This work provides a significant reference for the future design of EGDs, which may help a greater number of visually impaired people by reducing the cost of such devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Gori, M.; Cappagli, G.; Tonelli, A.; Baud-Bovy, G.; Finocchietti, S. Devices for visually impaired people: High technological devices with low user acceptance and no adaptability for children. Neurosci. Biobehav. Rev. 2016, 69, 79–88.

    Article  Google Scholar 

  2. Hu, M. H.; Chen, Y. Z.; Zhai, G. T.; Gao, Z. P.; Fan, L. An overview of assistive devices for blind and visually impaired people. Int. J. Rob. Autom. 2019, 34, 580–589.

    Google Scholar 

  3. Sielicki, I.; Bartkowski, Z.; Bartkowska, J. Significance of colors and contrasts in the life of the visually impaired. In Proceedings of SPIE 4535, Optical Sensing for Public Safety, Health, and Security, Warsaw, pp 41–44.

  4. King, G. A.; Shultz, I. Z.; Steel, K.; Gilpin, M.; Cathers, T. Self-evaluation and self-concept of adolescents with physical disabilities. Am. J. Occup. Ther. 1993, 47, 132–140.

    Article  CAS  Google Scholar 

  5. Roe, J.; Webster, A. Children with Visual Impairments: Social Interaction, Language and Learning; Routledge: London, 1998.

    Google Scholar 

  6. West, S. K.; Rubin, G. S.; Broman, A. T.; MuMuñozoz, B.; Bandeen-Roche, K.; Turano, K. How does visual impairment affect performance on tasks of everyday life. The SEE project. Arch. Ophthalmol. 2002, 720, 774–780.

    Article  Google Scholar 

  7. Whitmarsh, L. The benefits of guide dog ownership. Visual Impairment Res. 2005, 7, 27–42.

    Article  Google Scholar 

  8. Wiggett-Barnard, C.; Steel, H. The experience of owning a guide dog. Disabil. Rehabil. 2008, 30, 1014–1026.

    Article  Google Scholar 

  9. Camp, M. M. The use of service dogs as an adaptive strategy: A qualitative study. Am. J. Occup. Ther. 2001, 55, 509–517.

    Article  CAS  Google Scholar 

  10. Hong, B.; Lin, Z. X.; Chen, X.; Hou, J.; Lv, S. Y.; Gao, Z. D. Development and application of key technologies for guide dog robot: A systematic literature review. Rob. Auton. Syst. 2022, 154, 104104.

    Article  Google Scholar 

  11. McIver, S.; Hall, S.; Mills, D. S. The impact of owning a guide dog on owners’ quality of life: A longitudinal study. Anthrozoö s 2020, 33, 103–117.

    Article  Google Scholar 

  12. Batt, L.; Batt, M.; Baguley, J.; McGreevy, P. The effects of structured sessions for juvenile training and socialization on guide dog success and puppy-raiser participation. J. Vet. Behav. 2008, 3, 199–206.

    Article  Google Scholar 

  13. Batt, L. S.; Batt, M. S.; Baguley, J. A.; McGreevy, P. D. Factors associated with success in guide dog training. J. Vet. Behav. 2008, 3, 143–151.

    Article  Google Scholar 

  14. Bray, E. E.; Sammel, M. D.; Cheney, D. L.; Serpell, J. A.; Seyfarth, R. M. Effects of maternal investment, temperament, and cognition on guide dog success. Proc. Natl. Acad. Sci. USA 2017, 144, 9128–9133.

    Article  Google Scholar 

  15. Murphy, J. A. Describing categories of temperament in potential guide dogs for the blind. Appl. Anim. Behav. Sci. 1998, 58, 163–178.

    Article  Google Scholar 

  16. Ulrich, I.; Borenstein, J. The GuideCane-applying mobile robot technologies to assist the visually impaired. IEEE Trans. Syst. Man Cybern A Syst. Hum. 2001, 31, 131–136.

    Article  Google Scholar 

  17. Bruno, D. R.; De Assis, M. H.; Osorio, F. S. Development of a mobile robot: Robotic guide dog for aid of visual disabilities in urban environments. In Proceedings of the 2019 Latin American Robotics Symposium (LARS), 2019 Brazilian Symposium on Robotics (SBR) and 2019 Workshop on Robotics in Education (WRE), Rio Grande, 2019, pp 104–108.

  18. Due, B. L. Guide dog versus robot dog: Assembling visually impaired people with non-human agents and achieving assisted mobility through distributed co-constructed perception. Mobilities 2023, 18, 148–166.

    Article  Google Scholar 

  19. Feng, C.; Azenkot, S.; Cakmak, M. Designing a robot guide for blind people in indoor environments. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts, Portland, 2015, pp 107–108.

  20. Gao, T.; Chen, C. Y.; Ou, H. Y.; Wen, X. L.; Yang, Y. C.; Xie, Y. Z. Product design of intelligent blind guide stick based on humanistic care. In 2022 3rd International Conference on Language, Art and Cultural Exchange (ICLACE 2022), Amsterdam, 2022, pp 757–762.

  21. Huang, C. Q.; Liu, L. L. Application of the constitutive model in finite element simulation: Predicting the flow behavior for 5754 aluminum alloy during hot working. Metals 2017, 7, 331.

    Article  Google Scholar 

  22. Kulkarni, A.; Wang, A.; Urbina, L.; Steinfeld, A.; Dias, B. Robotic assistance in indoor navigation for people who are blind. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Christchurch, 2016, pp 461–462.

  23. Wei, Y.; Lee, M. A guide-dog robot system research for the visually impaired. In 2014 IEEE International Conference on Industrial Technology (ICIT), 2014; pp 800–805.

  24. Saegusa, S.; Yasuda, Y.; Uratani, Y.; Tanaka, E.; Makino, T.; Chang, J. Y. Development of a guide-dog robot: Leading and recognizing a visually-handicapped person using a LRF. J. Adv. Mech. Des. Syst. Manuf. 2010, 4, 194–205.

    Article  Google Scholar 

  25. Tachi, S.; Tanie, K.; Komoriya, K.; Abe, M. Electrocutaneous communication in a guide dog robot (MELDOG). IEEE Trans. Biomed. Eng. 1985, BME-32, 461–469.

    Article  Google Scholar 

  26. Tachi, S.; Tanie, K.; Komoriya, K.; Hosoda, Y.; Abe, M. Guide dog robot—Its basic plan and some experiments with Meldog Mark I. Mech. Mach. Theory 1981, 16, 21–29.

    Article  Google Scholar 

  27. Zeng, J.; Yang, P.; Xu, C. M.; Jiang, S. Q.; Peng, D. G. Design and implementation of guide-dog robot. Mechatronics 2005, 11, 22–25.

    Google Scholar 

  28. Wu, X. M.; Li, E. L.; Liu, Y. Q.; Lin, W. K.; Yu, R. J.; Chen, G. X.; Hu, Y. Y.; Chen, H. P.; Guo, T. L. Artificial multisensory integration nervous system with haptic and iconic perception behaviors. Nano Energy 2021, 85, 106000.

    Article  CAS  Google Scholar 

  29. You, J.; Wang, L. M.; Zhang, Y. C.; Lin, D. D.; Wang, B.; Han, Z.; Zhang, N. N.; Miao, T.; Liu, M. L.; Jiang, Z. M. et al. Simulating tactile and visual multisensory behaviour in humans based on an MoS2 field effect transistor. Nano Res., 2023, 16, 7405–7412.

    Article  Google Scholar 

  30. Yu, J. R.; Yang, X. X.; Gao, G. Y.; Xiong, Y.; Wang, Y. F.; Han, J.; Chen, Y. H.; Zhang, H.; Sun, Q. J.; Wang, Z. L. Bioinspired mechano-photonic artificial synapse based on graphene/MoS2 heterostructure. Sci. Adv. 2021, 7, eabd9117.

    Article  CAS  Google Scholar 

  31. Zhang, Z. X.; Shi, Q. F.; He, T. Y. Y.; Guo, X. G.; Dong, B. W.; Lee, J.; Lee, C. Artificial intelligence of toilet (AI-toilet) for an integrated health monitoring system (IHMS) using smart triboelectric pressure sensors and image sensor. Nano Energy 2021, 12, 106517.

    Article  Google Scholar 

  32. Pi, L.; Wang, P.; Liang, S.-J.; Luo, P.; Wang, H.; Li, D.; Li, Z.; Chen, P.; Zhou, X.; Miao, F.; Zhai, T. Broadband convolutional processing using band-alignment-tunable heterostructures. Nature Electronics 2022, 5, 248–254.

    Article  CAS  Google Scholar 

  33. Wang, Y.; Gan, L.; Chen, J.; Yang, R.; Zhai, T. Achieving highly uniform two-dimensional PbI(2) flakes for photodetectors via space confined physical vapor deposition. Sci Bull 2017, 62, 1654–1662.

    Article  CAS  Google Scholar 

  34. Zhang, Q.; Gao, W.; Xiong, Q. Advanced optical gain materials keep on giving. Science China Materials 2020, 63, 1345–1347.

    Article  Google Scholar 

  35. Wang, C.-Y.; Liang, S.-J.; Wang, S.; Wang, P.; Li, Z. A.; Wang, Z.; Gao, A.; Pan, C.; Liu, C.; Liu, J. Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor. Science Advances 2020, 6, eaba6173.

    Article  Google Scholar 

  36. Zha, C.; Yan, X.; Yuan, X.; Zhang, Y.; Zhang, X. An artificial optoelectronic synapse based on an InAs nanowire phototransistor with negative photoresponse. Optical and Quantum Electronics 2021, 53.

  37. Zhang, Z.; Wang, S.; Liu, C.; Xie, R.; Hu, W.; Zhou, P. All-in-one two-dimensional retinomorphic hardware device for motion detection and recognition. Nat Nanotechnol 2022, 17, 27–32.

    Article  Google Scholar 

  38. Wang, B.; Wang, L.; Zhang, Y.; Yang, M.; Lin, D.; Zhang, N.; Jiang, Z.; Liu, M.; Zhu, Z.; Hu, H. Mixed-dimensional MoS2/Ge heterostructure junction field—Effect transistors for logic operation and photodetection. Advanced Functional Materials 2021, 32.

  39. Jiang, Y.; Chen, S.; Zheng, W.; Zheng, B.; Pan, A. Interlayer exciton formation, relaxation, and transport in TMD van der Waals heterostructures. Light Sci Appl 2021, 12, 72.

    Article  Google Scholar 

  40. Street, R. A. Electronic structure and properties of organic bulk-heterojunction interfaces. Adv Mater 2016, 28, 3814–3830.

    Article  CAS  Google Scholar 

  41. Bayraci, S.; Susuz, O. A deep neural network (DNN) based classification model in application to loan default prediction. Theoretical and Applied Economics 2019, 4, 75–84.

    Google Scholar 

  42. Mohsen, H.; El-Dahshan, E.-S. A.; El-Horbaty, E.-S. M.; Salem, A.-B. M. Classification using deep learning neural networks for brain tumors. Future Computing and Informatics Journal 2018, 3, 68–71.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Natural Science Basic Research Program of Shaanxi (No. 2022JQ-650), the Natural Science Foundation of China (NSFC) (No. 62204188), and the Cooperation Program of XDU-Chongqing IC Innovation Research Institute (No.CQIRI-CXYHT-2022-13)

Author information

Authors and Affiliations

  1. Key Laboratory of Analog Integrated Circuits and Systems, Ministry of Education, School of Microelectronics, Xidian University, Xi’an, 710071, China

    Zhao Han, Bo Wang, Jie You, Qiancui Zhang, Yichi Zhang, Tian Miao, Ningning Zhang, Renxu Jia, Jincheng Zhang, Hui Guo, Huiyong Hu & Liming Wang

  2. Qian Xuesen Collaborative Research Center of Astrochemistry and Space Life Sciences, Department of Microelectronic Science and Engineering, Ningbo University, Ningbo, 315211, China

    Dongdong Lin

  3. Department of Physics, Fudan University, Shanghai, 200433, China

    Zuimin Jiang

Authors
  1. Zhao Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie You
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiancui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yichi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tian Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ningning Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Dongdong Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Zuimin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Renxu Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Jincheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Hui Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Huiyong Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Liming Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liming Wang.

Electronic Supplementary Material

12274_2023_5816_MOESM1_ESM.pdf

Simulating the obstacle avoidance behavior day and night based on the visible-infrared MoS2/Ge heterojunction field-effect phototransistor

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, Z., Wang, B., You, J. et al. Simulating the obstacle avoidance behavior day and night based on the visible-infrared MoS2/Ge heterojunction field-effect phototransistor. Nano Res. 16, 11296–11302 (2023). https://doi.org/10.1007/s12274-023-5816-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12274-023-5816-6

Keywords

Search

Navigation