Skip to main content
SpringerLink
Log in

Bioinspired sensing-memory-computing integrated vision systems: biomimetic mechanisms, design principles, and applications

  • Review
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

With the explosion of sensory data in the Internet of Things (IoT) era, conventional machine vision systems are becoming increasingly difficult to meet the requirements of high efficiency, low energy consumption, and low latency due to their inherent shortcomings of separate sensing, memory, and computing units. Inspired by the retina and neuromorphic computing, the sensing-memory-computing integrated vision system (SMCVS) that features low power consumption, low latency, and high parallelism has been considered a promising technology to surpass the von Neumann architecture and realize strong artificial intelligence. Meanwhile, novel materials like two-dimensional semiconductors and quantum dots with novel optoelectronic performance provide hardware carriers for implementing integrated sensing-memory-computing architectures, attracting considerable attention. This paper reviews the recent research progress in bioinspired vision systems in terms of biomimetic mechanisms, design principles, computational architectures, and applications. Firstly, the biomimetic mechanisms are illustrated to guide the design of high-performance artificial visual perception systems. Then the research progress of optoelectronic-synapse-based bioinspired vision systems in the device principles and applications including image filtering, color recognition, visual adaptation, and motion detection are summarized. Finally, the challenges and future developing directions of the SMCVS are provided regarding bionic application, architecture design, and device fabrication.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Mennel L, Symonowicz J, Wachter S, et al. Ultrafast machine vision with 2D material neural network image sensors. Nature, 2020, 579: 62–66

    Article  Google Scholar 

  2. Chai Y. In-sensor computing for machine vision. Nature, 2020, 579: 32–33

    Article  Google Scholar 

  3. Yang R, Huang H, Hong Q, et al. Synaptic suppression triplet-STDP learning rule realized in second-order memristors. Adv Funct Mater, 2018, 28: 1704455

    Article  Google Scholar 

  4. Zhong Y, Wang T, Gao X, et al. Synapse-like organic thin film memristors. Adv Funct Mater, 2018, 28: 1800854

    Article  Google Scholar 

  5. Zhou Y X, Li Y, Su Y T, et al. Nonvolatile reconfigurable sequential logic in a HfO2 resistive random access memory array. Nanoscale, 2017, 9: 6649–6657

    Article  Google Scholar 

  6. Zhou F, Chai Y. Near-sensor and in-sensor computing. Nat Electron, 2020, 3: 664–671

    Article  Google Scholar 

  7. Wan T, Shao B, Ma S, et al. In-sensor computing: materials, devices, and integration technologies. Adv Mater, 2023, 35

  8. van de Burgt Y, Lubberman E, Fuller E J, et al. A non-volatile organic electrochemical device as a low-voltage artificial synapse for neuromorphic computing. Nat Mater, 2017, 16: 414–418

    Article  Google Scholar 

  9. Xia Q, Yang J J. Memristive crossbar arrays for brain-inspired computing. Nat Mater, 2019, 18: 309–323

    Article  Google Scholar 

  10. Radovic A, Williams M, Rousseau D, et al. Machine learning at the energy and intensity frontiers of particle physics. Nature, 2018, 560: 41–48

    Article  Google Scholar 

  11. Wang C, Liang S J, Wang C Y, et al. Scalable massively parallel computing using continuous-time data representation in nanoscale crossbar array. Nat Nanotechnol, 2021, 16: 1079–1085

    Article  Google Scholar 

  12. Song Y M, Xie Y, Malyarchuk V, et al. Digital cameras with designs inspired by the arthropod eye. Nature, 2013, 497: 95–99

    Article  Google Scholar 

  13. Kim Y, Chortos A, Xu W, et al. A bioinspired flexible organic artificial afferent nerve. Science, 2018, 360: 998–1003

    Article  Google Scholar 

  14. Masland R H. The fundamental plan of the retina. Nat Neurosci, 2001, 4: 877–886

    Article  Google Scholar 

  15. Egmont-Petersen M, de Ridder D, Handels H. Image processing with neural networks-a review. Pattern Recogn, 2002, 35: 2279–2301

    Article  Google Scholar 

  16. Gollisch T, Meister M. Eye smarter than scientists believed: neural computations in circuits of the retina. Neuron, 2010, 65: 150–164

    Article  Google Scholar 

  17. Lee W, Lee J, Yun H, et al. High-resolution spin-on-patterning of perovskite thin films for a multiplexed image sensor array. Adv Mater, 2017, 29: 1702902

    Article  Google Scholar 

  18. Zhang L, Pasthukova N, Yao Y, et al. Self-suspended nanomesh scaffold for ultrafast flexible photodetectors based on organic semiconducting crystals. Adv Mater, 2018, 30: 1801181

    Article  Google Scholar 

  19. McCollough C. Color adaptation of edge-detectors in the human visual system. Science, 1965, 149: 1115–1116

    Article  Google Scholar 

  20. Baden T, Osorio D. The retinal basis of vertebrate color vision. Annu Rev Vis Sci, 2019, 5: 177–200

    Article  Google Scholar 

  21. Stockman A, MacLeod D I A, Johnson N E. Spectral sensitivities of the human cones. J Opt Soc Am A, 1993, 10: 2491–2521

    Article  Google Scholar 

  22. Kolb H. How the retina works. Am Sci, 2003, 91: 28–35

    Article  Google Scholar 

  23. Kandel E, Schwartz J, Jessell T, et al. Low-level visual processing: the retina. In: Principles of Neural Science. 5th ed. Columbus: McGraw-Hill Education, 2014

    Google Scholar 

  24. Gray J R, Blincow E, Robertson R M. A pair of motion-sensitive neurons in the locust encode approaches of a looming object. J Comp Physiol A, 2010, 196: 927–938

    Article  Google Scholar 

  25. Fotowat H, Gabbiani F. Collision detection as a model for sensory-motor integration. Annu Rev Neurosci, 2011, 34: 1–19

    Article  Google Scholar 

  26. Glantz R M. Defense reflex and motion detector responsiveness to approaching targets: The motion detector trigger to the defense reflex pathway. J Comp Physiol, 1974, 95: 297–314

    Article  Google Scholar 

  27. Chen T, Lu S. Object-level motion detection from moving cameras. IEEE Trans Circ Syst Video Technol, 2017, 27: 2333–2343

    Article  Google Scholar 

  28. Choo K, Xu L, Kim Y, et al. Energy-efficient low-noise CMOS image sensor with capacitor array-assisted charge-injection SAR ADC for motion-triggered low-power IoT applications. In: Proceedings of IEEE International Solid-State Circuits Conference, San Francisco, 2019

  29. Chen S, Lou Z, Chen D, et al. An artificial flexible visual memory system based on an UV-motivated memristor. Adv Mater, 2018, 30: 1705400

    Article  Google Scholar 

  30. Hu L, Yang J, Wang J, et al. All-optically controlled memristor for optoelectronic neuromorphic computing. Adv Funct Mater, 2021, 31: 2005582

    Article  Google Scholar 

  31. Cao G, Meng P, Chen J, et al. 2D material based synaptic devices for neuromorphic computing. Adv Funct Mater, 2021, 31: 2005443

    Article  Google Scholar 

  32. Liu C, Chen H, Wang S, et al. Two-dimensional materials for next-generation computing technologies. Nat Nanotechnol, 2020, 15: 545–557

    Article  Google Scholar 

  33. Pi L, Wang P, Liang S J, et al. Broadband convolutional processing using band-alignment-tunable heterostructures. Nat Electron, 2022, 5: 248–254

    Article  Google Scholar 

  34. Pei Y, Yan L, Wu Z, et al. Artificial visual perception nervous system based on low-dimensional material photoelectric memristors. ACS Nano, 2021, 15: 17319–17326

    Article  Google Scholar 

  35. Wang H, Zhao Q, Ni Z, et al. A ferroelectric/electrochemical modulated organic synapse for ultraflexible, artificial visual-perception system. Adv Mater, 2018, 30: 1803961

    Article  Google Scholar 

  36. He Z, Shen H, Ye D, et al. An organic transistor with light intensity-dependent active photoadaptation. Nat Electron, 2021, 4: 522–529

    Article  Google Scholar 

  37. Cui B, Fan Z, Li W, et al. Ferroelectric photosensor network: an advanced hardware solution to real-time machine vision. Nat Commun, 2022, 13: 1707

    Article  Google Scholar 

  38. Yan T, Cai Y C, Wang Y R, et al. Near-infrared optoelectronic synapses based on a Te/α-In2Se3 heterojunction for neuromorphic computing. Sci China Inf Sci, 2023, 66: 160404

    Article  Google Scholar 

  39. Wang Z, Zhou X, Liu X, et al. Van der Waals ferroelectric transistors: the all-round artificial synapses for high-precision neuromorphic computing. Chip, 2023, 2: 100044

    Article  Google Scholar 

  40. Yoo C, Ko T J, Kaium M G, et al. A minireview on 2D materials-enabled optoelectronic artificial synaptic devices. APL Mater, 2022, 10: 070702

    Article  Google Scholar 

  41. Peng Z R, Lin R F, Li Z, et al. Two-dimensional materials-based integrated hardware. Sci China Inf Sci, 2023, 66: 160401

    Article  Google Scholar 

  42. Chen Y, Kang Y, Hao H, et al. All two-dimensional integration-type optoelectronic synapse mimicking visual attention mechanism for multi-target recognition. Adv Funct Mater, 2023, 33: 2209781

    Article  Google Scholar 

  43. Wang C Y, Liang S J, Wang S, et al. Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor. Sci Adv, 2020, 6: 6173

    Article  Google Scholar 

  44. Zhou F, Zhou Z, Chen J, et al. Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat Nanotechnol, 2019, 14: 776–782

    Article  Google Scholar 

  45. Tan Y, Hao H, Chen Y, et al. A bioinspired retinomorphic device for spontaneous chromatic adaptation. Adv Mater, 2022, 34: 2206816

    Article  Google Scholar 

  46. Seo S, Jo S H, Kim S, et al. Artificial optic-neural synapse for colored and color-mixed pattern recognition. Nat Commun, 2018, 9: 5106

    Article  Google Scholar 

  47. Li D, Li C, Ilyas N, et al. Color-recognizing Si-based photonic synapse for artificial visual system. Adv Intell Syst, 2020, 2: 2000107

    Article  Google Scholar 

  48. Cai Y, Wang F, Wang X, et al. Broadband visual adaption and image recognition in a monolithic neuromorphic machine vision system. Adv Funct Mater, 2023, 33: 2212917

    Article  Google Scholar 

  49. Liao F, Zhou Z, Kim B J, et al. Bioinspired in-sensor visual adaptation for accurate perception. Nat Electron, 2022, 5: 84–91

    Article  Google Scholar 

  50. Kwon S M, Cho S W, Kim M, et al. Environment-adaptable artificial visual perception behaviors using a light-adjustable optoelectronic neuromorphic device array. Adv Mater, 2019, 31: 1906433

    Article  Google Scholar 

  51. Wang S, Wang C Y, Wang P, et al. Networking retinomorphic sensor with memristive crossbar for brain-inspired visual perception. Natl Sci Rev, 2021, 8: nwaa172

    Article  Google Scholar 

  52. Chen J, Zhou Z, Kim B J, et al. Optoelectronic graded neurons for bioinspired in-sensor motion perception. Nat Nanotechnol, 2023, 18: 882–888

    Article  Google Scholar 

  53. Zhang Z, Wang S, Liu C, et al. All-in-one two-dimensional retinomorphic hardware device for motion detection and recognition. Nat Nanotechnol, 2022, 17: 27–32

    Article  Google Scholar 

  54. Li Z P. A new framework for understanding vision from the perspective of the primary visual cortex. Curr Opin Neurobiol, 2019, 58: 1–10

    Article  Google Scholar 

  55. Watson C, Kirkcaldie M, Paxinos G. Gathering information-the sensory systems. In: The Brain. Sydney: Elsevier, 2010. 75–96

    Chapter  Google Scholar 

  56. Remington L A. Visual pathway. In: Clinical Anatomy and Physiology of the Visual System. Sydney: Elsevier, 2012. 233–252

    Chapter  Google Scholar 

  57. Alexander K R. Information processing: retinal adaptation. In: Encyclopedia of the Eye. New York: Academic Press, 2010, 379–386

    Chapter  Google Scholar 

  58. Xu X, Ichida J, Shostak Y, et al. Are primate lateral geniculate nucleus (LGN) cells really sensitive to orientation or direction? Vis Neurosci, 2002, 19: 97–108

    Article  Google Scholar 

  59. Tailby C, Dobbie W J, Solomon S G, et al. Receptive field asymmetries produce color-dependent direction selectivity in primate lateral geniculate nucleus. J Vision, 2010, 10: 1

    Article  Google Scholar 

  60. Lorach H, Goetz G, Smith R, et al. Photovoltaic restoration of sight with high visual acuity. Nat Med, 2015, 21: 476–482

    Article  Google Scholar 

  61. Gruhl T, Weinert T, Rodrigues M J, et al. Ultrafast structural changes direct the first molecular events of vision. Nature, 2023, 615: 939–944

    Article  Google Scholar 

  62. Palczewski K. Chemistry and biology of the initial steps in vision: the friedenwald lecture. Invest Ophthalmol Vis Sci, 2014, 55: 6651

    Article  Google Scholar 

  63. Buchsbaum G, Gottschalk A, Barlow H B. Trichromacy, opponent colours coding and optimum colour information transmission in the retina. Proc Royal Soc London Ser B Biol Sci, 1997, 220: 89–113

    Google Scholar 

  64. Freed M A. Contributions of bipolar cells to ganglion cell receptive fields. In: The Senses: A Comprehensive Reference. Orlando: Academic Press, 2008. 351–359

    Chapter  Google Scholar 

  65. Masland R H. The neuronal organization of the retina. Neuron, 2012, 76: 266–280

    Article  Google Scholar 

  66. Hubel D H, Wiesel T N. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol, 1962, 160: 106–154

    Article  Google Scholar 

  67. Brainard D H, Wandell B A. Analysis of the retinex theory of color vision. J Opt Soc Am A, 1986, 3: 1651–1661

    Article  Google Scholar 

  68. Maloney L T, Wandell B A. Color constancy: a method for recovering surface spectral reflectance. J Opt Soc Am A, 1986, 3: 29–33

    Article  Google Scholar 

  69. Werner A. Spatial and temporal aspects of chromatic adaptation and their functional significance for colour constancy. Vision Res, 2014, 104: 80–89

    Article  Google Scholar 

  70. Stockman A, Sharpe L T. Human cone spectral sensitivities: a progress report. Vision Res, 1998, 38: 3193–3206

    Article  Google Scholar 

  71. Viqueira P V, De F S, Martinez V F. Colour vision: theories and principles. In: Colour Measurement. Sawston Cambridge: Woodhead Publishing, 2010

    Google Scholar 

  72. Roy C A. Chromatic adaptation and colour constancy. In: Principles of Colour and Appearance Measurement. Amsterdam: Elsevier, 2015. 214–264

    Google Scholar 

  73. de Monasterio F M. Center and surround mechanisms of opponent-color X and Y ganglion cells of retina of macaques. J NeuroPhysiol, 1978, 41: 1418–1434

    Article  Google Scholar 

  74. Bowmaker J K, Dartnall H J. Visual pigments of rods and cones in a human retina. J Physiol, 1980, 298: 501–511

    Article  Google Scholar 

  75. Barlow H B. Temporal and spatial summation in human vision at different background intensities. J Physiol, 1958, 141: 337–350

    Article  Google Scholar 

  76. Jie J, Deng W, Zhang X, et al. A phototransistor with visual adaptation. Nat Electron, 2021, 4: 460–461

    Article  Google Scholar 

  77. Kolb H, Fernandez E, Nelson R. Webvision: The Organization of the Retina and Visual System. Lombardy: University of Utah Health Sciences Center, 1995

    Google Scholar 

  78. Darmont A. High Dynamic Range Imaging: Sensors and Architectures. Bellingham: SPIE, 2013

    Book  Google Scholar 

  79. Shen J. On the foundations of vision modeling. Phys D-NOnlinear Phenomena, 2003, 175: 241–251

    Article  MathSciNet  Google Scholar 

  80. Posch C, Serrano-Gotarredona T, Linares-Barranco B, et al. Retinomorphic event-based vision sensors: bioinspired cameras with spiking output. Proc IEEE, 2014, 102: 1470–1484

    Article  Google Scholar 

  81. Miller E K. Straight from the top. Nature, 1999, 401: 650–651

    Article  Google Scholar 

  82. Lockhofen D E L, Mulert C. Neurochemistry of visual attention. Front Neurosci, 2021, 15: 643597

    Article  Google Scholar 

  83. Proffitt D R. Distance perception. Curr Dir Psychol Sci, 2006, 15: 131–135

    Article  Google Scholar 

  84. Nityananda V, Read J C A. Stereopsis in animals: evolution, function and mechanisms. J Exp Biol, 2017, 220: 2502–2512

    Article  Google Scholar 

  85. Parker A J. Stereoscopic vision. In: Encyclopedia of Neuroscience. Orlando: Academic Press, 2009. 411–417

    Chapter  Google Scholar 

  86. Iehisa I, Ayaki M, Tsubota K, et al. Factors affecting depth perception and comparison of depth perception measured by the three-rods test in monocular and binocular vision. Heliyon, 2020, 6: e04904

    Article  Google Scholar 

  87. Knaus K R, Hipsley A M, Blemker S S. The action of ciliary muscle contraction on accommodation of the lens explored with a 3D model. Biomech Model Mechanobiol, 2021, 20: 879–894

    Article  Google Scholar 

  88. Hao H, Kang Y, Xu Z, et al. Neuromorphology in-sensor computing architecture based on an optical Fourier transform. Opt Lett, 2021, 46: 5501–5504

    Article  Google Scholar 

  89. Wan C, Cai P, Wang M, et al. Artificial sensory memory. Adv Mater, 2020, 32: 1902434

    Article  Google Scholar 

  90. Yang Q, Luo Z, Zhang D, et al. Controlled optoelectronic response in van der Waals heterostructures for in-sensor computing. Adv Funct Mater, 2022, 32

  91. Kang Y, Chen Y, Tan Y, et al. Bioinspired activation of silent synapses in layered materials for extensible neuromorphic computing. J Materiomics, 2023, 9: 787–797

    Article  Google Scholar 

  92. Yao P, Wu H, Gao B, et al. Fully hardware-implemented memristor convolutional neural network. Nature, 2020, 577: 641–646

    Article  Google Scholar 

  93. Pei J, Deng L, Song S, et al. Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature, 2019, 572: 106–111

    Article  Google Scholar 

  94. Wan W, Kubendran R, Schaefer C, et al. A compute-in-memory chip based on resistive random-access memory. Nature, 2022, 608: 504–512

    Article  Google Scholar 

  95. Huo Q, Yang Y, Wang Y, et al. A computing-in-memory macro based on three-dimensional resistive random-access memory. Nat Electron, 2022, 5: 469–477

    Article  Google Scholar 

  96. Zimmer B, Venkatesan R, Shao Y S, et al. A 0.32-128 TOPS, scalable multi-chip-module-based deep neural network inference accelerator with ground-referenced signaling in 16 nm. IEEE J Solid-State Circ, 2020, 55: 920–932

    Article  Google Scholar 

  97. Tye N J, Hofmann S, Stanley-Marbell P. Materials and devices as solutions to computational problems in machine learning. Nat Electron, 2023, 6: 479–490

    Article  Google Scholar 

  98. Wang Y, Liu M, Yang J, et al. Data-driven deep learning for automatic modulation recognition in cognitive radios. IEEE Trans Veh Technol, 2019, 68: 4074–4077

    Article  Google Scholar 

  99. Jiang T, Wang Y, Zheng Y, et al. Tetrachromatic vision-inspired neuromorphic sensors with ultraweak ultraviolet detection. Nat Commun, 2023, 14: 2281

    Article  Google Scholar 

  100. Park H, Kim H, Lim D, et al. Retina-inspired carbon nitride-based photonic synapses for selective detection of UV light. Adv Mater, 2020, 32: 1906899

    Article  Google Scholar 

  101. Seung H, Choi C, Kim D C, et al. Integration of synaptic phototransistors and quantum dot light-emitting diodes for visualization and recognition of UV patterns. Sci Adv, 2022, 8: eabq3101

    Article  Google Scholar 

  102. Qiu W, Huang Y, Kong L, et al. Optoelectronic In-Ga-Zn-O memtransistors for artificial vision system. Adv Funct Mater, 2020, 30: 2002325

    Article  Google Scholar 

  103. Yang X, Xiong Z, Chen Y, et al. A self-powered artificial retina perception system for image preprocessing based on photovoltaic devices and memristive arrays. Nano Energy, 2020, 78: 105246

    Article  Google Scholar 

  104. Shan X, Zhao C, Wang X, et al. Plasmonic optoelectronic memristor enabling fully light-modulated synaptic plasticity for neuromorphic vision. Adv Sci, 2022, 9: 2104632

    Article  Google Scholar 

  105. Dodda A, Jayachandran D, Pannone A, et al. Active pixel sensor matrix based on monolayer MoS2 phototransistor array. Nat Mater, 2022, 21: 1379–1387

    Article  Google Scholar 

  106. Li C, Hu M, Li Y, et al. Analogue signal and image processing with large memristor crossbars. Nat Electron, 2018, 1: 52–59

    Article  Google Scholar 

  107. Jang H, Liu C, Hinton H, et al. An atomically thin optoelectronic machine vision processor. Adv Mater, 2020, 32: 2002431

    Article  Google Scholar 

  108. Dang B, Liu K, Wu X, et al. One-phototransistor-one-memristor array with high-linearity light-tunable weight for optic neuromorphic computing. Adv Mater, 2023, 35

  109. Choi C, Leem J, Kim M, et al. Curved neuromorphic image sensor array using a MoS2-organic heterostructure inspired by the human visual recognition system. Nat Commun, 2020, 11: 5934

    Article  Google Scholar 

  110. Li Y, Wang J, Yang Q, et al. Flexible artificial optoelectronic synapse based on lead-free metal halide nanocrystals for neuromorphic computing and color recognition. Adv Sci, 2022, 9: 2202123

    Article  Google Scholar 

  111. Guo F, Song M, Wong M, et al. Multifunctional optoelectronic synapse based on ferroelectric van der Waals heterostructure for emulating the entire human visual system. Adv Funct Mater, 2022, 32: 2108014

    Article  Google Scholar 

  112. Hong S, Choi S H, Park J, et al. Sensory adaptation and neuromorphic phototransistors based on CsPb(Br1−xIx)3 perovskite and MoS2 hybrid structure. ACS Nano, 2020, 14: 9796–9806

    Article  Google Scholar 

  113. Xie D, Wei L, Xie M, et al. Photoelectric visual adaptation based on 0D-CsPbBr3-quantum-dots/2D-MoS2 mixed-dimensional heterojunction transistor. Adv Funct Mater, 2021, 31: 2010655

    Article  MathSciNet  Google Scholar 

  114. Xie D, Gao G, Tian B, et al. Porous metal-organic framework/ReS2 heterojunction phototransistor for polarization-sensitive visual adaptation emulation. Adv Mater, 2023, 35

  115. Ohta J O. Smart CMOS Image Sensors and Applications. 2nd ed. Boca Raton: CRC Press, 2017

    Book  Google Scholar 

  116. Chen Q, Zhang Y, Liu S, et al. Switchable perovskite photovoltaic sensors for bioinspired adaptive machine vision. Adv Intell Syst, 2020, 2: 2000122

    Article  Google Scholar 

  117. Liao C, Wang W, Sun Y, et al. A gate multiplexing architecture-based artificial visual sensor and memory system. Adv Intell Syst, 2023, 5: 2200298

    Article  Google Scholar 

  118. Jayachandran D, Oberoi A, Sebastian A, et al. A low-power biomimetic collision detector based on an in-memory molybdenum disulfide photodetector. Nat Electron, 2020, 3: 646–655

    Article  Google Scholar 

  119. Harrison R R. A biologically inspired analog IC for visual collision detection. IEEE Trans Circ Syst I, 2005, 52: 2308–2318

    Article  Google Scholar 

  120. Jayachandran D, Pannone A, Das M, et al. Insect-inspired, spike-based, in-sensor, and night-time collision detector based on atomically thin and light-sensitive memtransistors. ACS Nano, 2022, 17: 1068–1080

    Article  Google Scholar 

  121. Tanaka G, Yamane T, Héroux J B, et al. Recent advances in physical reservoir computing: a review. Neural Netw, 2019, 115: 100–123

    Article  Google Scholar 

  122. Du C, Cai F, Zidan M A, et al. Reservoir computing using dynamic memristors for temporal information processing. Nat Commun, 2017, 8: 2204

    Article  Google Scholar 

  123. Lao J, Yan M, Tian B, et al. Ultralow-power machine vision with self-powered sensor reservoir. Adv Sci, 2022, 9: 2106092

    Article  Google Scholar 

  124. Sun Y, Li Q, Zhu X, et al. In-sensor reservoir computing based on optoelectronic synapse. Adv Intell Syst, 2023, 5: 2200196

    Article  Google Scholar 

  125. Wan T Q, Ma S J, Liao F Y, et al. Neuromorphic sensory computing. Sci China Inf Sci, 2021, 65: 141401

    Article  Google Scholar 

  126. Tong L, Peng Z, Lin R, et al. 2D materials-based homogeneous transistor-memory architecture for neuromorphic hardware. Science, 2021, 373: 1353–1358

    Article  Google Scholar 

  127. Seo S, Lee J, Lee R, et al. An optogenetics-inspired flexible van der Waals optoelectronic synapse and its application to a convolutional neural network. Adv Mater, 2021, 33: 2102980

    Article  Google Scholar 

  128. Pan X, Shi J, Wang P, et al. Parallel perception of visual motion using light-tunable memory matrix. Sci Adv, 2023, 9: 4083

    Article  Google Scholar 

  129. Guo M H, Liu Z N, Mu T J, et al. Beyond self-attention: external attention using two linear layers for visual tasks. IEEE Trans Pattern Anal Mach Intell, 2023, 45: 5436–5447

    Google Scholar 

  130. Qian R, Lai X, Li X. 3D object detection for autonomous driving: a survey. Pattern Recogn, 2022, 130: 108796

    Article  Google Scholar 

  131. Laga H, Jospin L V, Boussaid F, et al. A survey on deep learning techniques for stereo-based depth estimation. IEEE Trans Pattern Anal Mach Intell, 2022, 44: 1738–1764

    Article  Google Scholar 

  132. Chen C, He Y, Mao H, et al. A photoelectric spiking neuron for visual depth perception. Adv Mater, 2022, 34: 2201895

    Article  Google Scholar 

  133. Su F, Chen W H, Xia L X, et al. A 462GOPs/J RRAM-based nonvolatile intelligent processor for energy harvesting IoE system featuring nonvolatile logics and processing-in-memory. In: Proceedings of Symposium on VLSI Circuits, Horikawa-Shiokoji, 2017

  134. Huang X, Liu C, Tang Z, et al. An ultrafast bipolar flash memory for self-activated in-memory computing. Nat Nanotechnol, 2023, 18: 486–492

    Article  Google Scholar 

  135. Ren L, Zhou C, Song X, et al. Efficient spin-orbit torque switching in a perpendicularly magnetized heusler alloy MnPtGe single layer. ACS Nano, 2023, 17: 6400–6409

    Article  Google Scholar 

  136. Lan X, Cao Y, Liu X, et al. Gradient descent on multilevel spin-orbit synapses with tunable variations. Adv Intell Syst, 2021, 3: 2000182

    Article  Google Scholar 

  137. Yang S, Shin J, Kim T, et al. Integrated neuromorphic computing networks by artificial spin synapses and spin neurons. NPG Asia Mater, 2021, 13: 11

    Article  Google Scholar 

  138. Liu J, Xu T, Feng H, et al. Compensated ferrimagnet based artificial synapse and neuron for ultrafast neuromorphic computing. Adv Funct Mater, 2022, 32: 2107870

    Article  Google Scholar 

  139. Gauthier D J, Bollt E, Griffith A, et al. Next generation reservoir computing. Nat Commun, 2021, 12: 5564

    Article  Google Scholar 

  140. Sun L, Wang Z, Jiang J, et al. In-sensor reservoir computing for language learning via two-dimensional memristors. Sci Adv, 2021, 7: eabg1455

    Article  Google Scholar 

  141. Chen Z, Li W, Fan Z, et al. All-ferroelectric implementation of reservoir computing. Nat Commun, 2023, 14: 3585

    Article  Google Scholar 

  142. Usami Y, van de Ven B, Mathew D G, et al. In-materio reservoir computing in a sulfonated polyaniline network. Adv Mater, 2021, 33: 2102688

    Article  Google Scholar 

  143. Han J, Yun S, Lee S, et al. A review of artificial spiking neuron devices for neural processing and sensing. Adv Funct Mater, 2022, 32: 2204102

    Article  Google Scholar 

  144. Li X, Zhong Y, Chen H, et al. A memristors-based dendritic neuron for high-efficiency spatial-temporal information processing. Adv Mater, 2023, 35

  145. Han C Y, Fang S L, Cui Y L, et al. Configurable NbOx memristors as artificial synapses or neurons achieved by regulating the forming compliance current for the spiking neural network. Adv Elect Mater, 2023, 9: 2300018

    Article  Google Scholar 

  146. Han S, Mao H Z, Dally W J. Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding. In: Proceedings of International Conference on Learning Representations, Vancouver, 2016

  147. Yang J, Zhang F, Xiao H M, et al. A perovskite memristor with large dynamic space for analog-encoded image recognition. ACS Nano, 2022, 16: 21324–21333

    Article  Google Scholar 

  148. Zhu C Z, Han S, Mao H, et al. Trained ternary quantization. In: Proceedings of International Conference on Learning Representations, Toulouse, 2017

  149. Courbariaux M, Bengio Y, David J P. BinaryConnect: training deep neural networks with binary weights during propagations. In: Proceedings of Advances in Neural Information Processing Systems, Montreal, 2015

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant Nos. 52103311, 62075240), Scientific Researches Foundation of National University of Defense Technology (Grant No. ZK18-01-03), and National Key Research and Development Program of China (Grant No. 2020YFB2205804).

Author information

Authors and Affiliations

  1. Institute for Quantum Information and State Key Laboratory of High Performance Computing, College of Computer, National University of Defense Technology, Changsha, 410073, China

    Yujie Huang, Yabo Chen & Yuhua Tang

  2. College of Advanced Interdisciplinary Studies, National University of Defense Technology, Changsha, 410073, China

    Yinlong Tan & Yan Kang

  3. Institute for Quantum Science and Technology, College of Science, National University of Defense Technology, Changsha, 410073, China

    Tian Jiang

Authors
  1. Yujie Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yinlong Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yabo Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuhua Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tian Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yinlong Tan, Yuhua Tang or Tian Jiang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, Y., Tan, Y., Kang, Y. et al. Bioinspired sensing-memory-computing integrated vision systems: biomimetic mechanisms, design principles, and applications. Sci. China Inf. Sci. 67, 151401 (2024). https://doi.org/10.1007/s11432-023-3888-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-023-3888-0

Keywords

Search

Navigation