Skip to main content
SpringerLink
Log in

Numerical simulations on optoelectronic deep neural network hardware based on self-referential holography

  • Special Section: Regular Paper
  • International Symposium on Imaging, Sensing, and Optical Memory (ISOM’ 22), Sapporo, Japan
  • Published:
Optical Review Aims and scope Submit manuscript

Abstract

We propose a novel optoelectronic deep neural network (OE-DNN) hardware called the self-referential holographic deep neural network (SR-HDNN). The SR-HDNN features a combination of an optical computing part utilizing a volume hologram and an electronic part connecting the optical elements virtually. Since the shape of a volume hologram, which is a 3-dimensional (3D) refractive index distribution in this case, can be changed by its recording conditions, it is expected to realize the flexible design of optical computing functions by coupling between specific nodes. In addition, the electronic part enables the construction of multi-layer networks without extending the optical system and enabling arbitrary signal processing, including nonlinear operations. By integrating flexible optical and electronic parts, the SR-HDNN consisting of both flexible optical and electronic parts has the potential to maximize the performance of OE-DNN. In this study, we numerically simulate image classification tasks to investigate the feasibility and potential of the SR-HDNN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

All data supporting the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Lin, X., Rivenson, Y., Yardimci, N.T., Veli, M., Luo, Y., Jarrahi, M., Ozcan, A.: All-optical machine learning using diffractive deep neural networks. Science 361(6406), 1004–1008 (2018). https://doi.org/10.1126/science.aat8084

    Article  ADS  MathSciNet  MATH  Google Scholar 

  2. Wu, Z., Zhou, M., Khoram, E., Liu, B., Yu, Z.: Neuromorphic metasurface. Photon. Res. 8(1), 46–50 (2020). https://doi.org/10.1364/PRJ.8.000046

    Article  Google Scholar 

  3. Chang, J., Sitzmann, V., Dun, X., Heidrich, W., Wetzstein, G.: Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification. Sci. Rep. 8(1), 12324 (2018). https://doi.org/10.1038/s41598-018-30619-y

    Article  ADS  Google Scholar 

  4. Miscuglio, M., Hu, Z., Li, S., George, J.K., Capanna, R., Dalir, H., Bardet, P.M., Gupta, P., Sorger, V.J.: Massively parallel amplitude-only Fourier neural network. Optica 7(12), 1812–1819 (2020). https://doi.org/10.1364/OPTICA.408659

    Article  ADS  Google Scholar 

  5. Shi, W., Huang, Z., Huang, H., Hu, C., Chen, M., Yang, S., Chen, H.: Loen: Lensless opto-electronic neural network empowered machine vision. Light: Sci. Appl. 11(1), 121 (2022). https://doi.org/10.1038/s41377-022-00809-5

    Article  ADS  Google Scholar 

  6. Sadeghzadeh, H., Koohi, S.: Translation-invariant optical neural network for image classification. Sci. Rep. 12(1), 17232 (2022). https://doi.org/10.1038/s41598-022-22291-0

    Article  ADS  Google Scholar 

  7. Rafayelyan, M., Dong, J., Tan, Y., Krzakala, F., Gigan, S.: Large-scale optical reservoir computing for spatiotemporal chaotic systems prediction. Phys. Rev. X 10, 041037 (2020). https://doi.org/10.1103/PhysRevX.10.041037

    Article  Google Scholar 

  8. Dou, H., Deng, Y., Yan, T., Wu, H., Lin, X., Dai, Q.: Residual D2NN: training diffractive deep neural networks via learnable light shortcuts. Opt. Lett. 45(10), 2688–2691 (2020). https://doi.org/10.1364/OL.389696

    Article  ADS  Google Scholar 

  9. Rahman, M.S.S., Li, J., Mengu, D., Rivenson, Y., Ozcan, A.: Ensemble learning of diffractive optical networks. Light: Sci. Appl. 10(1), 14 (2021). https://doi.org/10.1038/s41377-020-00446-w

    Article  ADS  Google Scholar 

  10. Gu, Z., Gao, Y., Liu, X.: Optronic convolutional neural networks of multi-layers with different functions executed in optics for image classification. Opt. Express 29(4), 5877–5889 (2021). https://doi.org/10.1364/OE.415542

    Article  ADS  Google Scholar 

  11. Zhou, T., Lin, X., Wu, J., Chen, Y., Xie, H., Li, Y., Fan, J., Wu, H., Fang, L., Dai, Q.: Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photonics 15(5), 367–373 (2021). https://doi.org/10.1038/s41566-021-00796-w

    Article  ADS  Google Scholar 

  12. Takabayashi, M., Okamoto, A.: Self-referential holography and its applications to data storage and phase-to-intensity conversion. Opt. Express 21(3), 3669–3681 (2013). https://doi.org/10.1364/OE.21.003669

    Article  ADS  Google Scholar 

  13. Popoff, S.M., Lerosey, G., Carminati, R., Fink, M., Boccara, A.C., Gigan, S.: Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media. Phys. Rev. Lett. 104, 100601 (2010). https://doi.org/10.1103/PhysRevLett.104.100601

    Article  ADS  Google Scholar 

  14. Popoff, S., Lerosey, G., Fink, M., Boccara, A.C., Gigan, S.: Image transmission through an opaque material. Nat. Commun. 1(1), 81 (2010). https://doi.org/10.1038/ncomms1078

    Article  ADS  Google Scholar 

  15. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  16. Hao, J., Lin, X., Chen, R., Lin, Y., Liu, H., Song, H., Lin, D., Tan, X.: Phase retrieval combined with the deep learning denoising method in holographic data storage. Opt. Continuum 1(1), 51–62 (2022). https://doi.org/10.1364/OPTCON.444882

    Article  Google Scholar 

  17. Katano, Y., Nobukawa, T., Muroi, T., Kinoshita, N., Ishii, N.: Efficient decoding method for holographic data storage combining convolutional neural network and spatially coupled low-density parity-check code. ITE Trans. Media Technol. Appl. 9(3), 161–168 (2021). https://doi.org/10.3169/mta.9.161

    Article  Google Scholar 

  18. Katano, Y., Muroi, T., Kinoshita, N., Ishii, N., Hayashi, N.: Data demodulation using convolutional neural networks for holographic data storage. Jpn. J. Appl. Phys. 57(9S1), 09SC01 (2018). https://doi.org/10.7567/JJAP.57.09SC01

    Article  Google Scholar 

  19. Shimobaba, T., Kuwata, N., Homma, M., Takahashi, T., Nagahama, Y., Sano, M., Hasegawa, S., Hirayama, R., Kakue, T., Shiraki, A., Takada, N., Ito, T.: Convolutional neural network-based data page classification for holographic memory. Appl. Opt. 56(26), 7327–7330 (2017). https://doi.org/10.1364/AO.56.007327

    Article  ADS  Google Scholar 

  20. Sakib Rahman, M.S., Ozcan, A.: Computer-free, all-optical reconstruction of holograms using diffractive networks. ACS Photonics 8(11), 3375–3384 (2021). https://doi.org/10.1021/acsphotonics.1c01365

    Article  Google Scholar 

  21. Li, Y., Luo, Y., Bai, B., Ozcan, A.: Analysis of diffractive neural networks for seeing through random diffusers. IEEE J. Select. Topics Quantum Electron. (2022). https://doi.org/10.1109/JSTQE.2022.3194574

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank Prof. Atsushi Shibukawa at Hokkaido University for the fruitful comments on the transmission matrix. We also acknowledge thank Editage (www.editage.com) for English language editing.

Author information

Authors and Affiliations

  1. Graduate School of Computer Science and Systems Engineering, Kyushu Institute of Technology, 680-4, Kawazu, Iizuka, Fukuoka, 820-8502, Japan

    Rio Tomioka

  2. Faculty of Computer Science and Systems Engineering, Kyushu Institute of Technology, 680-4 Kawazu, Iizuka, Fukuoka, 820-8502, Japan

    Masanori Takabayashi

  3. Research Center for Neuromorphic AI Hardware, Kyushu Institute of Technology, 2-4 Hibikino, Wakamatsu, Kitakyushu, 808-0196, Japan

    Masanori Takabayashi

Authors
  1. Rio Tomioka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Masanori Takabayashi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rio Tomioka.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tomioka, R., Takabayashi, M. Numerical simulations on optoelectronic deep neural network hardware based on self-referential holography. Opt Rev 30, 387–396 (2023). https://doi.org/10.1007/s10043-023-00810-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10043-023-00810-2

Keywords

Search

Navigation