Advertisement

Experiments in Fluids

, 60:170 | Cite as

Deep learning-based accurate and rapid tracking of 3D positional information of microparticles using digital holographic microscopy

  • Sang Joon LeeEmail author
  • Gun Young Yoon
  • Taesik Go
Research Article
  • 95 Downloads

Abstract

Digital holographic microscopy (DHM) encrypts three-dimensional (3D) volumetric information of a test sample in the form of diffraction patterns. Computationally intensive numerical reconstruction and autofocusing procedures, which require precise determination of some parameters, are essential for the extraction of 3D positional information. In this study, a method that can satisfy these requirements by combining digital in-line holographic microscopy (DIHM) with deep learning was proposed. Deep neural networks were trained with tens of thousands of defocused holograms of microparticles moving in a microtube. The in-plane positions of the spherical microparticles were detected by applying Segnet and circular Hough transform, and the depth positions of the microparticles were estimated using a convolutional neural network. The performance of the proposed method was verified by conducting a planar surface experiment and tracking the 3D motions of the particles in a circular microtube. Results indicated that the trained neural networks precisely determined the 3D positions and 3D trajectories of the particles, and the proposed method outperformed conventional methods in measuring 3D positional information in the in-plane and out-of-plane direction. The process time of the proposed method for 3D tracking was only 1.92% of that of the conventional method. The present deep learning-based DIHM method can facilitate the analysis of the 3D dynamic motions of particles or cells and 3D tracking of numerous samples in motion.

Graphic abstract

Notes

Acknowledgements

This work was supported by the National Research Foundation of Korea (NRF) (2019M3C1B7025088) and the Rural Development Administration of Korea (RDA) (PJ014352052019) grant funded by the Korean government.

Compliance with ethical standards

Conflict of interest

There are no conflicts to declare.

References

  1. Badrinarayanan V, Kendall A, Cipolla R (2015) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39:2481–2495CrossRefGoogle Scholar
  2. Baek S, Lee S (1996) A new two-frame particle tracking algorithm using match probability. Exp Fluids 22:23–32CrossRefGoogle Scholar
  3. Byeon H, Go T, Lee SJ (2019) Deep learning-based digital in-line holographic microscopy for high resolution with extended field of view. Opt Laser Technol 113:77–86CrossRefGoogle Scholar
  4. Cheong FC, Dreyfus BSR, Amato-Grill J, Xiao K, Dixon L, Grier DG (2009) Flow visualization and flow cytometry with holographic video microscopy. Opt Express 17:13071–13079CrossRefGoogle Scholar
  5. Choi YS, Lee SJ (2009) Three-dimensional volumetric measurement of red blood cell motion using digital holographic microscopy. Appl Opt 48:2983–2990CrossRefGoogle Scholar
  6. Choi YS, Seo KW, Sohn MH, Lee SJ (2012) Advances in digital holographic micro-PTV for analyzing microscale flows. Opt Lasers Eng 50:39–45CrossRefGoogle Scholar
  7. Coupland JM, Lobera J (2008) Holography, tomography and 3D microscopy as linear filtering operations. Meas Sci Technol 19:074012CrossRefGoogle Scholar
  8. Daloglu MU et al (2018) Label-free 3D computational imaging of spermatozoon locomotion, head spin and flagellum beating over a large volume. Light Sci Appl 7:17121CrossRefGoogle Scholar
  9. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38:295–307CrossRefGoogle Scholar
  10. Go T, Byeon H, Lee SJ (2017) Focusing and alignment of erythrocytes in a viscoelastic medium. Sci Rep 7:41162CrossRefGoogle Scholar
  11. Go T, Byeon H, Lee SJ (2018a) Label-free sensor for automatic identification of erythrocytes using digital in-line holographic microscopy and machine learning. Biosens Bioelectron 103:12–18CrossRefGoogle Scholar
  12. Go T, Kim JH, Byeon H, Lee SJ (2018b) Machine learning-based in-line holographic sensing of unstained malaria-infected red blood cells. J Biophotonics 11:e201800101CrossRefGoogle Scholar
  13. Horisaki R, Takagi R, Tanida J (2018) Deep-learning-generated holography. Appl Opt 57:3859–3863CrossRefGoogle Scholar
  14. Jo Y et al (2017) Holographic deep learning for rapid optical screening of anthrax spores. Sci Adv 3:e1700606CrossRefGoogle Scholar
  15. Jo Y, Cho H, Lee SY, Choi G, Kim G, Min H, Park Y (2019) Quantitative phase imaging and artificial intelligence: a review. IEEE J Sel Top Quantum Electron 25:1–14CrossRefGoogle Scholar
  16. Katz J, Sheng J (2010) Applications of holography in fluid mechanics and particle dynamics. Annu Rev Fluid Mech 42:531–555CrossRefGoogle Scholar
  17. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems 25, pp 1097–1105Google Scholar
  18. LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1:541–551CrossRefGoogle Scholar
  19. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436CrossRefGoogle Scholar
  20. Lee SJ, Seo KW, Choi YS, Sohn MH (2011) Three-dimensional motion measurements of free-swimming microorganisms using digital holographic microscopy. Meas Sci Technol 22:064004CrossRefGoogle Scholar
  21. Lee SJ, Go T, Byeon H (2016) Three-dimensional swimming motility of microorganism in the near-wall region. Exp Fluids 57:26CrossRefGoogle Scholar
  22. Memmolo P, Miccio L, Paturzo M, Di Caprio G, Coppola G, Netti PA, Ferraro P (2015) Recent advances in holographic 3D particle tracking. Adv Opt Photonics 7:713–755CrossRefGoogle Scholar
  23. Mirsky SK, Barnea I, Levi M, Greenspan H, Shaked NT (2017) Automated analysis of individual sperm cells using stain-free interferometric phase microscopy and machine learning. Cytometry Part A 91:893–900CrossRefGoogle Scholar
  24. Molaei M, Barry M, Stocker R, Sheng J (2014) Failed escape: solid surfaces prevent tumbling of Escherichia coli. Phys Rev Lett 113:068103CrossRefGoogle Scholar
  25. Mudanyali O, Oztoprak C, Tseng D, Erlinger A, Ozcan A (2010) Detection of waterborne parasites using field-portable and cost-effective lensfree microscopy. Lab Chip 10:2419–2423CrossRefGoogle Scholar
  26. Murata S, Yasuda N (2000) Potential of digital holography in particle measurement. Opt Laser Technol 32(7-8):567–574CrossRefGoogle Scholar
  27. Nguyen T, Bui V, Lam V, Raub CB, Chang LC, Nehmetallah G (2017) Automatic phase aberration compensation for digital holographic microscopy based on deep learning background detection. Opt Express 25:15043–15057CrossRefGoogle Scholar
  28. O’Connor T, Rawat S, Markman A, Javidi B (2018) Automatic cell identification and visualization using digital holographic microscopy with head mounted augmented reality devices. Appl Opt 57:B197–B204CrossRefGoogle Scholar
  29. Pan G, Meng H (2003) Digital holography of particle fields: reconstruction by use of complex amplitude. Appl Opt 42(5):827–833CrossRefGoogle Scholar
  30. Park HS, Rinehart MT, Walzer KA, Chi JTA, Wax A (2016) Automated detection of P. falciparum using machine learning algorithms with quantitative phase images of unstained cells. PLoS ONE 11:e0163045CrossRefGoogle Scholar
  31. Park Y, Depeursinge C, Popescu G (2018) Quantitative phase imaging in biomedicine. Nat Photonics 12:578–589CrossRefGoogle Scholar
  32. Pitkäaho T, Manninen A, Naughton TJ (2019) Focus prediction in digital holographic microscopy using deep convolutional neural networks. Appl Opt 58:A202–A208CrossRefGoogle Scholar
  33. Ren Z, Xu Z, Lam EY (2018a) Autofocusing in digital holography using deep learning. SPIE, Proc, p 104991VGoogle Scholar
  34. Ren Z, Xu Z, Lam EY (2018b) Learning-based nonparametric autofocusing for digital holography. Optica 5:337–344CrossRefGoogle Scholar
  35. Rivenson Y, Göröcs Z, Günaydin H, Zhang Y, Wang H, Ozcan A (2017) Deep learning microscopy. Optica 4:1437–1443CrossRefGoogle Scholar
  36. Rivenson Y, Zhang Y, Günaydın H, Teng D, Ozcan A (2018) Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci Appl 7:17141CrossRefGoogle Scholar
  37. Roitshtain D, Wolbromsky L, Bal E, Greenspan H, Satterwhite LL, Shaked NT (2017) Quantitative phase microscopy spatial signatures of cancer cells. Cytometry Part A 91:482–493CrossRefGoogle Scholar
  38. Seo KW, Lee SJ (2014) High-accuracy measurement of depth-displacement using a focus function and its cross-correlation in holographic PTV. Opt Express 22:15542–15553CrossRefGoogle Scholar
  39. Shen D, Wu G, Suk H-I (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221–248CrossRefGoogle Scholar
  40. Sheng J, Malkiel E, Katz J, Adolf J, Belas R, Place AR (2007) Digital holographic microscopy reveals prey-induced changes in swimming behavior of predatory dinoflagellates. Proc Natl Acad Sci USA 104:17512–17517CrossRefGoogle Scholar
  41. Singh DK, Panigrahi PK (2010) Improved digital holographic reconstruction algorithm for depth error reduction and elimination of out-of-focus particles. Opt Express 18(3):2426–2448CrossRefGoogle Scholar
  42. Singh DK, Panigrahi PK (2015) Three-dimensional investigation of liquid slug Taylor flow inside a micro-capillary using holographic velocimetry. Exp Fluids 56:6CrossRefGoogle Scholar
  43. Sinha A, Lee J, Li S, Barbastathis G (2017) Lensless computational imaging through deep learning. Optica 4:1117–1125CrossRefGoogle Scholar
  44. Vedaldi A, Lenc K (2015) Matconvnet: convolutional neural networks for matlab. In: Proceeding of the ACM international conference on multimedia, pp 689–692Google Scholar
  45. Wu YC et al (2017) Air quality monitoring using mobile microscopy and machine learning. Light Sci Appl 6:e17046CrossRefGoogle Scholar
  46. Wu Y, Rivenson Y, Zhang Y, Wei Z, Günaydin H, Lin X, Ozcan A (2018) Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery. Optica 5:704–710CrossRefGoogle Scholar
  47. Yi F, Moon I, Javidi B (2016) Cell morphology-based classification of red blood cells using holographic imaging informatics. Biomed Opt Express 7:2385–2399CrossRefGoogle Scholar
  48. Yi F, Moon I, Javidi B (2017) Automated red blood cells extraction from holographic images using fully convolutional neural networks. Biomed Opt Express 8:4466–4479CrossRefGoogle Scholar
  49. Yoon J, Jo Y, Kim M, Kim K, Lee S, Kang SJ, Park Y (2017) Identification of non-activated lymphocytes using three-dimensional refractive index tomography and machine learning. Sci Rep 7:6654CrossRefGoogle Scholar
  50. Yu L, Kim MK (2005) Wavelength-scanning digital interference holography for tomographic three-dimensional imaging by use of the angular spectrum method. Opt Lett 30:2092–2094CrossRefGoogle Scholar
  51. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017a) Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans Image Process 26:3142–3155MathSciNetzbMATHCrossRefGoogle Scholar
  52. Zhang Y, Wang H, Wu Y, Tamamitsu M, Ozcan A (2017b) Edge sparsity criterion for robust holographic autofocusing. Opt Lett 42:3824–3827CrossRefGoogle Scholar
  53. Zhang G et al (2018) Fast phase retrieval in off-axis digital holographic microscopy through deep learning. Opt Express 26:19388–19405CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Mechanical EngineeringPohang University of Science and TechnologyPohangRepublic of Korea

Personalised recommendations