Skip to main content
SpringerLink
Log in

Compressed High-Speed Imaging

  • Chapter
  • First Online:
Coded Optical Imaging

  • 526 Accesses

Abstract

Compressed high-speed imaging can capture a two-dimensional transient scene in a single image acquisition. It endows off-the-shelf CCD/CMOS cameras with at least one order of magnitude improvement in imaging speed. Combining optical engineering and image reconstruction techniques, compressed high-speed imaging consists of hardware encoding with customized system designs and software decoding with advanced algorithms. This rapidly evolving field has received increasing attention because of its attractive ability to cost-efficiently visualize transient events at their time of occurrence. This chapter surveys fundamental concepts, operating principles, and representative imaging modalities in compressed high-speed imaging. Hardware encoding is categorized into dynamic encoding and static encoding with temporal shearing. Software decoding is grouped into analytical-modeling-based approaches and deep-learning-based approaches. Finally, prospects in this field are provided for both technological advancement and potential applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 279.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Research, C.C.f.M. Thinking with the Eyes. Available from: https://www.ccmr.cornell.edu/wp-content/uploads/sites/2/2015/11/Thinking_with_the_eyes.pdf.

  2. Medicine, N.L.o. Webvision: The Organization of the Retina and Visual System [Internet]. Available from: https://www.ncbi.nlm.nih.gov/books/NBK11559/#:~:text=The%20eye%20appears%20to%20be,is%20 close%20to%2060%20Hz.

  3. Jiang, C., et al., High-speed dual-view band-limited illumination profilometry using temporally interlaced acquisition. Photonics Research, 2020. 8(11): p. 1808–1817.

    Google Scholar 

  4. Lazovsky, L., et al. CCD sensor and camera for 100 Mfps burst frame rate image capture. in Airborne Intelligence, Surveillance, Reconnaissance (ISR) Systems and Applications II. 2005. International Society for Optics and Photonics.

    Google Scholar 

  5. Suzuki, M., et al., A Preliminary Chip Evaluation toward Over 50Mfps Burst Global Shutter Stacked CMOS Image Sensor. Electronic Imaging, 2018. 2018(11): p. 398-1-398-4.

    Google Scholar 

  6. PetaPixel. Samsung Launches the First-Ever 200-Megapixel Smartphone Sensor. 2021; Available from: https://petapixel.com/2021/09/02/samsung-launches-the-first-ever-200-megapixel-smartphone-sensor/.

  7. World, I.S. Large Format 288MP Global Shutter Sensor. Available from: https://image-sensors-world.blogspot.com/2021/11/large-format-288mp-global-shutter-sensor.html.

  8. Etoh, T.G., et al., Toward one giga frames per second—evolution of in situ storage image sensors. Sensors, 2013. 13(4): p. 4640–4658.

    Google Scholar 

  9. Liang, J. and L.V. Wang, Single-shot ultrafast optical imaging. Optica, 2018. 5(9): p. 1113–1127.

    Google Scholar 

  10. Chin, C.T., et al., Brandaris 128: A digital 25 million frames per second camera with 128 highly sensitive frames. Review of Scientific Instruments, 2003. 74: p. 5026–5034.

    Google Scholar 

  11. Nakagawa, K., et al., Sequentially timed all-optical mapping photography (STAMP). Nature Photonics, 2014. 8(9): p. 695–700.

    Google Scholar 

  12. Sheinman, M., et al., Flatfield ultrafast imaging with single-shot non-synchronous array photography. Optics Letters, 2022. 47(3): p. 577–580.

    Google Scholar 

  13. Hebling, J., et al., Velocity matching by pulse front tilting for large-area THz-pulse generation. Optics Express, 2002. 10(21): p. 1161–1166.

    Google Scholar 

  14. Liu, X., et al., Single-shot compressed optical-streaking ultra-high-speed photography. Optics Letters, 2019. 44(6): p. 1387–1390.

    Google Scholar 

  15. Liu, X., et al., Single-shot real-time compressed ultrahigh-speed imaging enabled by a snapshot-to-video autoencoder. Photonics Research, 2021. 9(12): p. 2464–2474.

    Google Scholar 

  16. Liu, X., et al., Fast wide-field upconversion luminescence lifetime thermometry enabled by single-shot compressed ultrahigh-speed imaging. Nature Communications, 2021. 12(1): 6401.

    Google Scholar 

  17. Llull, P., et al., Coded aperture compressive temporal imaging. Optics Express, 2013. 21(9): p. 10526–10545.

    Google Scholar 

  18. Marquez, M., et al., Deep-Learning Supervised Snapshot Compressive Imaging Enabled by an End-to-End Adaptive Neural Network. IEEE Journal of Selected Topics in Signal Processing, 2022. 16(4): p. 688–699.

    Google Scholar 

  19. Park, J. and L. Gao, Continuously streaming compressed high-speed photography using time delay integration. Optica, 2021. 8(12): p. 1620–1623.

    Google Scholar 

  20. Reddy, D., A. Veeraraghavan, and R. Chellappa. P2C2: Programmable pixel compressive camera for high speed imaging. in CVPR 2011. 2011. IEEE.

    Google Scholar 

  21. Yuan, X., D.J. Brady, and A.K. Katsaggelos, Snapshot compressive imaging: Theory, algorithms, and applications. IEEE Signal Processing Magazine, 2021. 38(2): p. 65–88.

    Google Scholar 

  22. Liang, J., Punching holes in light: recent progress in single-shot coded-aperture optical imaging. Reports on Progress in Physics, 2020. 83(11): p. 116101.

    Google Scholar 

  23. Chen, Y., et al., Adaptive reconstruction for coded aperture temporal compressive imaging. Applied Optics, 2017. 56(17): p. 4940–4947.

    Google Scholar 

  24. Ma, X., et al., LED-based compressive spectral-temporal imaging. Optics Express, 2021. 29(7): p. 10698–10715.

    Google Scholar 

  25. Zhang, Z., et al., Ten-mega-pixel snapshot compressive imaging with a hybrid coded aperture. Photonics Research, 2021. 9(11): p. 2277–2287.

    Google Scholar 

  26. Koller, R., et al., High spatio-temporal resolution video with compressed sensing. Optics Express, 2015. 23(12): p. 15992–16007.

    Google Scholar 

  27. Qi, D., et al., Single-shot compressed ultrafast photography: a review. Advanced Photonics, 2020. 2(1): p. 014003.

    Google Scholar 

  28. Liang, J., L. Zhu, and L.V. Wang, Single-shot real-time femtosecond imaging of temporal focusing. Light: Science & Applications, 2018. 7(1): 42.

    Google Scholar 

  29. Liang, J., et al., Single-shot stereo-polarimetric compressed ultrafast photography for light-speed observation of high-dimensional optical transients with picosecond resolution. Nature Communications, 2020. 11(1): 5252.

    Google Scholar 

  30. Liang, J., et al., Single-shot real-time video recording of a photonic Mach cone induced by a scattered light pulse. Science Advances, 2017. 3(1): p. e1601814.

    Google Scholar 

  31. Lai, Y., et al., Single-Shot Ultraviolet Compressed Ultrafast Photography. Laser & Photonics Reviews, 2020. 14(10): p. 2000122.

    Google Scholar 

  32. Kim, T., et al., Picosecond-resolution phase-sensitive imaging of transparent objects in a single shot. Science Advances, 2020. 6(3): p. eaay6200.

    Google Scholar 

  33. Gao, L., et al., Single-shot compressed ultrafast photography at one hundred billion frames per second. Nature, 2014. 516(7529): p. 74–77.

    Google Scholar 

  34. Yang, C., et al., Compressed ultrafast photography by multi-encoding imaging. Laser Physics Letters, 2018. 15(11): p. 116202.

    Google Scholar 

  35. Liu, X., et al., Single-shot real-time sub-nanosecond electron imaging aided by compressed sensing: Analytical modeling and simulation. Micron, 2019. 117: p. 47–54.

    Google Scholar 

  36. Lai, Y., et al., Compressed ultrafast tomographic imaging by passive spatiotemporal projections. Optics Letters, 2021. 46(7): p. 1788–1791.

    Google Scholar 

  37. Bioucas-Dias, J.M. and M.A. Figueiredo, A new TwIST: Two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Transactions on Image Processing, 2007. 16(12): p. 2992–3004.

    Google Scholar 

  38. Duran, J., B. Coll, and C. Sbert, Chambolle’s projection algorithm for total variation denoising. Image Processing on Line, 2013. 3: p. 311–331.

    Google Scholar 

  39. Yang, C., et al., Improving the image reconstruction quality of compressed ultrafast photography via an augmented Lagrangian algorithm. Journal of Optics, 2019. 21(3): p. 035703.

    Google Scholar 

  40. Chan, S.H., Plug-and-play ADMM for Image Restoration: Fixed Point Convergence and Applications. 2016.

    Google Scholar 

  41. Iliadis, M., L. Spinoulas, and A.K. Katsaggelos, Deep fully-connected networks for video compressive sensing. Digital Signal Processing, 2018. 72: p. 9–18.

    Google Scholar 

  42. Qiao, M., et al., Deep learning for video compressive sensing. APL Photonics, 2020. 5(3): p. 030801.

    Google Scholar 

  43. Yang, C., et al., High-fidelity image reconstruction for compressed ultrafast photography via an augmented-Lagrangian and deep-learning hybrid algorithm. Photonics Research, 2021. 9(2): p. B30–B37.

    Google Scholar 

  44. Ronneberger, O., P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. in International Conference on Medical image computing and computer-assisted intervention. 2015. Springer.

    Google Scholar 

  45. Zhang, Z. and M. Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. in NIPS, Advances in Neural Information Processing Systems. 2018.

    Google Scholar 

  46. Jung, H., Organic molecule-based photothermal agents: an expanding photothermal therapy universe. Chemical Society Review, 2018. 47.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratory of Applied Computational Imaging, Centre Énergie, Matériaux, Télécommunication, Institut National de la Recherche Scientifique, Université du Québec, Varennes, QC, Canada

    Xianglei Liu & Jinyang Liang

Authors
  1. Xianglei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinyang Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jinyang Liang .

Editor information

Editors and Affiliations

  1. Laboratory of Applied Computational Imaging, Institut National de la Recherche Scientifique Université du Québec, Varennes, QC, Canada

    Jinyang Liang

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Liu, X., Liang, J. (2024). Compressed High-Speed Imaging. In: Liang, J. (eds) Coded Optical Imaging. Springer, Cham. https://doi.org/10.1007/978-3-031-39062-3_26

Download citation

Publish with us

Policies and ethics

Search

Navigation