Optical Review

, Volume 25, Issue 2, pp 215–226 | Cite as

Adaptive temporal compressive sensing for video with motion estimation

  • Yeru Wang
  • Chaoying Tang
  • Yueting Chen
  • Huajun Feng
  • Zhihai Xu
  • Qi Li
Regular Paper
First Online:
Received:
Accepted:
  • 105 Downloads

Abstract

In this paper, we present an adaptive reconstruction method for temporal compressive imaging with pixel-wise exposure. The motion of objects is first estimated from interpolated images with a designed coding mask. With the help of motion estimation, image blocks are classified according to the degree of motion and reconstructed with the corresponding dictionary, which was trained beforehand. Both the simulation and experiment results show that the proposed method can obtain accurate motion information before reconstruction and efficiently reconstruct compressive video.

Keywords

Computational imaging Compressive sensing Image reconstruction Motion estimation 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This work is supported by Fundamental Research Funds for the Central Universities and Space Innovation Fund Project, Jiangsu Science and Technology Program (BE2016119).

References

  1. 1.
    Hitomi, Y., Gu, J., Gupta, M., Mitsunaga, T., Nayar, S.K.: Video from a single coded exposure photograph using a learned over-complete dictionary. In: Proc. IEEE International Conference on Computer Vision: (ICCV), pp. 287–294, (2011)Google Scholar
  2. 2.
    Liu, D., Gu, J., Hitomi, Y., Gupta, M., Mitsunaga, T., Nayar, S.K.: Efficient space-time sampling with pixel-wise coded exposure for high-speed imaging.” IEEE Trans. Pattern Anal. Mach. Intell. 36, 248–260 (2014)CrossRefGoogle Scholar
  3. 3.
    Llull, P., Liao, X., Yuan, X., Yang, J., Kittle, D., Carin, L., Sapiro, G., Brady, D.J.: Coded aperture compressive temporal imaging.” Opt. Express. 21, 10526–10545 (2013)ADSCrossRefGoogle Scholar
  4. 4.
    Nagahara, H., Sonoda, T., Endo, K., Sugiyama, Y., Taniguchi, R.: High-speed imaging using CMOS image sensor with quasi pixel-wise exposure. In Proc. International Conference on Computational Photography, pp. 1–11 (2016)Google Scholar
  5. 5.
    Gao, L., Liang, J., Li, C., Wang, L.V.: Single-shot compressed ultrafast photography at one hundred billion frames per second.” Nature. 516, 74–77 (2014)ADSCrossRefGoogle Scholar
  6. 6.
    Tang, C., Chen, Y., Feng, H., Xu, Z., Li, Q.: Motion deblurring based on local temporal compressive sensing for remote sensing image.” Opt. Eng. 55, 093106 (2016)ADSCrossRefGoogle Scholar
  7. 7.
    Candes, E.J., Romberg, J.: Quantitative robust uncertainty principles and optimally sparse decompositions.” Found. Comput. Math. 6, 227–254 (2006)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Donoho, D.L.: Compressed sensing.” IEEE Trans. Inf. Theory. 52, 1289–1306 (2006)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Baraniuk, R.G., Goldstein, T., Sankaranarayanan, A.C., Studer, C., Veeraraghavan, A., Wakin, M.B.: Compressive video sensing: algorithms, architectures, and applications.” IEEE Signal Process. Mag. 34, 52–66 (2017)ADSCrossRefGoogle Scholar
  10. 10.
    Iliadis, M., Spinoulas, L., Katsaggelos, A.K., et al.: Deep fully-connected networks for video compressive sensing. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  11. 11.
    Chen, Y., Tang, C., Feng, H., Xu, Z., Li, Q.: Adaptive reconstruction for coded aperture temporal compressive imaging.” Appl. Opt. 56(17), 4940–4947 (2017)ADSCrossRefGoogle Scholar
  12. 12.
    Yuan, X., Yang, J., Llull, P., Liao, X., Sapiro, G., Brady, D.J., Carin, L.: Adaptive temporal compressive sensing for video. In: IEEE International Conference on Image Processing: (ICIP), pp. 14–18, (2013)Google Scholar
  13. 13.
    Yang, J., Liao, X., Yuan, X., Llull, P., Brady, D.J., Sapiro, G., Carin, L.: Compressive sensing by learning a Gaussian mixture model from measurements.” IEEE Trans. Image Process. 24, 106–119 (2015)ADSMathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Bioucas-Dias, J.M., Figueiredo, M.A.: A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration.” IEEE Trans. Image Process. 16, 2992–3004 (2007)ADSMathSciNetCrossRefGoogle Scholar
  15. 15.
    Tropp, J.A., Gilbert, A.C.: Signal recovery from random measurements via orthogonal matching pursuit.” IEEE Trans. Inf. Theory. 53, 4655–4666 (2007)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Aharon, M., Elad, M., Bruckstein, A.: “K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation.” IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)ADSCrossRefMATHGoogle Scholar
  17. 17.
    Reddy, D., Veeraraghavan, A., Chellappa, R.: P2C2: programmable pixel compressive camera for high speed imaging. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR),pp 329–336, (2011)Google Scholar
  18. 18.
    Carin, L., Liu, D., Guo, B.: “Coherence, compressive sensing, and random sensor arrays.” IEEE Trans. Antenn. Propag. 53, 28–39 (2011)ADSCrossRefGoogle Scholar
  19. 19.
    Duarte, M.F., Eldar, Y.C.: Structured compressed sensing: from theory to applications.” IEEE Trans. Signal Process. 59, 4053–4085 (2011)ADSMathSciNetCrossRefGoogle Scholar
  20. 20.
    Brox, T., et al.: High accuracy optical flow estimation based on a theory for warping. In: European Conference on Computer Vision, pp 25–36 ,(2004)Google Scholar

Copyright information

© The Optical Society of Japan 2018

Authors and Affiliations

  • Yeru Wang
    • 1
  • Chaoying Tang
    • 1
  • Yueting Chen
    • 1
  • Huajun Feng
    • 1
  • Zhihai Xu
    • 1
  • Qi Li
    • 1
  1. 1.State Key Lab of Modern Optical Instrumentation, College of Optical Science and EngineeringZhejiang UniversityHangzhouChina

Personalised recommendations