Advertisement

Is Super-Resolution with Optical Flow Feasible?

  • WenYi Zhao
  • Harpreet S. Sawhney
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2350)

Abstract

Reconstruction-based super-resolution from motion video has been an active area of study in computer vision and video analysis. Image alignment is a key component of super-resolution algorithms. Almost all previous super-resolution algorithms have assumed that standard methods of image alignment can provide accurate enough alignment for creating super-resolution images. However, a systematic study of the demands on accuracy of multi-image alignment and its effects on super-resolution has been lacking. Furthermore, implicitly or explicitly most algorithms have assumed that the multiple video frames or specific regions of interest are related through global parametric transformations. From previous works, it is not at all clear how super-resolution performs under alignment with piecewise parametric or local optical flow based methods. This paper is an attempt at understanding the influence of image alignment and warping errors on super-resolution. Requirements on the consistency of optical flow across multiple images are studied and it is shown that errors resulting from traditional flow algorithms may render super-resolution infeasible.

Keywords

Optical Flow Motion Estimation Motion Error Image Alignment Reprojection Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Irani, M. and Peleg, S. 1993. Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency. Journal of Visual Comm. and Image Repre., Vol. 4, pp. 324–335.CrossRefGoogle Scholar
  2. [2]
    Elad, M. and Feuer, A. 1997. Restoration of a single superresolution image form several blurred, noisy and undersampled measured images. IEEE Trans. on Image Processing, pp. 1646–1658.Google Scholar
  3. [3]
    Patti, A., Sezan, M., and Tekalp, M. 1997. Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Trans. on Image Processing, pp. 1064–1078.Google Scholar
  4. [4]
    Bascle, B., Blake, A., and Zisserman, A. 1996. Motion deblurring and super-resolution from an image sequence. In Proc. European Conf. Comp. Vision, pp 573–581.Google Scholar
  5. [5]
    Schultz, R.R. and Stevenson, R.L. 1996. Extraction of high resolution frames from video sequences. IEEE Trans. on Image Processing, pp. 996–1011.Google Scholar
  6. [6]
    Hardie, R., Barrard, K., and Armstrong E. 1997. Joint MAP Registration and High-resolution Image Estimation Using a Sequence of Undersampled Images. IEEE Trans. on Image Processing, pp. 1621–1633. Multi-resolutionGoogle Scholar
  7. [7]
    Marziliano, P. and Vetterlli, M. 1999. Reconstruction of Irregularly Sampled Discrete-Time Bandlimited Signals. IEEE Trans. on Signal Processing, pp. 3462–3471.Google Scholar
  8. [8]
    Shekarforoush, H. and Chellappa, R. 1999. Data-driven multi-channel super-resolution with application to video sequences. Journal of the Optical Society of America A, pp. 481–492.Google Scholar
  9. [9]
    Tsai, R.Y. and Huang, T.S. 1984. Multi-frame Image Restoration and Registration. Advances in Computer Vision and Image Processing, JAI Press Inc.Google Scholar
  10. [10]
    Baker, S. and Kanade, T. 1999. Super-Resolution Optical Flow. CMU-RI-TR-9936.Google Scholar
  11. [11]
    Baker, S. and Kanade, T. 2000. Limits on Super-Resolution and How to Break Them. In Proc. Conf. Comp. Vision and Patt. Recog. Google Scholar
  12. [12]
    Freeman, W. and Pasztor, E. 1999. Learning low-level vision. In Proc. Int. Conf. Comp. Vision.Google Scholar
  13. [13]
    Bonet, J.S.D. 1997. sampling procedure for analysis and synthesis of texture images. In Proceedings of SIGGRAPH, pp. 361–368.Google Scholar
  14. [14]
    B. D. Lucas and T. Kanade. 1981. An iterative image registration technique with an application to stereo vision. In Proc. 7th Int. Joint Conf. on Art. Intell..Google Scholar
  15. [15]
    Bergen, J., Anandan, P., Hanna, K., and Hingorani, R. 1992. Hierarchical Model-Based Motion Estimation. In Proc. European Conf. Comp. Vision, pp. 237–252.Google Scholar
  16. [16]
    Simoncelli, E.P. and Adelson, E.H. 1990. Computing Optical Flow Distribution Using Spatio-Temporal Filters. MIT Media Lab Technical Report 165.Google Scholar
  17. [17]
    Sha, N. R. and Zakhor, A. 1999. Resolution Enhancement of Color Video Sequences. In IEEE Trans. on IP, Vol. 8, No. 6, June, 1999, pp. 879–885.Google Scholar
  18. [18]
    Oppenheim, A. V. and Schafer, R.W. Discrete-Time Signal Processing. Prentice Hall, Englewood Cliffs, NJ, USA, 1989.zbMATHGoogle Scholar
  19. [19]
    Birchfield, S. Derivation of Kanade-Lucas-Tomasi Tracking Equation. Unpublished, May 1996. http://vision.stanford.edu/birch/klt/.

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • WenYi Zhao
    • 1
  • Harpreet S. Sawhney
    • 1
  1. 1.Sarnoff CorporationPrinceton

Personalised recommendations