Abstract
Normal video sequences contain substantial overlap between successive frames, and each region in the scene appears in multiple frames. The super-resolution process creates high-resolution pictures of regions that are sampled in multiple frames, having a higher spatial resolution than the original video frames.
In this chapter existing super resolution algorithms are analyzed in the framework of the solution of large sparse linear equations. It is shown that the gradient of the function which is minimized when solving these equations can be computed by means of image operations like warping, convolutions, etc. This analysis paves the way for new algorithms, by using known gradient-based optimization techniques. The gradient is computed efficiently in the image domain instead of multiplying large sparse matrices.
This framework allows versatile imaging conditions, including arbitrary motion models, camera blur models, etc. Prior knowledge can also be combined efficiently for obtaining a MAP super resolution solution.
As an example, super resolution is implemented with the conjugategradient method, and a considerable speedup in the convergence of the algorithm is achieved compared to other methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
M. Elad and A. Feuer, “Super-resolution reconstruction of continuous image sequence,” IEEE Trans. on Pattern Analysis and Machine Intelligence, September 1996.
V.N. Smelyanskiy, P. Cheeseman, D. Maluf, and R. Morris, “Bayesian super-resolved surface reconstruction from images,” in IEEE Conf. on Computer Vision and Pattern Recognition, 2000, pp. I:375–382.
J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in European Conf. on Computer Vision, 1992, pp. 237–252.
H.S. Sawhney and R. Kumar, “True multi-image alignment and its application to mosaicing and lens distortion correction,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 21, no. 3, pp. 245–243, March 1999.
L. Zelnik-Manor and M. Irani, “Multi-frame estimation of planar motion,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1105–1116, October 2000.
A. Papoulis, “Generalized sampling expansion,” IEEE Trans. Circuits Systems, vol. CAS-24, pp. 652–654, 1977.
D. Capel and A. Zisserman, “Super-resolution enhancement of text image sequences,” in Int. Conf. Pattern Recognition, 2000, pp. Vol I: 600–605.
M. Irani and S. Peleg, “Improving resolution by image registration,” Graphical Models and Image Processing, vol. 53, pp. 231–239, 1991.
S. Mann and R.W. Picard, “Virtual bellows: Constructing high quality stills from video,” in Int. Conf. on Image Processing, 1994.
H. Shekarforoush and R. Chellappa, “Data-driven multichannel superresolution with application to video sequences,” Journal of the Optical Society of America, vol. 16, no. 3, pp. 481–492, March 1999.
M. Elad and A. Feuer, “Super-resolution reconstruction of image sequences,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 21, no. 9, pp. 817–834, September 1999.
A.J. Patti, M.I. Sezan, and A.M. Tekalp, “Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Processing, vol. 6, no. 8, pp. 1064–1076, August 1997.
T.S. Huang and R.Y. Tsai, “Multi-frame image restoration and registration,” in Advances in Computer Vision and Image Processing, T.S. Huang, Ed., vol. 1, pp. 317–339. JAI Press Inc., 1984.
S. P. Kim, N. K. Bose, and H. M. Valenzuela, “Recursive reconstraction of high-resolution image from noisy undersampled frames,” IEEE Trans. Acoust., Speech and Sign. Proc., vol. 38, pp. 1013–1027, June 1990.
H. Ur and D. Gross, “Improved resolution from subpixel shifted pictures,” Graphical Models and Image Processing, vol. 54, no. 2, pp. 181–186, March 1992.
M. Irani and S. Peleg, “Motion analysis for image enhancement: Resolution, occlusion, and transparency,” Journal of Visual Communication and Image Representation, vol. 4, pp. 324–335, 1993.
D. Keren, S. Peleg, and R. Brada, “Image sequence enhancement using sub-pixel displacements,” in IEEE Conf. on Computer Vision and Pattern Recognition, 1988, pp. 742–746.
H. Stark and P. Oskoui, “High resolution image recovery from image-plane arrays, using convex projections,” J. Opt. Soc. Am. A, vol. 6, no. 11, pp. 1715–1726, 1989.
M. Berthod, H. Shekarforoush, M. Werman, and J. Zerubia, “Reconstruction of high resolution 3D visual information using subpixel camera displacements,” in IEEE Conf. on Computer Vision and Pattern Recognition, 1994, pp. 654–657.
R.R. Schultz and R.L. Stevenson, “Extraction of high-resolution frames from video sequences,” IEEE Trans. Image Processing, vol. 5, no. 6, pp. 996–1011, June 1996.
D. Capel and A. Zisserman, “Automated mosaicing with superresolution zoom,” in IEEE Conf. on Computer Vision and Pattern Recognition, June 1998, pp. 885–891.
C. T. Kelley, Iterative Methods for Linear and Nonlinear Equations, SIAM, Philadelphia, PA, 1995.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Kluwer Academic Publishers
About this chapter
Cite this chapter
Zomet, A., Peleg, S. (2002). Super-Resolution from Multiple Images Having Arbitrary Mutual Motion. In: Chaudhuri, S. (eds) Super-Resolution Imaging. The International Series in Engineering and Computer Science, vol 632. Springer, Boston, MA. https://doi.org/10.1007/0-306-47004-7_8
Download citation
DOI: https://doi.org/10.1007/0-306-47004-7_8
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-7923-7471-8
Online ISBN: 978-0-306-47004-2
eBook Packages: Springer Book Archive