Region-Based Super Resolution for Video Sequences Considering Registration Error

  • Osama A. Omer
  • Toshihisa Tanaka
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5414)


Super-resolution (SR) for video sequences is a technique to obtain a higher resolution image by fusing multiple low-resolution (LR) frames of the same scene. In a typical super-resolution algorithm, image registration is one of the most affective steps. The difficulty of this step results in the fact that most of the existing SR algorithms can not cope with local motions because they assume global motion. In this paper, we propose a SR algorithm that takes into account inaccurate estimates of the registration parameters and the point spread function. When frames obey the assumed global motion model, these inaccurate estimates, along with the additive Gaussian noise in the low-resolution image sequence, result in different noise level for each frame. However, in case of existence of local motion and/or occlusion, regions that have local motion and/or occlusion have different noise level. To cope with this problem, we propose to adaptively weight each segment according to its reliability. The segments are generated by segmenting the reference frame using watershed segmentation. The experimental results using real video sequences show the effectiveness of the proposed algorithm compared to three state-of-the-art SR algorithms.


Super-resolution affine model image registration resolution enhancement region-based global weight 


  1. 1.
    Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R.: Hierarchical model-based motion estimation. In: Sandini, G. (ed.) ECCV 1992. LNCS, vol. 588, pp. 237–252. Springer, Heidelberg (1992)CrossRefGoogle Scholar
  2. 2.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the International Joint Conference on Artificial Intelligence (1981)Google Scholar
  3. 3.
    Elad, M., Hel-Or, Y.: A fast super-resolution reconstruction algorithm for pure transnational motion and common space invariant blur. IEEE Trans. on Image Processing 10(8), 1187–1193 (2001)CrossRefzbMATHGoogle Scholar
  4. 4.
    Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Fast and robust multi-frame super-resolution. IEEE Trans. on Image Processing 13(10), 1327–1344 (2004)CrossRefGoogle Scholar
  5. 5.
    Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Robust shift-and-add approach to super-resolution. In: Proc. of the 2003 SPIE Conf. on Applications of Digital Signal and Image Processing, San Diego, California (August 2003)Google Scholar
  6. 6.
    Lee, E.S., Kang, M.G.: Regularized adaptive high-resolution image reconstruction considering inaccurate subpixel registration. IEEE Trans. on Image Processing 12(7) (July 2003)Google Scholar
  7. 7.
    He, H., Kondi, L.P.: An image super-resolution algorithm for different error levels per frame. IEEE Trans. on Image Processing 15(3), 592–603 (2006)CrossRefGoogle Scholar
  8. 8.
    Park, M.K., Kang, M.G., Katsaggelos, A.K.: Regularized Super-Resolution Image Reconstruction Considering Inaccurate Motion Information. SPIE Optical Engineering 46(11), 117004-1–117004-12 (2007)Google Scholar
  9. 9.
    Trimeche, M., Ciprian Bilcu, R., Yrjanainen, J.: Adaptive outlier rejection in image super-resolution. EURASIP Journal on Applied Signal Processing 2006, Article ID 38052 (2006)Google Scholar
  10. 10.
    Omer, O.A., Tanaka, T.: Multiframe image and video super-resolution algorithm with inaccurate motion registration errors rejection. In: Proc. of the 2008 SPIE Conf. on Visual Communication and Image Processing, San Jose, California (January 2008)Google Scholar
  11. 11.
    Ivanovski, Z.A., Panovski, L., Karam, L.J.: Robust super-resolution based on pixel-level selectivity. In: Proceedings of SPIE, vol. 6077 (2006)Google Scholar
  12. 12.
    Schultz, R.R., Stevenson, R.t.L.: Extraction of high-resolution frames from video sequences. IEEE Trans. on Image Processing 5(6) (June 1996)Google Scholar
  13. 13.
    Zhao, W.Y., Sawhney, S.: Is super-resolution with optical flow feasible? In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 599–613. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  14. 14.
    Andrew, J., Patti, M.I.: Robust methods for high-quality stills from interlaced video in the presence of dominant motion. IEEE Trans. on Circuits and Systems for Video Technology 7(2) (April 1997)Google Scholar
  15. 15.
    Choi, B., Ra, J.B.: Region-based super-resolution using multiple blurred and noisy undersampled images. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 2, pp. 609–612 (2006)Google Scholar
  16. 16.
    Choi, B., Kim, S.D., Ra, J.B.: Region-based super-resolution using adaptive diffusion regularization. Optical Engineering 47(2), 027006 (February 2008)CrossRefGoogle Scholar
  17. 17.
    Qiao, J., Liu, J.: HOS-based image super-resolution reconstruction. In: Sebe, N., Liu, Y., Zhuang, Y.-t., Huang, T.S. (eds.) MCAM 2007. LNCS, vol. 4577, pp. 213–222. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  18. 18.
    van Eekeren, A., Schutte, K., Dijk, J., de Lange, D.J.J., van Vliet, L.J.: Super-resolution on moving objects and background. In: Proc. Int. Conf. Image Processing (ICIP 2006), vol. 2, pp. 2709–2712 (2006)Google Scholar
  19. 19.
    Hardie, R.C., Barnard, K.J., Bognar, J.G., Armstrong, E., Watson, E.A.: High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Optical Engineering 37(1), 247–260 (1998)CrossRefGoogle Scholar
  20. 20.
    De Smet, P., De Vleschauwer, D.: Performance and scalability of highly optimized rainfalling watershed algorithm. In: Proc. Int. Conf. on Imaging Science, Systems and Technology, CISST 1998, Las Vegas, NV, USA, pp. 266–273 (July 1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Osama A. Omer
    • 1
  • Toshihisa Tanaka
    • 1
  1. 1.Department of Electrical and Electronic EngineeringTokyo University of Agriculture and TechnologyTokyoJapan

Personalised recommendations