Spatio-temporal Super-Resolution Using Depth Map

  • Yusaku Awatsu
  • Norihiko Kawai
  • Tomokazu Sato
  • Naokazu Yokoya
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5575)


This paper describes a spatio-temporal super-resolution method using depth maps for static scenes. In the proposed method, the depth maps are used as the parameters to determine the corresponding pixels in multiple input images by assuming that intrinsic and extrinsic camera parameters are known. Because the proposed method can determine the corresponding pixels in multiple images by a one-dimensional search for the depth values without the planar assumption that is often used in the literature, spatial resolution can be increased even for complex scenes. In addition, since we can use multiple frames, temporal resolution can be increased even when large parts of the image are occluded in the adjacent frame. In experiments, the validity of the proposed method is demonstrated by generating spatio-temporal super-resolution images for both synthetic and real movies.


Super-resolution Depth map View interpolation 


  1. 1.
    Ikeda, S., Sato, T., Yokoya, N.: Panoramic Movie Generation Using an Omnidirectional Multi-camera System for Telepresence. In: Proc. Scandinavian Conf. on Image Analysis, pp. 1074–1081 (2003)Google Scholar
  2. 2.
    Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based Super-Resolution. IEEE Computer Graphics and Applications 22, 56–65 (2002)CrossRefGoogle Scholar
  3. 3.
    Hong, M.C., Stathaki, T., Katsaggelos, A.K.: Iterative Regularized Image Restoration Using Local Constraints. In: Proc. IEEE Workshop on Nonlinear Signal and Image Processing, pp. 145–148 (1997)Google Scholar
  4. 4.
    Zhao, W.Y.: Super-Resolving Compressed Video with Large Artifacts. In: Proc. Int. Conf. on Pattern Recognition, vol. 1, pp. 516–519 (2004)Google Scholar
  5. 5.
    Chiang, M.C., Boult, T.E.: Efficient Super-Resolution via Image Warping. Image and Vision Computing, 761–771 (2000)Google Scholar
  6. 6.
    Ben-Ezra, M., Zomet, A., Nayar, S.K.: Jitter Camera: High Resolution Video from a Low Resolution Detector. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 135–142 (2004)Google Scholar
  7. 7.
    Irani, M., Peleg, S.: Improving Resolution by Image Registration. Graphical Models and Image Processing 53(3), 231–239 (1991)CrossRefGoogle Scholar
  8. 8.
    Yamazaki, S., Ikeuchi, K., Shingawa, Y.: Determining Plausible Mapping Between Images Without a Priori Knowledge. In: Proc. Asian Conf. on Computer Vision, pp. 408–413 (2004)Google Scholar
  9. 9.
    Chen, S.E., William, L.: View Interpolation for Image Synthesis. In: Proc. Int. Conf. on Computer Graphics and Interactive Techniques, vol. 1, pp. 279–288 (1993)Google Scholar
  10. 10.
    Shechtman, E., Caspi, Y., Irani, M.: Space-Time Super-Resolution. IEEE Trans. on Pattern Analysis and Machine Intelligence 27(4), 531–545 (2005)CrossRefzbMATHGoogle Scholar
  11. 11.
    Imagawa, T., Azuma, T., Sato, T., Yokoya, N.: High-spatio-temporal-resolution image-sequence reconstruction from two image sequences with different resolutions and exposure times. In: ACCV 2007 Satellite Workshop on Multi-dimensional and Multi-view Image Processing, pp. 32–38 (2007)Google Scholar
  12. 12.
    Kimura, K., Nagai, T., Nagayoshi, H., Sako, H.: Simultaneous Estimation of Super-Resolved Image and 3D Information Using Multiple Stereo-Pair Images. In: IEEE Int. Conf. on Image Processing, vol. 5, pp. 417–420 (2007)Google Scholar
  13. 13.
    Sato, T., Kanbara, M., Yokoya, N., Takemura, H.: Camera parameter estimation from a long image sequence by tracking markers and natural features. Systems and Computers in Japan 35, 12–20 (2004)CrossRefGoogle Scholar
  14. 14.
    Sato, T., Yokoya, N.: New multi-baseline stereo by counting interest points. In: Proc. Canadian Conf. on Computer and Robot Vision, pp. 96–103 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Yusaku Awatsu
    • 1
  • Norihiko Kawai
    • 1
  • Tomokazu Sato
    • 1
  • Naokazu Yokoya
    • 1
  1. 1.Graduate School of Information ScienceNara Institute of Science and TechnologyNaraJapan

Personalised recommendations