Direct Energy Minimization for Super-Resolution on Nonlinear Manifolds

  • Tien-Lung Chang
  • Tyng-Luh Liu
  • Jen-Hui Chuang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3954)


We address the problem of single image super-resolution by exploring the manifold properties. Given a set of low resolution image patches and their corresponding high resolution patches, we assume they respectively reside on two non-linear manifolds that have similar locally-linear structure. This manifold correlation can be realized by a three-layer Markov network that connects performing super-resolution with energy minimization. The main advantage of our approach is that by working directly with the network model, there is no need to actually construct the mappings for the underlying manifolds. To achieve such efficiency, we establish an energy minimization model for the network that directly accounts for the expected property entailed by the manifold assumption. The resulting energy function has two nice properties for super-resolution. First, the function is convex so that the optimization can be efficiently done. Second, it can be shown to be an upper bound of the reconstruction error by our algorithm. Thus, minimizing the energy function automatically guarantees a lower reconstruction error— an important characteristic for promising stable super-resolution results.


Reconstruction Error Image Patch Locally Linear Embedding Markov Network Nonlinear Manifold 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baker, S., Kanade, T.: Limits on Super-Resolution and How to Break Them. IEEE Trans. Pattern Analysis and Machine Intelligence 24(9), 1167–1183 (2002)CrossRefGoogle Scholar
  2. 2.
    Cao, F.: Good Continuations in Digital Image Level Lines. In: International Conference on Computer Vision, pp. 440–447 (2003)Google Scholar
  3. 3.
    Chang, H., Yeung, D.Y., Xiong, Y.: Super-Resolution through Neighbor Embedding. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. I, pp. 275–282 (2004)Google Scholar
  4. 4.
    Desolneux, A., Moisan, L., Morel, J.: Partial Gestalts. Tech. Rep., CMLA (2001)Google Scholar
  5. 5.
    Elad, M., Feuer, A.: Restoration of a Single Superresolution Image from Several Blurred, Noisy, and Undersampled Measured Images. IEEE Trans. Image Processing 6(12), 1646–1658 (1997)CrossRefGoogle Scholar
  6. 6.
    Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and Robust Multiframe Super Resolution. IEEE Trans. Image Processing 13(10), 1327–1344 (2004)CrossRefGoogle Scholar
  7. 7.
    Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. IEEE Computer Graphics and Applications 22(2), 56–65 (2002)CrossRefGoogle Scholar
  8. 8.
    Fridman, A.: Mixed Markov Models, Ph.D. thesis, Department of Mathematics, Brown University (2000)Google Scholar
  9. 9.
    Guo, C.E., Zhu, S.C., Wu, Y.N.: Towards a Mathematical Theory of Primal Sketch and Sketchability. In: International Conference on Computer Vision, pp. 1228–1235 (2003)Google Scholar
  10. 10.
    Hardie, R.C., Barnard, K.J., Armstrong, E.E.: Joint MAP Registration and High Resolution Image Estimation Using a Sequence of Undersampled Images. IEEE Trans. Image Processing 6(12), 1621–1633 (1997)CrossRefGoogle Scholar
  11. 11.
    Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., Salesin, D.H.: Image Analogies. In: Proc. of ACM SIGGRAPH 2001, pp. 327–340 (2001)Google Scholar
  12. 12.
    Huang, T.S., Tsai, R.Y.: Multi-Frame Image Restoration and Registration. Advances in Computer Vision and Image Processing 1, 317–339 (1984)Google Scholar
  13. 13.
    Irani, M., Peleg, S.: Improving Resolution by Image Registration. GMIP 53, 231–239 (1991)Google Scholar
  14. 14.
    Lee, A.B., Pedersen, K.S., Mumford, D.: The Nonlinear Statistics of High-Contrast Patches in Natural Images. Int’l J. Computer Vision 54(1-3), 83–103 (2003)CrossRefMATHGoogle Scholar
  15. 15.
    Pickup, L.C., Roberts, S.J., Zisserman, A.: A Sampled Texture Prior for Image Super-Resolution. In: Advances in Neural Information Processing Systems 16 (2004)Google Scholar
  16. 16.
    Roweis, S.T., Saul, L.K.: Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science 290, 2323–2326 (2000)CrossRefGoogle Scholar
  17. 17.
    Schultz, R.R., Stevenson, R.L.: Extraction of High-Resolution Frames from Video Sequences. IEEE Trans. Image Processing 5(6), 996–1011 (1996)CrossRefGoogle Scholar
  18. 18.
    Stark, H., Oskoui, P.: High Resolution Image Recovery form Image-Plane Arrays, Using Convex Projections. J. Opt. Soc. Am. A 6, 1715–1726 (1989)CrossRefGoogle Scholar
  19. 19.
    Sun, J., Zheng, N.N., Tao, H., Shum, H.Y.: Image Hallucination with Primal Sketch Priors. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 729–736 (2003)Google Scholar
  20. 20.
    Youla, D.C.: Generalized Image Restoration by the Method of Alternating Projections. IEEE Trans. Circuits Syst. CAS-25, 694–702 (1978)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Zomet, A., Peleg, S.: Efficient Super-Resolution and Applications to Mosaics. In: International Conference on Pattern Recognition, vol. I, pp. 579–583 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Tien-Lung Chang
    • 1
    • 2
  • Tyng-Luh Liu
    • 1
  • Jen-Hui Chuang
    • 2
  1. 1.Institute of Information ScienceAcademia SinicaTaipeiTaiwan
  2. 2.Dept. of Computer ScienceNational Chiao Tung UniversityHsinchuTaiwan

Personalised recommendations