Image Super-Resolution with Fast Approximate Convolutional Sparse Coding

  • Christian Osendorfer
  • Hubert Soyer
  • Patrick van der Smagt
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8836)


We present a computationally efficient architecture for image super-resolution that achieves state-of-the-art results on images with large spatial extend. Apart from utilizing Convolutional Neural Networks, our approach leverages recent advances in fast approximate inference for sparse coding. We empirically show that upsampling methods work much better on latent representations than in the original spatial domain. Our experiments indicate that the proposed architecture can serve as a basis for additional future improvements in image super-resolution.


Image Processing Sparse Coding Convolutional Neural Networks 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Olshausen, B.J., Field, D.J.: Sparse coding with an over complete basis set: a strategy employed by v1? Vision Research (1997)Google Scholar
  2. 2.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Discriminative learned dictionaries for local image analysis. In: CVPR (2008)Google Scholar
  3. 3.
    Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: NIPS (2007)Google Scholar
  4. 4.
    Ranzato, M.A., Poultney, C., Chopra, S., Lecun, Y.: Efficient learning of sparse representations with an energy-based model. In: NIPS (2006)Google Scholar
  5. 5.
    Couzinie-Devy, F., Mairal, J., Bach, F., Ponce, J.: Dictionary learning for deblurring and digital zoom. arXiv preprint arXiv:1110.0957 (2011)Google Scholar
  6. 6.
    Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Transactions on Image Processing 19(11) (2010)Google Scholar
  7. 7.
    Hu, T., Nunez-Iglesias, J., Vitaladevuni, S., Scheffer, L., Xu, S., Bolorizadeh, M., Hess, H., Fetter, R., Chklovskii, D.: Super-resolution using sparse representations over learned dictionaries: Reconstruction of brain structure using electron microscopy. arXiv preprint arXiv:1210.0564 (2012)Google Scholar
  8. 8.
    Zhang, Y., Wu, G., Yap, P.T., Feng, Q., Lian, J., Chen, W., Shen, D.: Reconstruction of super-resolution lung 4d-ct using patch-based sparse representation. In: CVPR (2012)Google Scholar
  9. 9.
    Petrou, M., Petrou, C.: Image Processing: The Fundamentals. Wiley & Sons (2010)Google Scholar
  10. 10.
    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE 86 (1998)Google Scholar
  11. 11.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)Google Scholar
  12. 12.
    Ciresan, D.C., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: CVPR (2012)Google Scholar
  13. 13.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR (2014)Google Scholar
  14. 14.
    Masci, J., Giusti, A., Ciresan, D.C., Fricout, G., Schmidhuber, J.: A fast learning algorithm for image segmentation with max-pooling convolutional networks. In: ICIP (2013)Google Scholar
  15. 15.
    Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: Overfeat: Integrated recognition, localization and detection using convolutional networks. In: ICLR (2014)Google Scholar
  16. 16.
    Kavukcuoglu, K., Ranzato, M., LeCun, Y.: Fast inference in sparse coding algorithms with applications to object recognition. Technical report, Computational and Biological Learning Lab, Courant Institute, NYU (2008)Google Scholar
  17. 17.
    Kavukcuoglu, K., Ranzato, M., Fergus, R., Le-Cun, Y.: Learning invariant features through topographic filter maps. In: CVPR (2009)Google Scholar
  18. 18.
    Kavukcuoglu, K., Sermanet, P., Boureau, Y.L., Gregor, K., Mathieu, M., Cun, Y.L.: Learning convolutional feature hierarchies for visual recognition. In: NIPS (2010)Google Scholar
  19. 19.
    Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences 2(1), 183–202 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    He, L., Qi, H., Zaretzki, R.: Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution. In: CVPR (2013)Google Scholar
  21. 21.
    Lu, X., Yuan, H., Yan, P., Yuan, Y., Li, X.: Geometry constrained sparse coding for single image super-resolution. In: CVPR (2012)Google Scholar
  22. 22.
    van Hateren, J.H., van der Schaaf, A.: Independent component filters of natural images compared with simple cells in primary visual cortex. Proceedings: Biological Sciences 265(1394) (1998)Google Scholar
  23. 23.
    Zhang, K., Gao, X., Tao, D., Li, X.: Multi-scale dictionary for single image super-resolution. In: CVPR (2012)Google Scholar
  24. 24.
    Sprechmann, D., Bronstein, A.M., Sapiro, G.: Learning efficient sparse and low rank models. arXiv preprint arXiv:1212.3631 (2012)Google Scholar
  25. 25.
    Lin, M., Chen, Q., Yan, S.: Network in network. In: ICLR (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Christian Osendorfer
    • 1
  • Hubert Soyer
    • 1
  • Patrick van der Smagt
    • 1
  1. 1.Fakultät für Informatik, Lehrstuhl für Robotik und EchtzeitsystemeTechnische Universität MünchenMünchenGermany

Personalised recommendations