Computational Visual Media

, Volume 3, Issue 3, pp 295–304 | Cite as

Batch image alignment via subspace recovery based on alternative sparsity pursuit

  • Xianhui Lin
  • Zhu Liang Yu
  • Zhenghui Gu
  • Jun Zhang
  • Zhaoquan Cai
Open Access
Research Article


The problem of robust alignment of batches of images can be formulated as a low-rank matrix optimization problem, relying on the similarity of well-aligned images. Going further, observing that the images to be aligned are sampled from a union of low-rank subspaces, we propose a new method based on subspace recovery techniques to provide more robust and accurate alignment. The proposed method seeks a set of domain transformations which are applied to the unaligned images so that the resulting images are made as similar as possible. The resulting optimization problem can be linearized as a series of convex optimization problems which can be solved by alternative sparsity pursuit techniques. Compared to existing methods like robust alignment by sparse and low-rank models, the proposed method can more effectively solve the batch image alignment problem, and extract more similar structures from the misaligned images.


image alignment subspace recovery sparse representation convex optimization image similarity 



This work was partly supported by the National Natural Science Foundation of China (Grant Nos. 61573150, 61573152, 61370185, 61403085, and 51275094), and Guangzhou Project Nos. 201604016113 and 201604046018.


  1. [1]
    Frey, B. J.; Jojic, N. Transformation-invariant clustering using the EM algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 25, No. 1, 1–17, 2003.CrossRefGoogle Scholar
  2. [2]
    Pluim, J. P. W.; Maintz, J. B. A.; Viergever, M. A. Mutual-information-based registration of medical images: A survey. IEEE Transactions on Medical Imaging Vol. 22, No. 8, 986–1004, 2003.CrossRefzbMATHGoogle Scholar
  3. [3]
    Peng, Y.; Ganesh, A.; Wright, J.; Xu, W.; Ma, Y. RASL: Robust alignment by sparse and lowrank decomposition for linearly correlated images. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. 11, 2233–2246, 2012.CrossRefGoogle Scholar
  4. [4]
    Candès, E. J.; Li, X.; Ma, Y.; Wright, J. Robust principal component analysis? Journal of the ACM Vol. 58, No. 3, Article No. 11, 2011.Google Scholar
  5. [5]
    Liu, G.; Lin, Z.; Yan, S.; Sun, J.; Yu, Y.; Ma, Y. Robust recovery of subspace structures by lowrank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 1, 171–184, 2013.CrossRefGoogle Scholar
  6. [6]
    Elhamifar, E.; Vidal, R. Sparse subspace clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2790–2797, 2009.Google Scholar
  7. [7]
    Bian, X.; Krim, H. BI-sparsity pursuit for robust subspace recovery. In: Proceedings of the IEEE International Conference on Image Processing, 3535–3539, 2015.Google Scholar
  8. [8]
    Rubinstein, R.; Faktor, T.; Elad, M. K-SVD dictionary-learning for the analysis sparse model. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 5405–5408, 2012.Google Scholar
  9. [9]
    Bian, X.; Krim, H. Robust subspace recovery via bi-sparsity pursuit. arXiv preprint arXiv:1403.8067, 2014.Google Scholar
  10. [10]
    Elad, M. Sparse and redundant representation modeling—What next? IEEE Signal Processing Letters Vol. 19, No. 12, 922–928, 2012.CrossRefGoogle Scholar
  11. [11]
    Wright, J.; Yang, A. Y.; Ganesh, A.; Sastry, S. S.; Ma, Y. Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 31, No. 2, 210–227, 2009.CrossRefGoogle Scholar
  12. [12]
    Candès, E. J.; Romberg, J. K.; Tao, T. Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics Vol. 59, No. 8, 1207–1223, 2006.MathSciNetCrossRefzbMATHGoogle Scholar
  13. [13]
    Donoho, D. L. For most large underdetermined systems of linear equations the minimal l 1-norm solution is also the sparsest solution. Communications on Pure and Applied Mathematics Vol. 59, No. 6, 797–829, 2006.MathSciNetCrossRefzbMATHGoogle Scholar
  14. [14]
    Ma, Y.; Soatto, S.; Kosecka, J.; Sastry, S. S. An Invitation to 3-D Vision: From Images to Geometric Models, Volume 26. Springer Science & Business Media, 2012.zbMATHGoogle Scholar
  15. [15]
    Baker, S.; Matthews, I. Lucas–Kanade 20 years on: A unifying framework. International Journal of Computer Vision Vol. 56, No. 3, 221–255, 2004.CrossRefGoogle Scholar
  16. [16]
    Vedaldi, A.; Guidi, G.; Soatto, S. Joint data alignment up to (lossy) transformations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–8, 2008.Google Scholar
  17. [17]
    Boyd, S.; Parikh, N.; Chu, E.; Peleato, B.; Eckstein, J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends R in Machine Learning Vol. 3, No. 1, 1–122, 2011.zbMATHGoogle Scholar
  18. [18]
    Liu, G.; Lin, Z.; Yu, Y. Robust subspace segmentation by low-rank representation. In: Proceedings of the 27th International Conference on Machine Learning, 663–670, 2010.Google Scholar
  19. [19]
    Rockafellar, R. T. Augmented Lagrange multiplier functions and duality in nonconvex programming. SIAM Journal on Control Vol. 12, No. 2, 268–285, 1974.MathSciNetCrossRefzbMATHGoogle Scholar
  20. [20]
    Lin, Z.; Liu, R.; Su, Z. Linearized alternating direction method with adaptive penalty for low-rank representation. In: Proceedings of the Advances in Neural Information Processing Systems 24, 612–620, 2011.Google Scholar
  21. [21]
    Huang, G. B.; Ramesh, M.; Berg, T.; Learned-Miller, E. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, 2007.Google Scholar
  22. [22]
    Hore, A.; Ziou, D. Image quality metrics: PSNR vs. SSIM. In: Proceedings of the 20th International Conference on Pattern Recognition, 2366–2369, 2010.Google Scholar
  23. [23]
    Wang, Z.; Bovik, A. C.; Sheikh, H. R.; Simoncelli, E. P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing Vol. 13, No. 4, 600–612, 2004.CrossRefGoogle Scholar
  24. [24]
    LeCun, Y.; Cortes, C.; Burges, C. J. C. The MNIST database of handwritten digits. 2010. Available at Scholar

Copyright information

© The Author(s) 2017

Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from To submit a manuscript, please go to

Authors and Affiliations

  • Xianhui Lin
    • 1
  • Zhu Liang Yu
    • 1
  • Zhenghui Gu
    • 1
  • Jun Zhang
    • 2
  • Zhaoquan Cai
    • 3
  1. 1.College of Automation Science and EngineeringSouth China University of TechnologyGuangzhouChina
  2. 2.School of Information EngineeringGuangdong University of TechnologyGuangzhouChina
  3. 3.School of Computer ScienceHuizhou UniversityHuizhouChina

Personalised recommendations