Journal of Mathematical Imaging and Vision

, Volume 53, Issue 2, pp 131–150 | Cite as

Edge-Based Multi-modal Registration and Application for Night Vision Devices

  • Camille Sutour
  • Jean-François Aujol
  • Charles-Alban Deledalle
  • Baudouin Denis de Senneville


Multi-modal image sequence registration is a challenging problem that consists in aligning two image sequences of the same scene acquired with a different sensor, hence containing different characteristics. We focus in this paper on the registration of optical and infra-red image sequences acquired during the flight of a helicopter. Both cameras are located at different positions and they provide complementary informations. We propose a fast registration method based on the edge information: a new criterion is defined in order to take into account both the magnitude and the orientation of the edges of the images to register. We derive a robust technique based on a gradient ascent and combined with a reliability test in order to quickly determine the optimal transformation that matches the two image sequences. We show on real multi-modal data that our method outperforms classical registration methods, thanks to the shape information provided by the contours. Besides, results on synthetic images and real experimental conditions show that the proposed algorithm manages to find the optimal transformation in few iterations, achieving a rate of about 8 frames per second.


Multi-modal Image sequence registration Night vision Optimization 



C. Sutour thanks the DGA and the Aquitaine region for funding her PhD. J.-F. Aujol acknowledges the support of the Institut Universitaire de France. This study has been carried out with financial support from the French State, managed by the French National Research Agency (ANR) in the frame of the “Investments for the future” Programme IdEx Bordeaux-CPU (ANR-10-IDEX-03-02).


  1. 1.
    Allaire, S., Kim, J.J., Breen, S.L., Jaffray, D.A., Pekar, V.: Full orientation invariance and improved feature selectivity of 3d sift with application to medical image analysis. In: IEEE computer society conference on computer vision and pattern recognition workshops, 2008. CVPRW’08, pp. 1–8. IEEE (2008)Google Scholar
  2. 2.
    Aubert, G., Kornprobst, P.: Mathematical problems in image processing: partial differential equations and the calculus of variationsx, vol. 147. Springer, New York (2006)Google Scholar
  3. 3.
    Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: SURF: Speeded up robust features. Comput. Vis. Image Underst. (CVIU) 110(3), 346–359 (2008)CrossRefGoogle Scholar
  4. 4.
    Brown, L.: A survey of image registration techniques. ACM Comput. Surv. 24(4), 325–376 (1992)CrossRefGoogle Scholar
  5. 5.
    Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490–530 (2005)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Dellinger, F., Delon, J., Gousseau, Y., Michel, J., Tupin, F.: SAR-SIFT: a SIFT-like algorithm for applications on sar images. In: 2012 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 3478–3481. IEEE (2012)Google Scholar
  7. 7.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Haber, E., Modersitzki, J.: Intensity gradient based registration and fusion of multi-modal images. Medical image computing and computer-assisted intervention: MICCAI 46, 292–299 (2006)Google Scholar
  9. 9.
    Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge university press, Cambridge (2003)Google Scholar
  10. 10.
    Heinrich, M.P., Jenkinson, M., Bhushan, M., Matin, T., Gleeson, F.V., Brady, S.M., Schnabel, J.A.: Mind: Modality independent neighbourhood descriptor for multi-modal deformable registration. Med. Image Anal. 16(7), 1423–1435 (2012)CrossRefGoogle Scholar
  11. 11.
    Hermosillo, G., Chefd’Hotel, C., Faugeras, O.: Variational methods for multimodal image matching. Int. J. Comput. Vis. 50(3), 329–343 (2002)CrossRefGoogle Scholar
  12. 12.
    Irani, M., Anandan, P.: Robust multi-sensor image alignment. In: Sixth International Conference on Computer Vision, 1998, pp. 959–966. IEEE (1998)Google Scholar
  13. 13.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  14. 14.
    Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imaging 16(2), 187–198 (1997)CrossRefGoogle Scholar
  15. 15.
    Mann, S., Picard, R.W.: Video orbits of the projective group a simple approach to featureless estimation of parameters. IEEE Trans. Image Process. 6(9), 1281–1295 (1997)CrossRefGoogle Scholar
  16. 16.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)CrossRefGoogle Scholar
  17. 17.
    Odobez, J.M., Bouthemy, P.: Robust multiresolution estimation of parametric motion models. J. Vis. Commun. Image Represent. 6(4), 348–365 (1995)CrossRefGoogle Scholar
  18. 18.
    Paganelli, C., Peroni, M., Pennati, F., Baroni, G., Summers, P., Bellomi, M., Riboldi, M.: Scale invariant feature transform as feature tracking method in 4 D imaging: a feasibility study. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 6543–6546 (2012)Google Scholar
  19. 19.
    Pluim, J., Maintz, J., Viergever, M.: Image registration by maximization of combined mutual information and gradient information. IEEE Trans. Med. Imaging 19(8), 809–814 (2000)CrossRefGoogle Scholar
  20. 20.
    Pratt, W.: Digital Image Processing. Wiley, New York (1978)Google Scholar
  21. 21.
    Roshni, V., Revathy, K.: Using mutual information and cross correlation as metrics for registration of images. J. Theor. Appl. Inf. Technol. 4 (6) (2008)Google Scholar
  22. 22.
    Spontón, H., Cardelino, J.: A review of classic edge detectors. In: Image Processing On Line (2012)Google Scholar
  23. 23.
    Sun, Y., Jolly, M.P., Moura, J.F.: Integrated registration of dynamic renal perfusion MRimages. In: ICIP, pp. 1923–1926 (2004)Google Scholar
  24. 24.
    Toews, M., Zöllei, L., Wells, W.M.: Feature-based alignment of volumetric multi-modal images. In: Information Processing in Medical Imaging, pp. 25–36. Springer (2013)Google Scholar
  25. 25.
    Viola, P., Wells III, W.M.: Alignment by maximization of mutual information. Int. J. Comput. Vis. 24(2), 137–154 (1997)CrossRefGoogle Scholar
  26. 26.
    Yu, L., Zhang, D., Holden, E.J.: A fast and fully automatic registration approach based on point features for multi-source remote-sensing images. Computers & Geosciences 34(7), 838–848 (2008)CrossRefGoogle Scholar
  27. 27.
    Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Camille Sutour
    • 1
  • Jean-François Aujol
    • 2
  • Charles-Alban Deledalle
    • 2
  • Baudouin Denis de Senneville
    • 2
    • 3
  1. 1.IMB and LaBRIUniversité de BordeauxTalenceFrance
  2. 2.IMB, CNRS, UMR 5251Université de BordeauxTalenceFrance
  3. 3.Imaging DivisionUMC UtrechtUtrechtThe Netherlands

Personalised recommendations