Mutual Information Refinement for Flash-no-Flash Image Alignment

  • Sami Varjo
  • Jari Hannuksela
  • Olli Silvén
  • Sakari Alenius
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6915)


Flash-no-flash imaging aims to combine ambient light images with details available in flash images. Flash can alter color intensities radically leading to changes in gradient directions and strengths, as well as natural shadows possibly being removed and new ones created. This makes flash-no-flash image pair alignment a challenging problem. In this paper, we present a new image registration method utilizing mutual information driven point matching accuracy refinement. For a phase correlation based method, accuracy improvement through the suggested point refinement was over 40 %. The new method also performed better than the reference methods SIFT and SURF by 3.0 and 9.1 % respectively in alignment accuracy. Visual inspection also confirmed that in several cases the proposed method succeeded in registering flash-no-flash image pairs where the tested reference methods failed.


registration illumination flash 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agrawal, A., Raskar, R., Nayar, S.K., Li, Y.: Removing photography artifacts using gradient projection and flash-exposure sampling. ACM Trans. Graph. 24, 828–835 (2005)CrossRefGoogle Scholar
  2. 2.
    Alenius, S., Bilcu, R.: Combination of multiple images for flash re-lightning. In: Proc. of IEEE 3rd Int. Symp. Commun. Contr. Sig., pp. 322–327 (2008)Google Scholar
  3. 3.
    Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  4. 4.
    Chum, O., Matas, J.: Geometric Hashing with Local Affine Frames. In: IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 879–884 (2006)Google Scholar
  5. 5.
    Eisemann, E., Durand, F.: Flash photography enhancement via intrinsic relighting. ACM Trans. Graph. 23, 673–678 (2004)CrossRefGoogle Scholar
  6. 6.
    Estévez, P.A., Tesmer, M., Perez, C.A., Zurada, J.M.: Normalized mutual information feature selection. Trans. Neur. Netw. 20, 189–201 (2009)CrossRefGoogle Scholar
  7. 7.
    Evans, C.: Notes on the opensurf library. Tech. Rep. CSTR-09-001, University of Bristol (2009)Google Scholar
  8. 8.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Foroosh, H., Zerubia, J., Berthod, M.: Extension of phase correlation to subpixel registration. IEEE Trans. Image Process. 11(3), 188–200 (2002)CrossRefGoogle Scholar
  10. 10.
    Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of the 4th Alvey Vision Conference, pp. 147–151 (1988)Google Scholar
  11. 11.
    Jacquet, W., Nyssen, W., Bottenberg, P., Truyen, B., de Groen, P.: 2D image registration using focused mutual information for application in dentistry. Computers in Biology and Medicine 39, 545–553 (2009)CrossRefGoogle Scholar
  12. 12.
    Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High dynamic range video. ACM Trans. Graph. 22, 319–325 (2003)CrossRefGoogle Scholar
  13. 13.
    Kovesi, P.: Image features from phase congruency. Journal of Computer Vision Research, 2–26 (1999)Google Scholar
  14. 14.
    Kuglin, C., Hines, D.: The phase correlation image alignment method. In: IEEE Proc. Int. Conference on Cybernetics and Society, pp. 163–165 (1975)Google Scholar
  15. 15.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)CrossRefGoogle Scholar
  16. 16.
    Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide-baseline stereo from maximally stable extremal regions. In: Proceedings of the British Machine Vision Conference, pp. 384–393 (2002)Google Scholar
  17. 17.
    Nistér, D., Stewénius, H.: Linear time maximally stable extremal regions. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 183–196. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  18. 18.
    Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20, 21–36 (2003)CrossRefGoogle Scholar
  19. 19.
    Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23, 664–672 (2004)CrossRefGoogle Scholar
  20. 20.
    Pluim, J.P.W.: Mutual-information-based registration of medical images: A survey. IEEE Trans. Med. Imag. 22, 986–1004 (2003)CrossRefGoogle Scholar
  21. 21.
    Pulli, K., Tico, M., Xiong, Y.: Mobile panoramic imaging system. In: Sixth IEEE Workshop on Embedded Computer Vision, ECVW 2010 (2010)Google Scholar
  22. 22.
    Reddy, B.S., Chatterji, B.N.: An FFT-based Technique for Translation, Rotation and Scale-Invariant Image Registration. IEEE Trans. Im. Proc. 5, 1266–1271 (1996)CrossRefGoogle Scholar
  23. 23.
    Smith, S.M., Brady, J.M.: Susan – a new approach to low level image processing. Int. J. Comput. Vis. 23, 47–78 (1997)CrossRefGoogle Scholar
  24. 24.
    Studholme, C., Hill, D.L.G., Hawkes, D.J.: Overlap invariant entropy measure of 3D medical image alignment. Pattern Recognition 32(1), 71–86 (1999)CrossRefGoogle Scholar
  25. 25.
    Tico, M., Pulli, K.: Low-light imaging solutions for mobile devices. In: Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers, pp. 851–855 (2009)Google Scholar
  26. 26.
    Vandewalle, P., Süsstrunk, S., Vetterli, M.: A frequency domain approach to registration of aliased images with application to super-resolution. EURASIP J. Appl. Signal Process., p. 233 (2006)Google Scholar
  27. 27.
    Vedaldi, A., Fulkerson, B.: VLFeat: An open and portable library of computer vision algorithms (2008),
  28. 28.
    Viola, P., Wells III, W.M.: Alignment by maximization of mutual information. Int. J. Comp. Vis., 137–154 (1997)Google Scholar
  29. 29.
    Xie, X., Lam, K.-M.: An efficient illumination normalization method for face recognition. In: Pattern Recognition Letters, pp. 609–617 (2006)Google Scholar
  30. 30.
    Xiong, Y., Pulli, K.: Fast panorama stitching on mobile devices. In: IEEE Digest of Technical Papers: Int. Conference on Consumer Electronics, pp. 319–320 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Sami Varjo
    • 1
  • Jari Hannuksela
    • 1
  • Olli Silvén
    • 1
  • Sakari Alenius
    • 2
  1. 1.Machine Vision Group, Infotech Oulu and Department of Electrical and Information EngineeringUniversity of OuluFinland
  2. 2.Nokia Research CenterTampereFinland

Personalised recommendations