Spectral Edge Image Fusion: Theory and Applications

  • David Connah
  • Mark Samuel Drew
  • Graham David Finlayson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8693)


This paper describes a novel approach to the fusion of multidimensional images for colour displays. The goal of the method is to generate an output image whose gradient matches that of the input as closely as possible. It achieves this using a constrained contrast mapping paradigm in the gradient domain, where the structure tensor of a high-dimensional gradient representation is mapped exactly to that of a low-dimensional gradient field which is subsequently reintegrated to generate an output. Constraints on the output colours are provided by an initial RGB rendering to produce ‘naturalistic’ colours: we provide a theorem for projecting higher-D contrast onto the initial colour gradients such that they remain close to the original gradients whilst maintaining exact high-D contrast. The solution to this constrained optimisation is closed-form, allowing for a very simple and hence fast and efficient algorithm. Our approach is generic in that it can map any N-D image data to any M-D output, and can be used in a variety of applications using the same basic algorithm. In this paper we focus on the problem of mapping N-D inputs to 3-D colour outputs. We present results in three applications: hyperspectral remote sensing, fusion of colour and near-infrared images, and colour visualisation of MRI Diffusion-Tensor imaging.


Image fusion gradient-based contrast dimensional reduction colour colour display 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agrawal, A., Chellappa, R., Raskar, R.: An algebraic approach to surface reconstruction from gradient fields. In: Int. Conf. on Comp. Vision, pp. 174–181 (2005)Google Scholar
  2. 2.
    Arsigny, V., Fillard, P., Pennec, X., Ayache, N.: Log-euclidean metrics for fast and simple calculus. Mag. Res. in Medicine 56, 411–421 (2006)CrossRefGoogle Scholar
  3. 3.
    Bhat, P., Zitnick, C.L., Cohen, M., Curless, B.: Gradientshop: A gradient-domain optimization framework for image and video filtering. ACM Trans. Graph. 10, 10:1–10:14 (2010)Google Scholar
  4. 4.
    Burt, P.J., Adelson, E.H.: Merging images through pattern decomposition. In: Proc. SPIE 0575, Applications of Digital Image Processing VIII. pp. 173–181 (1985)Google Scholar
  5. 5.
    Campbell, J.B., Wynne, H.: Introduction to Remote Sensing, 5th edn. Guilford Press (2011)Google Scholar
  6. 6.
    Connah, D., Drew, M., Finlayson, G.: Method and system for generating accented image data. In: U.S. patent No. 8682093 and UK patent GB0914982.4 (March 25, 2014)Google Scholar
  7. 7.
    Cornsweet, T.: Visual Perception. Academic Press, New York (1970)Google Scholar
  8. 8.
    Cui, M., Hu, J., Razdan, A., Wonka, P.: Color to gray conversion using ISOMAP. Vis. Comput. 26, 1349–1360 (2010)CrossRefGoogle Scholar
  9. 9.
    Davis, J., Sharma, V.: Background-subtraction using contour-based fusion of thermal and visible imagery. Comp. Vis. and Im. Und.; IEEE OTCBVS WS Series Bench 106 (2-3), 162–182 (2007)Google Scholar
  10. 10.
    Di Zenzo, S.: A note on the gradient of a multi-image. Comp. Vision, Graphics, and Image Proc. 33, 116–125 (1986)CrossRefzbMATHGoogle Scholar
  11. 11.
    Drew, M., Finlayson, G.: Improvement of colorization realism via the structure tensor. Int. J. Image and Graphics 11(4), 589–609 (2011)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Fattal, R., Lischinski, D., Werman, M.: Gradient domain high dynamic range compression. ACM Trans. on Graphics 21, 249–256 (2002)CrossRefGoogle Scholar
  13. 13.
    Fay, D., Waxman, A.M., Aguilar, M., Ireland, D., Racamato, J., Ross, W., Streilein, W.W., Braun, M.I.: Fusion of multi-sensor imagery for night vision: Color visualization, target learning and search. In: 3rd Int. Conf. Information Fusion, pp. 215–219 (2000)Google Scholar
  14. 14.
    Finlayson, G., Connah, D., Drew, M.: Image reconstruction method and system, U.S. and U.K. filing, British Patent Office Application Number GB0914603.6 (August 20, 2009)Google Scholar
  15. 15.
    Finlayson, G.D., Connah, D., Drew, M.S.: Lookup-table-based gradient field reconstruction. IEEE Trans. Im. Proc. 20(10), 2827–2836 (2011)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Frankot, R.T., Chellappa, R.: A method for enforcing integrability in shape from shading algorithms. IEEE Trans. on Patt. Anal. and Mach. Intell. 10, 439–451 (1988)CrossRefzbMATHGoogle Scholar
  17. 17.
    Fredembach, C., Barbuscia, N., Süsstrunk, S.: Combining visible and near-infrared images for realistic skin smoothing. In: Color Imaging Conf. (2009)Google Scholar
  18. 18.
    Golub, G., van Loan, C.: Matrix Computations. John Hopkins U. Press (1983)Google Scholar
  19. 19.
    Hamarneh, G., McIntosh, C., Drew, M.S.: Perception-based visualization of manifold-valued medical images using distance-preserving dimensionality reduction. IEEE Trans. on Medical Imaging 30(7), 1314–1327 (2011)CrossRefGoogle Scholar
  20. 20.
    Jacobson, N., Gupta, M., Cole, J.: Linear fusion of image sets for display. IEEE Trans. on Geosciences and Remote Sensing 45, 3277–3288 (2007)CrossRefGoogle Scholar
  21. 21.
    Kotwal, K., Chaudhuri, S.: Visualization of hyperspectral images using bilateral filtering. IEEE Trans. Geosci. and Remote Sen. 48(5), 2308–2316 (2010)CrossRefGoogle Scholar
  22. 22.
    Lau, C., Heidrich, W., Mantiuk, R.: Cluster-based color space optimizations. In: Int. Conf. on Comp. Vision, pp. 1172–1179 (2011)Google Scholar
  23. 23.
    Lewis, J., O’Callaghan, R., Nikolov, S., Bull, D., Canagarajah, C.: Region-based image fusion using complex wavelets. In: 7th Int. Conf. on Information Fusion, vol. 1, pp. 555–562 (2004)Google Scholar
  24. 24.
    Li, H., Manjunath, B., Mitra, S.: Multisensor image fusion using the wavelet transform. Graphical Models and Im. Proc. 57(3), 235–245 (1995)CrossRefGoogle Scholar
  25. 25.
    Morovic, J.: Color Gamut Mapping. John Wiley & Sons (2008)Google Scholar
  26. 26.
    NASA: Aviris: Airborne visible / infrared imaging spectrphotometer (2013),
  27. 27.
    NASA: Landsat imagery (2013),
  28. 28.
    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2005)CrossRefGoogle Scholar
  29. 29.
    Piella, G.: Image fusion for enhanced visualization: a variational approach. Int. J. Comput. Vision 83(1), 1–11 (2009)CrossRefGoogle Scholar
  30. 30.
    Pohl, C., Genderen, J.L.V.: Multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. of Remote Sensing 19(5), 823–854 (1998)CrossRefGoogle Scholar
  31. 31.
    Schaul, L., Fredembach, C., Süsstrunk, S.: Color image dehazing using the near-infrared. In: Int. Conf. on Im. Proc. (2009)Google Scholar
  32. 32.
    Scheunders, P.: A multivalued image wavelet representation based on multiscale fundamental forms. IEEE Trans. Im. Proc. 11(5), 568–575 (2002)CrossRefMathSciNetGoogle Scholar
  33. 33.
    Socolinsky, D., Wolff, L.: Multispectral image visualization through first-order fusion. IEEE Trans. Im. Proc. 11, 923–931 (2002)CrossRefGoogle Scholar
  34. 34.
    Stokes, M., Anderson, M., Chandrasekar, S., Motta, R.: A standard default color space for the internet – sRGB (1996),
  35. 35.
    Toet, A.: Natural colour mapping for multiband nightvision imagery. Infor. Fusion 4, 155–166 (2003)CrossRefGoogle Scholar
  36. 36.
    Toet, A., Ruyven, J.J.V., Valeton, J.M.: Merging thermal and visual images by a contrast pyramid. Optical Eng. 28(7), 789–792 (1989)CrossRefGoogle Scholar
  37. 37.
    Tyo, J., Konsolakis, A., Diersen, D., Olsen, R.: Principal-components-based display strategy for spectral imagery. IEEE Trans. on Geosciences and Remote Sensing 41, 708–718 (2003)CrossRefGoogle Scholar
  38. 38.
    Waxman, A., Gove, A., Fay, D., Racamoto, J., Carrick, J., Seibert, M., Savoye, E.: Color night vision: Opponent processing in the fusion of visible and ir imagery. Neural Networks 10, 1–6 (1997)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • David Connah
    • 1
  • Mark Samuel Drew
    • 2
  • Graham David Finlayson
    • 3
  1. 1.University of BradfordUK
  2. 2.Simon Fraser UniversityVancouverCanada
  3. 3.University of East AngliaNorwichUK

Personalised recommendations