Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

A Solution of the Dichromatic Model for Multispectral Photometric Invariance

Abstract

In this paper, we address the problem of photometric invariance in multispectral imaging making use of an optimisation approach based upon the dichromatic model. In this manner, we cast the problem of recovering the spectra of the illuminant, the surface reflectance and the shading and specular factors in a structural optimisation setting. Making use of the additional information provided by multispectral imaging and the structure of image patches, we recover the dichromatic parameters of the scene. To do this, we formulate a target cost function combining the dichromatic error and the smoothness priors for the surfaces under study. The dichromatic parameters are recovered through minimising this cost function in a coordinate descent manner. The algorithm is quite general in nature, admitting the enforcement of smoothness constraints and extending in a straightforward manner to trichromatic settings. Moreover, the objective function is convex with respect to the subset of variables to be optimised in each alternating step of the minimisation strategy. This gives rise to an optimal closed-form solution for each of the iterations in our algorithm. We illustrate the effectiveness of our method for purposes of illuminant spectrum recovery, skin recognition, material clustering and specularity removal. We also compare our results to a number of alternatives.

This is a preview of subscription content, log in to check access.

References

  1. Angelopoulou, E. (2000). Objective colour from multispectral imaging. In European conf. on computer vision (pp. 359–374).

  2. Angelopoulou, E. (2007). Specular highlight detection based on the Fresnel reflection coefficient. In IEEE international conference on computer vision (pp. 1–8). Los Alamitos: IEEE Computer Society.

  3. Barnard, K., Martin, L., & Funt, B. V. (2000). Colour by correlation in a three-dimensional colour space. In ECCV ’00: Proceedings of the 6th European conference on computer vision – part I (pp. 375–389). London: Springer.

  4. Barnard, K., Martin, L., Coath, A., & Funt, B. V. (2002a). A comparison of computational color constancy Algorithms – Part II: Experiments with image data. IEEE Transactions on Image Processing, 11(9), 985–996.

  5. Barnard, K., Martin, L., Funt, B., & Coath, A. (2002b). A data set for colour research. Color Research and Application, 27(3), 147–151.

  6. Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge: Cambridge University Press.

  7. Brainard, D. H., & Freeman, W. T. (1997). Bayesian color constancy. Journal of Optical Society America A, 14(7), 1393–1411.

  8. Brainard, D., & Wandell, B. (1986). Analysis of the retinex theory of color vision. Journal of Optical Society America A, 3, 1651–1661.

  9. Brainard, D. H., Delahunt, P. B., Freeman, W. T., Kraft, J. M., & Xiao, B. (2006). Bayesian model of human color constancy. Journal of Vision, 6(11), 1267–1281.

  10. Brelstaff, G., & Blake, A. (1988). Detecting specular reflection using lambertian constraints. In Int. conf. on comp. vision (pp. 297–302).

  11. Brooks, M., & Horn, B. (1985). Shape and source from shading. In MIT AI Memo.

  12. Buchsbaum, G. (1980). A spatial processor model for object color perception. Journal of The Franklin Institute, 310, 1–26.

  13. Chang, C.-C., & Lin, C.-J. (2001). LIBSVM: a library for support vector machines.

  14. Chang, J. Y., Lee, K. M., & Lee, S. U. (2003). Shape from shading using graph cuts. In Proc. of the int. conf. on image processing.

  15. CIE (1932). Commission Internationale de l’Éclairage Proceedings, 1931. Cambridge: Cambridge University Press.

  16. Dror, R. O., Adelson, E. H., & Willsky, A. S. (2001). Recognition of surface reflectance properties from a single image under unknown real-world illumination. In Proc. of the IEEE workshop on identifying objects across variations in lighting.

  17. Ferrie, F., & Lagarde, J. (1992). Curvature consistency improves local shading analysis. CVGIP: Image Understanding, 55(1), 95–105.

  18. Finlayson, G., & Hordley, S. (2000). Improving gamut mapping color constancy. IEEE Transactions on Image Processing, 9, 10.

  19. Finlayson, G. D., & Schaefer, G. (2001a). Convex and non-convex illuminant constraints for dichromatic colour constancy. CVPR, 1, 598–604.

  20. Finlayson, G. D., & Schaefer, G. (2001b). Solving for colour constancy using a constrained dichromatic reflection model. International Journal of Computer Vision, 42(3), 127–144.

  21. Finlayson, G. D., Hordley, S. D., & Hubel, P. M. (2001). Color by correlation: a simple, unifying framework for color constancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11), 1209–1221.

  22. Finlayson, G. D., Hordley, S. D., & Tastl, I. (2006). Gamut constrained illuminant estimation. International Journal of Computer Vision, 67(1), 93–109.

  23. Forsyth, D. A. (1990). A novel algorithm for color constancy. International Journal of Computer Vision, 5(1), 5–36.

  24. Healey, G. (1991). Estimating spectral reflectance using highlights. Image and Vision Computing, 9(5), 333–337.

  25. Healey, G., & Slater, D. (1999). Invariant recognition in hyperspectral images. In IEEE conf. on computer vision and pattern recognition (pp. 1438–1043).

  26. Hoaglin, D. C., Mosteller, F., & Tukey, J. W. (2000). Understanding robust and exploratory data analysis. New York: Wiley.

  27. Hofmann, T., & Buhmann, M. (1997). Pairwise data clustering by deterministic annealing. IEEE Tansactions on Pattern Analysis and Machine Intelligence, 19(1), 1–14.

  28. Horn, B. K. P. (1986). Robot vision. Cambridge: MIT.

  29. Horn, B. K. P., & Brooks, M. J. (1986). The variational approach to shape from shading. Computer Vision, Graphics and Image Processing, 33(2), 174–208.

  30. Huber, P. J. (1981). Robust statistics. New York: Wiley.

  31. Ikeuchi, K., & Horn, B. (1981). Numerical shape from shading and occluding boundaries. Artificial Intelligence, 17(1–3), 141–184.

  32. Klinker, G., Shafer, S., & Kanade, T. (1990). A physical approach to color image understanding. International Journal of Computer Vision, 4(1), 7–38.

  33. Klinker, G. J., Shafer, S. A., & Kanade, T. (1988). The measurement of highlights in color images. International Journal of Computer Vision, 2, 7–32.

  34. Koenderink, J. J., & van Doorn, A. J. (1992). Surface shape and curvature scales. Image and Vision Computing, 10(8), 557–565.

  35. Land, E. H. (1986). Recent advances in retinex theory. Vision Research, 26(1)

  36. Land, E., & McCann, J. (1971). Lightness and retinex theory. Journal of Optical Society America, 61(1), 1–11.

  37. Lee, H.-C. (1986). Method for computing the scene-illuminant chromaticity from specular highlights. Journal of the Optical Society of America A, 3, 1694–1699.

  38. Li, S. Z. (1995). Discontinuous mrf prior and robust statistics: a comparative study. Image and Vision Computing, 13(3), 227–233.

  39. Lin, S., & Shum, H.-Y. (2001). Separation of diffuse and specular reflection in color images. In Computer vision and pattern recognition, ieee computer society conference on 1 (p. 341).

  40. Mallick, S. P., Zickler, T., Belhumeur, P. N., & Kriegman, D. J. (2006). Specularity removal in images and videos: A PDE approach. In ECCV (1) (pp. 550–563).

  41. Narasimhan, S. G., Ramesh, V., & Nayar, S. K. (2003). A class of photometric invariants: separating material from shape and illumination. In International conference on computer vision (pp. 1387–1394). Washington: IEEE Computer Society.

  42. Nayar, S. K., & Bolle, R. M. (1993). Computing reflectance ratios from an image. Pattern on Recognition, 26, 1529–1542.

  43. Nayar, S., & Bolle, R. (1996). Reflectance based object recognition. International Journal of Computer Vision, 17(3), 219–240.

  44. Phung, S. L., Bouzerdoum, A., & Chai, D. (2005). Skin segmentation using color pixel classification: Analysis and comparison. IEEE Transactions on Pattern Analyses and Machine Intelligence, 27(1), 148–154.

  45. Poggio, T., & Torre, V. (1984). Ill-posed problems and regularization analysis in early vision. Tech. rep., Cambridge, MA, USA.

  46. Ragheb, H., & Hancock, E. R. (2002). Highlight removal using shape-from-shading. In European conf. on comp. vision, LNCS (Vol. 2351, pp. 626–641).

  47. Raskar, R., Tumblin, J., Mohan, A., Agrawal, A., & Li, Y. (2006). Computational photography. In Proceeding of Eurographics: state of the art report (STAR).

  48. Shafer, S. A. (1985). Using color to separate reflection components. Color Research and Applications, 10(4), 210–218.

  49. Schaefer, G., Hordley, S., & Finlayson, G. (2005). A combined physical and statistical approach to colour constancy. In Computer vision and pattern recognition, IEEE computer society conference on 1 (pp. 148–153).

  50. Slater, D., & Healey, G. (1997). Object recognition using invariant profiles. In Computer vision and pattern recognition (pp. 827–832).

  51. Stockman, H., & Gevers, T. (1999). Detection and classification of hyperspectral edges. In British machine vision conference (pp. 643–651).

  52. Suen, P. H., & Healey, G. (2001). Invariant mixture recognition in hyperspectral images. In Int. conference on computer vision (pp. 262–267).

  53. Sunshine, J. M., Pieters, C. M., & Pratt, S. F. (1990). Deconvolution of mineral absorption bands: an improved approach. Journal of Geophysical Research, 95(B5), 6955–6966.

  54. Tan, R. T., Nishino, K., & Ikeuchi, K. (2004). Separating reflection components based on chromaticity and noise analysis. IEEE Transactions on Pattern Analyses and Machine Intelligence, 26(10), 1373–1379.

  55. Tominaga, S., & Wandell, B. A. (1989). Standard surface-reflectance model and illuminant estimation. Journal of the Optical Society of America A, 6, 576–584.

  56. Tominaga, S., & Wandell, B. A. (1990). Component estimation of surface spectral reflectance. Journal of the Optical Society of America A, 7(2), 312–317.

  57. van de Weijer, J., & Gevers, T. (2005). Color constancy based on the grey-edge hypothesis. In IEEE international conference on image processing (Vol. 2, pp. 722–725).

  58. Wang, J., & Dana, K. J. (2006). Relief texture from specularities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(3), 446–457.

  59. Worthington, P. L., & Hancock, E. R. (1999). New constraints on data-closeness and needle map consistency for shape-from-shading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(12), 1250–1267.

  60. Zheng, Q., & Chellapa, R. (1991). Estimation of illuminant direction, albedo, and shape from shading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 680–702.

  61. Zickler, T., Mallick, S. P., Kriegman, D. J., & Belhumeur, P. N. (2008). Color subspaces as photometric invariants. International Journal of Computer Vision, 79(1), 13–30.

Download references

Author information

Correspondence to Cong Phuoc Huynh.

Additional information

NICTA is funded by the Australian Government as represented by the Department of Broadband, Communications and the Digital Economy and the Australian Research Council through the ICT Centre of Excellence program.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Huynh, C.P., Robles-Kelly, A. A Solution of the Dichromatic Model for Multispectral Photometric Invariance. Int J Comput Vis 90, 1–27 (2010). https://doi.org/10.1007/s11263-010-0333-y

Download citation

Keywords

  • Photometric invariance
  • Multispectral imaging
  • Dichromatic reflection model
  • Reflectance