International Journal of Computer Vision

, Volume 85, Issue 1, pp 35–57 | Cite as

Entropy Minimization for Shadow Removal

Article

Abstract

Recently, a method for removing shadows from colour images was developed (Finlayson et al. in IEEE Trans. Pattern Anal. Mach. Intell. 28:59–68, 2006) that relies upon finding a special direction in a 2D chromaticity feature space. This “invariant direction” is that for which particular colour features, when projected into 1D, produce a greyscale image which is approximately invariant to intensity and colour of scene illumination. Thus shadows, which are in essence a particular type of lighting, are greatly attenuated. The main approach to finding this special angle is a camera calibration: a colour target is imaged under many different lights, and the direction that best makes colour patch images equal across illuminants is the invariant direction. Here, we take a different approach. In this work, instead of a camera calibration we aim at finding the invariant direction from evidence in the colour image itself. Specifically, we recognize that producing a 1D projection in the correct invariant direction will result in a 1D distribution of pixel values that have smaller entropy than projecting in the wrong direction. The reason is that the correct projection results in a probability distribution spike, for pixels all the same except differing by the lighting that produced their observed RGB values and therefore lying along a line with orientation equal to the invariant direction. Hence we seek that projection which produces a type of intrinsic, independent of lighting reflectance-information only image by minimizing entropy, and from there go on to remove shadows as previously. To be able to develop an effective description of the entropy-minimization task, we go over to the quadratic entropy, rather than Shannon’s definition. Replacing the observed pixels with a kernel density probability distribution, the quadratic entropy can be written as a very simple formulation, and can be evaluated using the efficient Fast Gauss Transform. The entropy, written in this embodiment, has the advantage that it is more insensitive to quantization than is the usual definition. The resulting algorithm is quite reliable, and the shadow removal step produces good shadow-free colour image results whenever strong shadow edges are present in the image. In most cases studied, entropy has a strong minimum for the invariant direction, revealing a new property of image formation.

Keywords

Illumination Reflectance Intrinsic images Illumination invariants Color Shadows Entropy Quadratic entropy 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Barrow, H., & Tenenbaum, J. (1978). Recovering intrinsic scene characteristics from images. In A. Hanson & E. Riseman (Eds.), Computer vision systems (pp. 3–26). New York: Academic Press. Google Scholar
  2. Baxter, B., & Roussos, G. (2002). A new error estimate of the fast Gauss transform. SIAM Journal on Scientific Computing, 24(1), 257–259. MATHCrossRefMathSciNetGoogle Scholar
  3. Beatson, R., & Greengard, L. (1997). A short course on fast multipole methods. In M. Ainsworth, J. Levesley, W. Light, & M. Marletta (Eds.), Wavelets, multilevel methods and elliptic PDEs. Oxford: Oxford University Press. Google Scholar
  4. Bergner, S., Drew, M. S., & Möller (2009). A tool to create illuminant and reflectance spectra for light-driven graphics and visualization. ACM Transactions on Graphics, 28(5), 1–11. CrossRefGoogle Scholar
  5. Cho, J.-H., Kwon, T.-G., Jang, D.-G., & Hwang, C.-S. (2005). Moving cast shadow detection and removal for visual traffic surveillance. In: Australian conference on artificial intelligence (pp. 746–755). Google Scholar
  6. CIE, (1995). Method of measuring and specifying colour rendering properties of light sources. Publication 13.3, ISBN 978-3900734572. http://www.cie.co.at/publ/abst/13-3-95.html.
  7. Comaniciu, D., & Meer, P. (2002). Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 603–619. CrossRefGoogle Scholar
  8. Daly, S. (1992). The visible difference predictor: an algorithm for the assessment of image fidelity. In: A. Rogowitz and Klein (Eds.): Proceedings of SPIE: Vol. 1666. Human vision, visual processing, and digital display III (pp. 2–15). Google Scholar
  9. Drew, M., Chen, C., Hordley, S., & Finlayson, G. (2002). Sensor transforms for invariant image enhancement. In: Tenth color imaging conference: color, science, systems and applications (pp. 325–329). Google Scholar
  10. Drew, M., Finlayson, G., & Hordley, S. (2003). Recovery of chromaticity image free from shadows via illumination invariance. In: IEEE workshop on color and photometric methods in computer vision, ICCV’03 (pp. 32–39). Google Scholar
  11. Drew, M., Salahuddin, M., & Fathi, A. (2007). A standardized workflow for illumination-invariant image extraction. In: 15th color imaging conference: color, science, systems and applications. Google Scholar
  12. Elgammal, A., Duraiswami, R., & Davis, L. S. (2003). Efficient kernel density estimation using the fast Gauss transform with applications to color modeling and tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(11), 1499–1504. CrossRefGoogle Scholar
  13. Finlayson, G., & Drew, M. (2001). 4-sensor camera calibration for image representation invariant to shading, shadows, lighting, and specularities. In: ICCV’01: international conference on computer vision (pp. II: 473–480). Google Scholar
  14. Finlayson, G., & Hordley, S. (2001). Colour constancy at a pixel. Journal of the Optical Society of America A, 18(2), 253–264. CrossRefGoogle Scholar
  15. Finlayson, G., Drew, M., & Funt, B. (1994). Spectral sharpening: sensor transformations for improved color constancy. Journal of the Optical Society of America A, 11(5), 1553–1563. CrossRefGoogle Scholar
  16. Finlayson, G., Hordley, S., & Drew, M. (2002). Removing shadows from images. In Lecture Notes in Computer Science : Vol. 2353. ECCV 2002: European conference on computer vision (pp. 823–836). Berlin: Springer. CrossRefGoogle Scholar
  17. Finlayson, G., Drew, M., & Lu, C. (2004). Intrinsic images by entropy minimization. In Lecture Notes in Computer Science : Vol. 3023. ECCV 2004: European conference on computer vision (pp. 582–595). Berlin: Springer. Google Scholar
  18. Finlayson, G., Hordley, S., Lu, C., & Drew, M. (2006). On the removal of shadows from images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 59–68. CrossRefGoogle Scholar
  19. Finlayson, G., Fredembach, C., & Drew, M. S. (2007). Detecting illumination in images. In: ICCV’07: international conference on computer vision. Google Scholar
  20. Greengard, L., & Strain, J. (1991). The fast Gauss transform. SIAM Journal on Scientific and Statistical Computing, 12(1), 79–94. MATHCrossRefMathSciNetGoogle Scholar
  21. Hsu, E., Mertens, T., Paris, S., Avidan, S., & Durand, F. (2008). Light mixture estimation for spatially varying white balance. ACM Transactions on Graphics, 27(3), 1–7. CrossRefGoogle Scholar
  22. Jiang, H., & Drew, M. (2003) Shadow-resistant tracking in video. In: ICME’03: international conference on multimedia and expo (Vol. III, pp. 77–80). Google Scholar
  23. Jiang, H., & Drew, M. (2007). Shadow resistant tracking using inertia constraints. Pattern Recognition, 40, 1929–1945. MATHCrossRefGoogle Scholar
  24. Land, E., & McCann, J. (1971). Lightness and retinex theory. Journal of the Optical Society of America, 61, 1–11. CrossRefGoogle Scholar
  25. Li, Z.-N., & Drew, M. (2004). Fundamentals of multimedia. New York: Prentice-Hall. Google Scholar
  26. Liu, Z., Huang, K., Tan, T., & Wang, L. (2006). Cast shadow removal with GMM for surface reflectance component. In: ICPR06 (pp. 727–730). Google Scholar
  27. Martel-Brisson, N., & Zaccarin, A. (2007). Learning and removing cast shadows through a multidistribution approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 1133–1146. CrossRefGoogle Scholar
  28. Nadimi, S., & Bhanu, B. (2004). Physical models for moving shadow and object detection in video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 1079–1087. CrossRefGoogle Scholar
  29. Parzen, E. (1962). On estimation of a probability density function and mode. Annals of Mathematical Statistics, 33, 1065–1076. MATHCrossRefMathSciNetGoogle Scholar
  30. Prati, A., Mikic, I., Trivedi, M., & Cucchiara, R. (2003). Detecting moving shadows: algorithms and evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, 918–923. CrossRefGoogle Scholar
  31. Ramanath, R., Snyder, W., Yoo, Y., & Drew, M. S. (2005). Color image processing pipeline in digital still cameras. IEEE Signal Processing, 22(1), 34–43. CrossRefGoogle Scholar
  32. Renyi, A. (1987). A diary on information theory. New York: Wiley. MATHGoogle Scholar
  33. Scott, D. (1992). Multivariate density estimation: theory, practice and visualization. New York: Wiley and Kegan Paul. MATHCrossRefGoogle Scholar
  34. Stauder, J., Mech, R., & Ostermann, J. (1999). Detection of moving cast shadows for object segmentation. IEEE Transactions on Multimedia, 1, 65–76. CrossRefGoogle Scholar
  35. Tappen, M., Freeman, W., & Adelson, E. (2003). Recovering intrinsic images from a single image. In: Advances in neural information processing systems 15. Google Scholar
  36. Tappen, M., Freeman, W., & Adelson, E. (2005). Recovering intrinsic images from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 1459–1472. CrossRefGoogle Scholar
  37. Vrhel, M., Gershon, R., & Iwan, L. (1994). Measurement and analysis of object reflectance spectra. Color Research and Application, 19, 4–9. Google Scholar
  38. Weiss, Y. (2001). Deriving intrinsic images from image sequences. In: ICCV01 (Vol. II, pp. 68–75. Google Scholar
  39. Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Wann Jensen, H., & Gross, M. (2006). Analysis of human faces using a measurement-based skin reflectance model. ACM Transactions on Graphics, 25, 1013–1024. CrossRefGoogle Scholar
  40. Wyszecki, G., & Stiles, W. (1982). Color science: concepts and methods, quantitative data and formulas (2nd ed.). New York: Wiley, Google Scholar
  41. Xu, D., & Principe, J. (1998). Learning from examples with quadratic mutual information. In: Neural networks for signal processing (pp. 155–164). Google Scholar
  42. Yang, C., Duraiswami, R., Gumerov, N., & Davis, L. (2003). Improved fast Gauss transform and efficient kernel density estimation. In: International conference on computer vision (pp. 464–471). Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.University of East AngliaNorwichUK
  2. 2.Simon Fraser UniversityVancouverCanada
  3. 3.Micron Technology, Inc.San JoseUSA

Personalised recommendations