Entropy Minimization for Shadow Removal
Recently, a method for removing shadows from colour images was developed (Finlayson et al. in IEEE Trans. Pattern Anal. Mach. Intell. 28:59–68, 2006) that relies upon finding a special direction in a 2D chromaticity feature space. This “invariant direction” is that for which particular colour features, when projected into 1D, produce a greyscale image which is approximately invariant to intensity and colour of scene illumination. Thus shadows, which are in essence a particular type of lighting, are greatly attenuated. The main approach to finding this special angle is a camera calibration: a colour target is imaged under many different lights, and the direction that best makes colour patch images equal across illuminants is the invariant direction. Here, we take a different approach. In this work, instead of a camera calibration we aim at finding the invariant direction from evidence in the colour image itself. Specifically, we recognize that producing a 1D projection in the correct invariant direction will result in a 1D distribution of pixel values that have smaller entropy than projecting in the wrong direction. The reason is that the correct projection results in a probability distribution spike, for pixels all the same except differing by the lighting that produced their observed RGB values and therefore lying along a line with orientation equal to the invariant direction. Hence we seek that projection which produces a type of intrinsic, independent of lighting reflectance-information only image by minimizing entropy, and from there go on to remove shadows as previously. To be able to develop an effective description of the entropy-minimization task, we go over to the quadratic entropy, rather than Shannon’s definition. Replacing the observed pixels with a kernel density probability distribution, the quadratic entropy can be written as a very simple formulation, and can be evaluated using the efficient Fast Gauss Transform. The entropy, written in this embodiment, has the advantage that it is more insensitive to quantization than is the usual definition. The resulting algorithm is quite reliable, and the shadow removal step produces good shadow-free colour image results whenever strong shadow edges are present in the image. In most cases studied, entropy has a strong minimum for the invariant direction, revealing a new property of image formation.
KeywordsIllumination Reflectance Intrinsic images Illumination invariants Color Shadows Entropy Quadratic entropy
Unable to display preview. Download preview PDF.
- Barrow, H., & Tenenbaum, J. (1978). Recovering intrinsic scene characteristics from images. In A. Hanson & E. Riseman (Eds.), Computer vision systems (pp. 3–26). New York: Academic Press. Google Scholar
- Beatson, R., & Greengard, L. (1997). A short course on fast multipole methods. In M. Ainsworth, J. Levesley, W. Light, & M. Marletta (Eds.), Wavelets, multilevel methods and elliptic PDEs. Oxford: Oxford University Press. Google Scholar
- Cho, J.-H., Kwon, T.-G., Jang, D.-G., & Hwang, C.-S. (2005). Moving cast shadow detection and removal for visual traffic surveillance. In: Australian conference on artificial intelligence (pp. 746–755). Google Scholar
- CIE, (1995). Method of measuring and specifying colour rendering properties of light sources. Publication 13.3, ISBN 978-3900734572. http://www.cie.co.at/publ/abst/13-3-95.html.
- Daly, S. (1992). The visible difference predictor: an algorithm for the assessment of image fidelity. In: A. Rogowitz and Klein (Eds.): Proceedings of SPIE: Vol. 1666. Human vision, visual processing, and digital display III (pp. 2–15). Google Scholar
- Drew, M., Chen, C., Hordley, S., & Finlayson, G. (2002). Sensor transforms for invariant image enhancement. In: Tenth color imaging conference: color, science, systems and applications (pp. 325–329). Google Scholar
- Drew, M., Finlayson, G., & Hordley, S. (2003). Recovery of chromaticity image free from shadows via illumination invariance. In: IEEE workshop on color and photometric methods in computer vision, ICCV’03 (pp. 32–39). Google Scholar
- Drew, M., Salahuddin, M., & Fathi, A. (2007). A standardized workflow for illumination-invariant image extraction. In: 15th color imaging conference: color, science, systems and applications. Google Scholar
- Finlayson, G., & Drew, M. (2001). 4-sensor camera calibration for image representation invariant to shading, shadows, lighting, and specularities. In: ICCV’01: international conference on computer vision (pp. II: 473–480). Google Scholar
- Finlayson, G., Drew, M., & Lu, C. (2004). Intrinsic images by entropy minimization. In Lecture Notes in Computer Science : Vol. 3023. ECCV 2004: European conference on computer vision (pp. 582–595). Berlin: Springer. Google Scholar
- Finlayson, G., Fredembach, C., & Drew, M. S. (2007). Detecting illumination in images. In: ICCV’07: international conference on computer vision. Google Scholar
- Jiang, H., & Drew, M. (2003) Shadow-resistant tracking in video. In: ICME’03: international conference on multimedia and expo (Vol. III, pp. 77–80). Google Scholar
- Li, Z.-N., & Drew, M. (2004). Fundamentals of multimedia. New York: Prentice-Hall. Google Scholar
- Liu, Z., Huang, K., Tan, T., & Wang, L. (2006). Cast shadow removal with GMM for surface reflectance component. In: ICPR06 (pp. 727–730). Google Scholar
- Tappen, M., Freeman, W., & Adelson, E. (2003). Recovering intrinsic images from a single image. In: Advances in neural information processing systems 15. Google Scholar
- Vrhel, M., Gershon, R., & Iwan, L. (1994). Measurement and analysis of object reflectance spectra. Color Research and Application, 19, 4–9. Google Scholar
- Weiss, Y. (2001). Deriving intrinsic images from image sequences. In: ICCV01 (Vol. II, pp. 68–75. Google Scholar
- Wyszecki, G., & Stiles, W. (1982). Color science: concepts and methods, quantitative data and formulas (2nd ed.). New York: Wiley, Google Scholar
- Xu, D., & Principe, J. (1998). Learning from examples with quadratic mutual information. In: Neural networks for signal processing (pp. 155–164). Google Scholar
- Yang, C., Duraiswami, R., Gumerov, N., & Davis, L. (2003). Improved fast Gauss transform and efficient kernel density estimation. In: International conference on computer vision (pp. 464–471). Google Scholar