International Journal of Computer Vision

, Volume 94, Issue 1, pp 54–77 | Cite as

Lightness Recovery for Pictorial Surfaces

  • Anna Paviotti
  • David A. Forsyth
  • Guido M. Cortelazzo


An important technique in cultural heritage preservation is multispectral acquisition, where one recovers a detailed spectral record of a painting using carefully calibrated lighting. This is difficult to do with frescoes, because it is hard to recover the spatial variation in light intensity that results from factors like the imaging setup and the curvature of the fresco. We introduce a new formulation of the lightness problem applied to images of pictorial artworks. The problem is different from the conventional lightness problem, because artists often paint the effects of light, so the albedo field contains a component that mimics an illumination field. Our method distinguishes between physical illumination and painted shading through spatial frequency effects and dynamic range considerations. We evaluate our method using multispectral images of paintings, where the physical illumination field is known. Our method produces estimates of the illumination intensity field that compare very well with the known ground truth, and outperforms other state-of-the art lightness recovery algorithms. For frescoes, ground truth is not available, but we show that our method produces consistent results, in the sense that the illumination functions estimated on the image and on (some of) its subimages are very similar on the overlap. We show our method produces qualitatively good color corrections for images of frescoes found on the web.


Lightness problem Color constancy Multispectral imaging Color correction Cultural heritage 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Antonioli, G., Fermi, F., Oleari, C., & Riverberi, R. (2004). Spectrophotometric scanner for imaging of paintings and other works of art. In Proceedings of CGIV (pp. 219–224). Google Scholar
  2. Barni, M., Pelagotti, A., & Piva, A. (2005). Image processing for the analysis and conservation of paintings: opportunities and challenges. IEEE Signal Processing Magazine, 22(5), 141–144. CrossRefGoogle Scholar
  3. Barrow, H. G., & Tenenbaum, J. M. (2008). Recovering intrinsic scene characteristics from images. In A. Hanson & E. Risenman (Eds.), Computer vision systems. San Diego: Academic Press. Google Scholar
  4. Blake, A. (1985). Boundary conditions for lightness computation in Mondrian world. Computer Vision, Graphics and Image Processing, 32, 314–327. CrossRefGoogle Scholar
  5. Bousseau, A., Paris, S., & Durand, F. (2009). User-assisted intrinsic images. In SIGGRAPH Asia. Google Scholar
  6. Brainard, D. H., & Wandell, B. (1986). Analysis of the retinex theory of color vision. Journal of the Optical Society of America A, 3, 1651–1661. CrossRefGoogle Scholar
  7. Brelstaff, G., & Blake, A. (1987). Computing lightness. Pattern Recognition Letters, 5(2), 129–138. CrossRefGoogle Scholar
  8. Brusco, N., Capeleto, S., Fedel, M., Paviotti, A., Poletto, L., Cortelazzo, G. M., & Tondello, G. (2006). A system for 3d modeling frescoed historical buildings with multispectral texture information. Machine Vision and Applications, 17, 373–393. CrossRefGoogle Scholar
  9. Carcagni, P., Patria, A. D., Fontana, R., Greco, M., Mastroianni, M., Materazzi, M., Pampaloni, E., & Pezzati, L. (2007). Multispectral imaging of paintings by optical scanning. Optics and Lasers in Engineering, 45(3), 360–367. CrossRefGoogle Scholar
  10. Castello del Buonconsiglio in Trento (Italy) (2006).
  11. Dacorogna, B. (1989). Direct methods in the calculus of variations. Berlin: Springer. zbMATHGoogle Scholar
  12. D’Zmura, M., & Iverson, G. (1993). Color constancy I. basic theory of two-stage linear recovery of spectral descriptions for lights and surfaces. Journal of the Optical Society of America A, 10(10), 2148–2165. CrossRefGoogle Scholar
  13. Elad, M., Kimmel, R., Shaked, D., & Keshet, R. (2003). Reduced complexity retinex algorithm via the variational approach. Journal of Visual Communication and Image Representation, 14(4), 369–388. CrossRefGoogle Scholar
  14. Farenzena, M., & Fusiello, A. (2007). Recovering intrinsic images using an illumination invariant image. In Proceedings of ICIP (Vol. 3, pp. 485–488). Google Scholar
  15. Finlayson, G. D., Hordley, S. D., & Hubel, P. M. (2001). Color by correlation: a simple, unifying framework for color constancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11), 1209–1221. CrossRefGoogle Scholar
  16. Forsyth, D. (2009). Light in space. Tech. rep., UIUC, IL. Google Scholar
  17. Funt, B., Drew, M., & Brockington, M. (1992a). Recovering shading from color images. In ECCV. Google Scholar
  18. Funt, B. V., Drew, M. S., & Brockington, M. (1992b). Reovering shading from color images. In Proceedings of ECCV, S. Margherita Ligure, Italy (pp. 124–132). Google Scholar
  19. Gilchrist, A. (1977). Perceived lightness depends on perceived spatial arrangement. Science, 195, 185–187. CrossRefGoogle Scholar
  20. Gilchrist, A. (2006). Seeing black and white. London: Oxford University Press. CrossRefGoogle Scholar
  21. Grosse, R., Johnson, M. K., Adelson, E. H., & Freeman, W. T. (2009). Ground truth dataset and baseline evaluations for intrinsic image algorithms. In ICCV. Google Scholar
  22. Hordley, S. D. (2006). Scene illuminant estimation: past, present, and future. Color Research and Application, 31(4), 303–314. CrossRefGoogle Scholar
  23. Horn, B. K. P. (1974a). Determining lightness from an image. Computer Vision, Graphics and Image Processing, 3, 277–299. CrossRefGoogle Scholar
  24. Horn, B. K. P. (1974b). Determining lightness from an image. Computer Graphics and Image Processing, 3(4), 277–299. MathSciNetCrossRefGoogle Scholar
  25. Kay, S. M. (1993). Fundamentals of statistical signal processing. Estimation theory (Vol. I). Upper Saddle River: Prentice Hall. zbMATHGoogle Scholar
  26. Kimmel, R., Elad, M., Shaked, D., Keshet, R, & Sobel, I (2003). A variational framework for retinex. International Journal of Computer Vision, 52(1), 7–23. zbMATHCrossRefGoogle Scholar
  27. Koenderink, J. J., Pont, S. C., van Doorn, A. J., Kappers, A. M. L., & Todd, J. T. (2007). The visual light field. Perception, 36, 1595–1610. CrossRefGoogle Scholar
  28. Land, E., & McCann, J. (1971). Lightness and retinex theory. Journal of the Optical Society of America, 61(1), 1–11. CrossRefGoogle Scholar
  29. Lenz, R. (2002). Spaces of spectral distributions and their natural geometry. In Proceedings of CGIV, Poitiers, France (pp. 249–254). Google Scholar
  30. Martinez, K., Cupitt, J., Saunders, D., & Pillay, R. (2002). Ten years of art imaging research. Proceedings of the IEEE, 90(1), 28–41. CrossRefGoogle Scholar
  31. Mohen, J. P., Menu, M., & Mottin, B. (2006). Mona Lisa: inside the painting. New York: Harry N. Abrams Inc. Google Scholar
  32. Novati, G., Pellegri, P., & Schettini, R. (2005). An affordable multispectral imaging system for the digital museum. International Journal on Digital Libraries, 5(3), 167–178. CrossRefGoogle Scholar
  33. Paviotti, A., Ratti, F., Poletto, L., & Cortelazzo, G. M. (2009). Multispectral acquisition of large-sized pictorial surfaces. EURASIP Journal on Image and Video Processing. Google Scholar
  34. Pelagotti, A., Mastio, A. D., & Razionale, A. V. (2007). Active and passive sensors for art works analysis and investigations. In SPIE conference series (Vol. 6491). Google Scholar
  35. Pelagotti, A., Mastio, A. D., Rosa, A. D., & Piva, A. (2008). Multispectral imaging of paintings. IEEE Signal Processing Magazine, 25(4), 25–36. CrossRefGoogle Scholar
  36. Romeiro, F., Vasilyev, Y., & Zickler, T. (2008). Passive reflectometry. In Proceedings of ECCV (Vol. 4, pp. 859–872). Google Scholar
  37. Shen, L., Tan, P., & Lin, S. (2008). Intrinsic image decomposition with non-local texture cues. In CVPR. Google Scholar
  38. Tappen, M., Freeman, W., & Adelson, E. (2005). Recovering intrinsic images from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(9), 1459–1472. CrossRefGoogle Scholar
  39. Tappen, M., Freeman, W., & Adelson, E. (2006). Estimating intrinsic component images using non-linear regression. In CVPR. Google Scholar
  40. Weiss, Y. (2001). Deriving intrinsic images from image sequences. In ICCV. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Anna Paviotti
    • 1
    • 2
  • David A. Forsyth
    • 3
  • Guido M. Cortelazzo
    • 1
  1. 1.Department of Information EngineeringUniversity of PadovaPadovaItaly
  2. 2.Nidek Technologies SrlPadovaItaly
  3. 3.Computer Science DepartmentUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations