Skip to main content
SpringerLink
Log in

Artistic relighting of paintings and drawings

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We present a practical solution to the problem of subject relighting in paintings and drawings. Our interactive technique uses 3-D shading proxies and can be applied to objects with arbitrary geometries. Given a user-provided guess for the shading of an object in a painting/drawing and its corresponding target shading, we refine them using shading-color correlation and a multi-scale scheme. These refined shadings are then used to create a multi-channel shading-ratio image to perform relighting, while taking into account the colors used by the artists to convey shading information. We demonstrate the effectiveness of our solution on a variety of artistic styles, including paintings with strong brush strokes and unconventional shading encodings, drawings, and other types of artwork. Our method is the first to perform relighting of paintings and drawings and, in addition to relighting, can transfer smooth normal and depth maps from 3-D proxies to images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28

Similar content being viewed by others

References

  1. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., Cohen, M.: Interactive digital photomontage. ACM TOG 23(3), 294–302 (2004)

    Article  Google Scholar 

  2. Akers, D., Losasso, F., Klingner, J., Agrawala, M., Rick, J., Hanrahan, P.: Conveying shape and features with image-based relighting. In: VIS’03, pp. 46–51. IEEE Computer Society, Washington, DC (2003)

  3. Anrys, F., Dutré, P., Willems, Y.D.: Image-based lighting design. In: Proceedings of the 4th IASTED international conference on visualization, imaging, and image processing (2004)

  4. Anthropics: PortraitPro v12.4–Trial Version (2015). http://www.portraitprofessional.com/. Accessed June 2015

  5. Barron, J.T., Malik, J.: Color constancy, intrinsic images, and shape estimation. In: ECCV’12, pp. 57–70. Springer, Berlin, Heidelberg (2012)

  6. Barron, J.T., Malik, J.: Shape, illumination, and reflectance from shading. IEEE Trans Pattern Anal Mach Intell 37(8), 1670–1687 (2015). doi:10.1109/TPAMI.2014.2377712

  7. Basso, A., Graf, H., Gibbon, D., Cosatto, E., Liu, S.: Virtual light: digitally-generated lighting for video conferencing applications. In: ICIP’01, vol. 2, pp. 1085–1088 (2001)

  8. Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d faces. In: SIGGRAPH ’99, pp. 187–194. ACM Press, Addison-Wesley Publishing Co, New York (1999)

  9. Boivin, S., Gagalowicz, A.: Inverse rendering from a single image. In: CGIV’02, pp. 268–277 (2002)

  10. Chen, X., Chen, M., Jin, X., Zhao, Q.: Face illumination transfer through edge-preserving filters. In: CVPR ’11, pp. 281–287. IEEE (2011)

  11. Chen, T., Zhu, Z., Shamir, A., Hu, S.M., Cohen-Or, D.: 3Sweep: extracting editable objects from a single photo. ACM TOG 32(6), 195:1–195:10 (2013)

  12. Chen, X., Jin, X., Zhao, Q., Wu, H.: Artistic illumination transfer for portraits. CGF 31(4), 1425–1434 (2012)

    Google Scholar 

  13. Costa, A., Sousa, A., Nunes Ferreira, F.: Lighting design: A goal based approach using optimisation. In: EGWR’99, pp. 317–328. Springer, Vienna (1999)

  14. CrazyBump: Demo Version 1.2 (2014). http://www.crazybump.com/. Accessed Jan 2015

  15. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.P., Sarokin, W., Sagar, M.: Acquiring the reflectance field of a human face. In: SIGGRAPH ’00, pp. 145–156. ACM Press, Addison-Wesley Publishing Co, New York (2000)

  16. Hu, K., Liu, Y., Dong, Q., Liu, H., Xing, G.: Color face image decomposition under complex lighting conditions. Vis. Comput. 30(6–8), 685–695 (2014)

    Article  Google Scholar 

  17. Johnston, S.F.: Lumo: Illumination for cel animation. In: NPAR ’02, pp. 45-ff. ACM, New York (2002)

  18. Kholgade, N., Simon, T., Efros, A., Sheikh, Y.: 3d object manipulation in a single photograph using stock 3d models. ACM TOG 33(4), 127:1–127:12 (2014)

  19. Li, Q., Yin, W., Deng, Z.: Image-based face illumination transferring using logarithmic total variation models. Vis. Comput. 26(1), 41–49 (2009)

    Article  Google Scholar 

  20. Li, G., Wu, C., Stoll, C., Liu, Y., Varanasi, K., Dai, Q., Theobalt, C.: Capturing relightable human performances under general uncontrolled illumination. CGF 32(2), 275–284 (2013)

    Google Scholar 

  21. Lipman, Y., Levin, D., Cohen-Or, D.: Green coordinates. ACM TOG 27(3), 78:1–78:10 (2008)

  22. Lopez-Moreno, J., Jimenez, J., Hadap, S., Reinhard, E., Anjyo, K., Gutierrez, D.: Stylized depiction of images based on depth perception. In: NPAR ’10. ACM, New York (2010)

  23. Loscos, C., Drettakis, G., Robert, L.: Interactive virtual relighting of real scenes. IEEE TVCG 6(3), 289–305 (2000)

    Google Scholar 

  24. Malzbender, T., Gelb, D., Wolters, H.J.: Polynomial texture maps. In: SIGGRAPH’01, pp. 519–528. ACM, New York (2001)

  25. Marschner, S.R.: Inverse rendering for computer graphics. Ph.D. thesis, Cornell University (1998)

  26. Mortensen, E.N., Barrett, W.A.: Intelligent scissors for image composition. In: SIGGRAPH ’95, pp. 191–198. ACM, New York (1995)

  27. Okabe, M., Zeng, G., Matsushita, Y., Igarashi, T., Quan, L., yeung Shum, H.: Single-view relighting with normal map painting. In: Pacific Graphics, pp. 27–34 (2006)

  28. Peers, P., Tamura, N., Matusik, W., Debevec, P.: Post-production facial performance relighting using reflectance transfer. ACM TOG 26(3), 52:1–52:10 (2007)

  29. Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21(5), 34–41 (2001)

    Article  Google Scholar 

  30. Shapira, D., Avidan, S., Hel-Or, Y.: Multiple histogram matching. In: Proc. IEEE ICIP, pp. 2269–2273 (2013)

  31. Shashua, A., Riklin-Raviv, T.: The quotient image: Class-based re-rendering and recognition with varying illuminations. IEEE TPAMI 23(2), 129–139 (2001)

    Article  Google Scholar 

  32. Shi, J., Tomasi, C.: Good features to track. In: CVPR’94, pp. 593–600. IEEE (1994)

  33. Shih, Y., Paris, S., Barnes, C., Freeman, W.T., Durand, F.: Style transfer for headshot portraits. ACM TOG 33(4), 148:1–148:14 (2014)

  34. Suzuki, S., Abe, K.: Topological structural analysis of digitized binary images by border following. CVGIP 30(1), 32–46 (1985)

    MATH  Google Scholar 

  35. Sýkora, D., Sedlacek, D., Jinchao, S., Dingliana, J., Collins, S.: Adding depth to cartoons using sparse depth (in)equalities. CGF 29(2), 615–623 (2010)

  36. Sýkora, D., Kavan, L., Čadík, M., Jamriška, O., Jacobson, A., Whited, B., Simmons, M., Sorkine-Hornung, O.: Ink-and-ray: Bas-relief meshes for adding global illumination effects to hand-drawn characters. ACM TOG 33(2), 16 (2014)

    Article  Google Scholar 

  37. Tunwattanapong, B., Debevec, P.: Interactive image-based relighting with spatially-varying lights. In: ACM SIGGRAPH’09 Poster (2009)

  38. Tunwattanapong, B., Ghosh, A., Debevec, P.: Practical image-based relighting and editing with spherical-harmonics and local lights. In: CVMP’11, pp. 138–147. IEEE Computer Society, Washington, DC (2011)

  39. Wang, Y., Liu, Z., Hua, G., Wen, Z., Zhang, Z., Samaras, D.: Face re-lighting from a single image under harsh lighting conditions. In: CVPR’07, pp. 1–8. IEEE (2007)

  40. Wang, O., Davis, J., Chuang, E., Rickard, I., de Mesa, K., Dave, C.: Video relighting using infrared illumination. CGF 27(2), 271–279 (2008)

    Google Scholar 

  41. Welsh, T., Ashikhmin, M., Mueller, K.: Transferring color to greyscale images. ACM TOG 21(3), 277–280 (2002)

    Article  Google Scholar 

  42. Wen, Z., Liu, Z., Huang, T.S.: Face relighting with radiance environment maps. In: CVPR’03, pp. 158–165. IEEE (2003)

  43. Wu, T.P., Sun, J., Tang, C.K., Shum, H.Y.: Interactive normal reconstruction from a single image. ACM TOG 27(5), 119:1–119:9 (2008)

  44. Wu, T.P., Tang, C.K., Brown, M.S., Shum, H.Y.: Shapepalettes: interactive normal transfer via sketching. ACM TOG 26(3), 44:1–44:5 (2007)

  45. Wu, C., Varanasi, K., Liu, Y., Seidel, H.P., Theobalt, C.: Shading-based dynamic shape refinement from multi-view video under general illumination. In: IEEE ICCV, pp. 1108–1115. IEEE (2011)

  46. Wu, C., Wilburn, B., Matsushita, Y., Theobalt, C.: High-quality shape from multi-view stereo and shading under general illumination. In: IEEE CVPR, pp. 969–976. IEEE Computer Society, Washington, DC (2011)

  47. Yu, T., Wang, H., Ahuja, N., Chen, W.C.: Sparse lumigraph relighting by illumination and reflectance estimation from multi-view images. In: EGSR, pp. 41–50. Eurographics Association, Aire-la-Ville (2006)

Download references

Acknowledgments

This work was sponsored by CAPES and CNPq-Brazil (Grants 134134/2013-3, 306196/2014-0, and 482271/2012-4). We would like to thank the artists for allowing us to use their artworks.

Author information

Authors and Affiliations

  1. Instituto de Informática, UFRGS, Porto Alegre, RS, Brazil

    Bernardo Henz & Manuel M. Oliveira

Authors
  1. Bernardo Henz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manuel M. Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bernardo Henz.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Henz, B., Oliveira, M.M. Artistic relighting of paintings and drawings. Vis Comput 33, 33–46 (2017). https://doi.org/10.1007/s00371-015-1150-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-015-1150-7

Keywords

Search

Navigation