Image-Based Relighting of Moving Objects with Specular Reflection

  • Hanhoon Park
  • Jong-Il Park
  • Sang Hwa Lee
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3617)


In the fields of Augmented Reality (AR) and Virtual Reality (VR), inserting an object into a scene (real or virtual) requires proper matching of their lighting conditions. If not, the resulting image may look unnatural. In particular, it is important to describe the position and shape of specular reflection accurately if the object has specular reflection. In this paper, we propose an approach to relighting a moving object based on the separation of specular and diffuse reflection. To relight an object, two or more images taken under the condition that the position of the object is fixed but the lighting condition is different, we call synchronized images, are required. However, it is impossible to obtain such images in case of a moving object. Therefore, we propose a method that computationally obtains the synchronized images using the consecutive fields of a controlled video sequence containing a moving object. For example, if the virtual (n + 1) – th field is interpolated from nth field and (n + 2) – th field using the motion compensation technique, both the virtual (n + 1) – th field and the real (n + 1) – th field have the condition that the position of the object is fixed. If the virtual and real image have different lighting condition, the method applied to static object is applicable to moving object as it is. After the specular and diffuse reflection are separted, the relit image is synthesized using the linear interpolation and morphing technique. The experimental results of applying the proposed method to real and synthetic images are given. We verify the effectiveness of the proposed method by comparing the resulting image with a ground-truth image.


Lighting Condition Video Sequence Augmented Reality Static Object Specular Reflection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Fournier, A., Gunawan, S., Romanzin, C.: Common Illumination between Real and Computer Generated Scenes. In: Proc. of Graphics Interface, pp. 254–262 (1993)Google Scholar
  2. 2.
    Gibson, S., Murta, A.: Interactive Rendering with Real-World Illumination. In: Proc. of Eurographics Workshop on Rendering, pp. 365–376 (2000)Google Scholar
  3. 3.
    Loscos, C., Drettakis, G., Robert, L.: Interactive Virtual Relighting of Real Scenes. IEEE Transactions on Visualization and Computer Graphics, 289–305 (2000)Google Scholar
  4. 4.
    Kim, H.: Virtualizing Facial Images Considering Lighting Condition. MS thesis, Hanyang University, Seoul, Korea (2001)Google Scholar
  5. 5.
    Klinker, G.J., Shafer, S.A., Kanade, T.: The Measurement of Highlights in Color Images. International Journal of Computer Vision, 7–32 (1990)Google Scholar
  6. 6.
    Lin, S., Shum, H.Y.: Separation of Diffuse and Specular Reflection in Color Images. In: Proc. of CVPR, pp. 341–346 (2001)Google Scholar
  7. 7.
    Wolff, L.B.: Using Polarization to Separate Reflection Components. In: Proc. of CVPR, pp. 363–369 (1989)Google Scholar
  8. 8.
    Nayar, S.K., Fang, X., Boult, T.E.: Removal of Specularities Using Color and Polarization. In: Proc. of CVPR, pp. 583–590 (1993)Google Scholar
  9. 9.
    Lee, B.-H.: Image-Based Relighting Using Separation of Specular and Diffuse Reflectance. MS thesis, Hanyang Univesity, Seoul, Korea (2004) (in Korean)Google Scholar
  10. 10.
    Lee, H.-C.: Method for Computing the Scene-Illuminant Chromaticity from Specular Highlights. Journal of the Optical Society of America A, 1694–1699 (1986)Google Scholar
  11. 11.
    Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., Sagar, M.: Acquiring the Reflectance Field of a Human Face. In: Proc. of SIGGRAPH, pp. 145–156 (2000)Google Scholar
  12. 12.
    Haeberli, P.: Synthetic Lighting for Photography (1992), Available on
  13. 13.
    Wong, T.-T., Heng, P.-A., Or, S.-H., Ng, W.-Y.: Image-Based Rendering with Controllable Illumination. In: Proc. of Eurographics Workshop on Rendering, pp. 13–22 (1997)Google Scholar
  14. 14.
    Shafer, S.A.: Using Color to Separate Reflection Components. Color Research and Applications, 210–218 (1985)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Hanhoon Park
    • 1
  • Jong-Il Park
    • 1
  • Sang Hwa Lee
    • 2
  1. 1.Division of Electrical and Computer EngineeringHanyang UniversitySeoulKorea
  2. 2.Department of Electrical EngineeringSeoul National UniversitySeoulKorea

Personalised recommendations