On Shape and Material Recovery from Motion

  • Manmohan Chandraker
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8695)


We present a framework for the joint recovery of the shape and reflectance of an object with dichromatic BRDF, using motion cues. We show that four (small or differential) motions of the object, or three motions of the camera, suffice to yield a linear system that decouples shape and BRDF. The theoretical benefit is that precise limits on shape and reflectance recovery using motion cues may be derived. We show that shape may be recovered for unknown isotropic BRDF and light source. Simultaneous reflectance estimation is shown ambiguous for general isotropic BRDFs, but possible for restricted BRDFs representing commong materials like metals, plastics and paints. The practical benefit of the decoupling is that joint shape and BRDF recovery need not rely on alternating methods, or restrictive priors. Further, our theory yields conditions for the joint estimability of shape, albedo, BRDF and directional lighting using motion cues. Surprisingly, such problems are shown to be well-posed even for some non-Lambertian material types. Experiments on measured BRDFs from the MERL database validate our theory.


Object Motion Camera Motion Shape Recovery Photometric Stereo Surface Depth 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Alldrin, N., Zickler, T., Kriegman, D.: Photometric stereo with non-parametric and spatially-varying reflectance. In: CVPR (2008)Google Scholar
  2. 2.
    Barron, J.T., Malik, J.: Shape, albedo, and illumination from a single image of an unknown object. In: CVPR, pp. 334–341 (2012)Google Scholar
  3. 3.
    Blinn, J.F., Newell, M.E.: Texture and reflection in computer generated images. Comm. ACM 19, 542–547 (1976)CrossRefGoogle Scholar
  4. 4.
    Canas, G.D., Vasilyev, Y., Adato, Y., Zickler, T., Gortler, S.J., Ben-Shahar, O.: A linear formulation of shape from specular flow. In: ICCV, pp. 191–198 (2009)Google Scholar
  5. 5.
    Chandraker, M.: What camera motion reveals about shape with unknown BRDF. In: CVPR, pp. 2179–2186 (2014)Google Scholar
  6. 6.
    Chandraker, M., Bai, J., Ramamoorthi, R.: On differential photometric reconstruction for unknown, isotropic BRDFs. PAMI 35(12), 2941–2955 (2013)CrossRefGoogle Scholar
  7. 7.
    Chandraker, M., Ramamoorthi, R.: What an image reveals about material reflectance. In: ICCV, pp. 1076–1083 (2011)Google Scholar
  8. 8.
    Chandraker, M., Reddy, D., Wang, Y., Ramamoorthi, R.: What object motion reveals about shape with unknown BRDF and lighting. In: CVPR, pp. 2523–2530 (2013)Google Scholar
  9. 9.
    Chandraker, M.: On joint shape and material recovery from motion cues. Tech. rep., NEC Labs America (2014)Google Scholar
  10. 10.
    Goldman, D.B., Curless, B., Hertzmann, A., Seitz, S.M.: Shape and spatially-varying BRDFs from photometric stereo. PAMI 32(6), 1060–1071 (2010)CrossRefGoogle Scholar
  11. 11.
    Horn, B., Schunck, B.: Determining optical flow. Art. Intell. 17, 185–203 (1981)CrossRefGoogle Scholar
  12. 12.
    Lawrence, J., Ben-Artzi, A., Decoro, C., Matusik, W., Pfister, H., Ramamoorthi, R., Rusinkiewicz, S.: Inverse shade trees for non-parametric material representation and editing. In: ACM ToG (SIGGRAPH), pp. 735–745 (2006)Google Scholar
  13. 13.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Image Understanding Workshop, pp. 121–130 (1981)Google Scholar
  14. 14.
    Matusik, W., Pfister, H., Brand, M., McMillan, L.: A data-driven reflectance model. ToG 22(3), 759–769 (2003)CrossRefGoogle Scholar
  15. 15.
    Nagel, H.H.: On a constraint equation for the estimation of displacement rates in image sequences. PAMI 11(1), 13–30 (1989)CrossRefzbMATHGoogle Scholar
  16. 16.
    Ngan, A., Durand, F., Matusik, W.: Experimental analysis of BRDF models. In: EGSR, pp. 117–126 (2005)Google Scholar
  17. 17.
    Oxholm, G., Nishino, K.: Shape and reflectance from natural illumination. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part I. LNCS, vol. 7572, pp. 528–541. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  18. 18.
    Romeiro, F., Zickler, T.: Blind reflectometry. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 45–58. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  19. 19.
    Romeiro, F., Vasilyev, Y., Zickler, T.: Passive reflectometry. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 859–872. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  20. 20.
    Sato, I., Okabe, T., Yu, Q., Sato, Y.: Shape reconstruction based on similarity in radiance changes under varying illumination. In: ICCV, pp. 1–8 (2007)Google Scholar
  21. 21.
    Seitz, S., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multiview stereo algorithms. In: CVPR, pp. 519–526 (2006)Google Scholar
  22. 22.
    Torrance, K.E., Sparrow, E.M.: Theory for off-specular reflection from roughened surfaces. JOSA 57, 1105–1112 (1967)CrossRefGoogle Scholar
  23. 23.
    Verri, A., Poggio, T.: Motion field and optical flow: Qualitative properties. PAMI 11(5), 490–498 (1989)CrossRefGoogle Scholar
  24. 24.
    Zhang, L., Curless, B., Hertzmann, A., Seitz, S.: Shape from motion under varying illumination. In: ICCV, pp. 618–625 (2003)Google Scholar
  25. 25.
    Zickler, T., Belhumeur, P., Kriegman, D.: Helmholtz stereopsis: Exploiting reciprocity for surface reconstruction. IJCV 49(2/3), 1215–1227 (2002)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Manmohan Chandraker
    • 1
  1. 1.NEC Labs AmericaCupertinoUSA

Personalised recommendations