Abstract
Inverse graphics attempts to take sensor data and infer 3D geometry, illumination, materials, and motions such that a graphics renderer could realistically reproduce the observed scene. Renderers, however, are designed to solve the forward process of image synthesis. To go in the other direction, we propose an approximate differentiable renderer (DR) that explicitly models the relationship between changes in model parameters and image observations. We describe a publicly available OpenDR framework that makes it easy to express a forward graphics model and then automatically obtain derivatives with respect to the model parameters and to optimize over them. Built on a new auto-differentiation package and OpenGL, OpenDR provides a local optimization method that can be incorporated into probabilistic programming frameworks. We demonstrate the power and simplicity of programming with OpenDR by using it to solve the problem of estimating human body shape from Kinect depth and RGB data.
Chapter PDF
Similar content being viewed by others
References
Allen, B., Curless, B., Popović, Z.: The space of human body shapes: Reconstruction and parameterization from range scans. ACM Trans. Graph. 22(3), 587–594 (2003)
Bastian, J.W.: Reconstructing 3D geometry from multiple images via inverse rendering. Ph.D. thesis, University of Adelaide (2008)
Baumgart, B.G.: Geometric modeling for computer vision. Tech. Rep. AI Lab Memo AIM-249, Stanford University (Oct 1974)
Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., Bengio, Y.: Theano: a CPU and GPU math expression compiler. In: Proceedings of the Python for Scientific Computing Conference (SciPy) (June 2010)
Bertero, M., Poggio, T., Torre, V.: Ill-posed problems in early vision. Proc. IEEE 76(8), 869–889 (1988)
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1999, pp. 187–194. ACM Press/Addison-Wesley Publishing Co., New York (1999)
Bradski, G., Kaehler, A.: Learning OpenCV. O’Reilly Media Inc. (2008), http://oreilly.com/catalog/9780596516130
Goodman, N., Mansinghka, V., Roy, D., Bonawitz, K., Tenenbaum, J.: Church: A language for generative models. In: McAllester, D.A., Myllymäki, P. (eds.) Proc. Uncertainty in Artificial Intelligence (UAI), pp. 220–229 (July 2008)
Grenander, U.: Lectures in Pattern Theory I, II and III: Pattern Analysis, Pattern Synthesis and Regular Structures. Springer, Heidelberg (1976–1981)
Griewank, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. Frontiers in Appl. Math., vol. 19. SIAM, Philadelphia (2000)
Heikkila, J., Silven, O.: A four-step camera calibration procedure with implicit image correction. In: Proceedings of 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1106–1112 (June 1997)
Hirshberg, D.A., Loper, M., Rachlin, E., Black, M.J.: Coregistration: Simultaneous alignment and modeling of articulated 3D shape. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 242–255. Springer, Heidelberg (2012)
Horn, B.: Understanding image intensities. Artificial Intelligence 8, 201–231 (1977)
Jalobeanu, A., Kuehnel, F., Stutz, J.: Modeling images of natural 3D surfaces: Overview and potential applications. In: Conference on Computer Vision and Pattern Recognition Workshop, CVPRW 2004, pp. 188–188 (June 2004)
Jampani, V., Nowozin, S., Loper, M., Gehler, P.V.: The informed sampler: A discriminative approach to Bayesian inference in generative computer vision models. CoRR abs/1402.0859 (February 2014), http://arxiv.org/abs/1402.0859
Jones, E., Oliphant, T., Peterson, P., et al.: SciPy: Open source scientific tools for Python (2001), http://www.scipy.org/
Jones, M.J., Poggio, T.: Model-based matching by linear combinations of prototypes. In: A.I. Memo 1583, pp. 1357–1365. MIT (1996)
Kersten, D.: Inverse 3-D graphics: A metaphor for visual perception. Behavior Research Methods, Instruments & Computers 29(1), 37–46 (1997)
Knill, D.C., Richards, W.: Perception as Bayesian Inference. The Press Syndicate of the University of Cambridge, Cambridge (1996)
Gorce, M.d.L., Paragios, N., Fleet, D.J.: Model-based hand tracking with texture, shading and self-occlusions. In: 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008), Anchorage, Alaska, USA, June 24-26. IEEE Computer Society (2008)
Lee, A.D.: ad: a python package for first- and second-order automatic differentation (2012), http://pythonhosted.org/ad/
Mansinghka, V., Kulkarni, T.D., Perov, Y.N., Tenenbaum, J.: Approximate Bayesian image interpretation using generative probabilistic graphics programs. In: Burges, C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K. (eds.) Advances in Neural Information Processing Systems 26, pp. 1520–1528 (2013)
Minka, T., Winn, J., Guiver, J., Knowles, D.: Infer.NET 2.4. Microsoft Research Cambridge (2010), http://research.microsoft.com/infernet.
Mumford, D.: Neuronal architectures for pattern-theoretic problems. In: Koch, C., Davis, J.L. (eds.) Large-scale Neuronal theories of the Brain, pp. 125–152. Bradford (1994)
Oliphant, T.E.: Python for scientific computing. Computing in Science and Engineering 9(3), 10–20 (2007)
Patow, G., Pueyo, X.: A survey of inverse rendering problems. Computer Graphics Forum 22(4), 663–687 (2003)
Richards, W.: Natural Computation. The MIT Press (A Bradford Book), Cambridge (1988)
Robinette, K., Blackwell, S., Daanen, H., Boehmer, M., Fleming, S., Brill, T., Hoeferlin, D., Burnsides, D.: Civilian American and European Surface Anthropometry Resource (CAESAR) final report. Tech. Rep. AFRL-HE-WP-TR-2002-0169, US Air Force Research Laboratory (2002)
Smelyansky, V.N., Morris, R.D., Kuehnel, F.O., Maluf, D.A., Cheeseman, P.: Dramatic improvements to feature based stereo. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part II. LNCS, vol. 2351, pp. 247–261. Springer, Heidelberg (2002)
Sminchisescu, C., Telea, A.: A framework for generic state estimation in computer vision applications. In: International Conference on Computer Vision Systems (ICVS), pp. 21–34 (2001)
Sminchisescu, C.: Estimation algorithms for ambiguous visual models: Three dimensional human modeling and motion reconstruction in monocular video sequences. Ph.D. thesis, Inst. National Polytechnique de Grenoble (July 2002)
Terzopoulos, D.: Regularization of inverse visual problems involving discontinuities. IEEE PAMI 8(4), 413–424 (1986)
Terzopoulos, D.: Physically-based modeling: Past, present, and future. In: ACM SIGGRAPH 89 Panel Proceedings, pp. 191–209 (1989)
Viola, F., Fitzgibbon, A., Cipolla, R.: A unifying resolution-independent formulation for early vision. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 494–501 (June 2012)
Walter, S.F.: Pyadolc 2.4.1 (2012), https://github.com/b45ch1/pyadolc
Weiss, A., Hirshberg, D., Black, M.: Home 3D body scans from noisy image and range data. In: Int. Conf. on Computer Vision (ICCV), pp. 1951–1958. IEEE, Barcelona (2011)
Wood, F., van de Meent, J.W., Mansinghka, V.: A new approach to probabilistic programming inference. In: Artificial Intelligence and Statistics (2014)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Loper, M.M., Black, M.J. (2014). OpenDR: An Approximate Differentiable Renderer. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8695. Springer, Cham. https://doi.org/10.1007/978-3-319-10584-0_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-10584-0_11
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10583-3
Online ISBN: 978-3-319-10584-0
eBook Packages: Computer ScienceComputer Science (R0)