A Neural Network for Simultaneously Reconstructing Transparent and Opaque Surfaces

  • Mohamad Ivan Fanany
  • Itsuo Kumazawa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4142)


This paper presents a neural network (NN) to recover three-dimensional (3D) shape of an object from its multiple view images. The object may contain non-overlapping transparent and opaque surfaces. The challenge is to simultaneously reconstruct the transparent and opaque surfaces given only a limited number of views. By minimizing the pixel error between the output images of this NN and teacher images, we want to refine vertices position of an initial 3D polyhedron model to approximate the true shape of the object. For that purpose, we incorporate a ray tracing formulation into our NN’s mapping and learning. At the implementation stage, we develop a practical regularization learning method using texture mapping instead of ray tracing. By choosing an appropriate regularization parameter and optimizing using hierarchical learning and annealing strategies, our NN gives more approximate shape.


Texture Mapping Vertex Position True Shape Transparent Surface Transparent Object 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Matusik, M., Hanspeter, P., Zieglar, R., Ngan, N., McMillan, L.: Acquisition and Rendering of Transparent and Refractive Objects, Rendering Techniques, 267–278 (2002)Google Scholar
  2. 2.
    Zongker, D.E., Werner, D.M., Curless, B., Salesin, D.: Environment Matting and Compositing. In: SIGGRAPH, pp. 205–214 (1999)Google Scholar
  3. 3.
    Bolle, R.M., Vemuri, B.C.: On three-dimensional surface reconstruction methods. IEEE Trans. on PAMI 13(1), 1–13 (1991)Google Scholar
  4. 4.
    Saito, M., Kashiwagi, H., Sato, Y., Ikeuchi, K.: Measurement of Surface Orientations of Transparent Objects Using Polarization in Highlight. In: CVPR, p. 1381- (1999)Google Scholar
  5. 5.
    Interrante, V., Fuchs, H., Pizer, S.M.: Conveying the 3D Shape of Smoothly Curving Transparent Surfaces via Texture. IEEE Trans. on VCG 3(2) (1997)Google Scholar
  6. 6.
    Murase, H.: Surface Shape Reconstruction of a Nonrigid Transport Object Using Refraction and Motion. IEEE Trans. on PAMI 14(10), 1045–1052 (1992)Google Scholar
  7. 7.
    Hata, S., Saitoh, Y., Kumamura, S., Kaida, K.: Shape Extraction of Transparent Object Using Genetic Algorithm. In: ICPR 1996, D93.6 (1996)Google Scholar
  8. 8.
    Wolff, L.B.: Shape from Polarization Images. In: CVWS 1997, pp. 79–85 (1987)Google Scholar
  9. 9.
    Fanany, M.I., Kobayashi, K., Kumazawa, I.: A combinatorial transparent surface modeling from polarization images. In: Klette, R., Žunić, J. (eds.) IWCIA 2004. LNCS, vol. 3322, pp. 65–76. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  10. 10.
    Miyazaki, D., Saito, M., Sato, Y., Ikeuchi, K.: Determining surface orientations of transparent objects based on polarization degrees in visible and infrared wavelengths. JOSA-A 19(4) (2002)Google Scholar
  11. 11.
    Miyazaki, D., Kagesawa, M., Ikeuchi, K.: Polarization-based Transparent Surface Modeling from Two Views. In: ICCV, pp. 1381–1386 (2003)Google Scholar
  12. 12.
    Rahman, S., Centerakis, N.: Reconstruction of Specular Surfaces using Polarization Imaging. In: Figueiredo, M., Zerubia, J., Jain, A.K. (eds.) EMMCVPR 2001. LNCS, vol. 2134, pp. 149–155. Springer, Heidelberg (2001)Google Scholar
  13. 13.
    Chuang, Y., Zongker, E., Hindorff, J., Curless, B., Salesin, D., Szeliski, R.: Environment matting extensions: towards higher accuracy. In: SIGGRAPH, pp. 121–130 (2000)Google Scholar
  14. 14.
    Wexler, Y., Fitzgibbon, A.W., Zisserman, A.: Image-based Environment matting. Rendering Techniques, 279–290 (2002)Google Scholar
  15. 15.
    Szeliski, R.: Avidan, Shai., Anandan, P.: Layer Extraction from Multiple Images Containing Reflections and Transparency. In: CVPR, p. 1246- (2000)Google Scholar
  16. 16.
    Fanany, M.I., Kumazawa, I.: Multiple-View Shape Extraction from Shading as Local Regression by Analytic NN Scheme. Mathematical and Computer Modelling 40(9-10), 959–975 (2004)MATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Fanany, M.I., Kumazawa, I.: A neural network for recovering 3D shape from erroneous and few depth maps of shaded images. Pattern Recognition Letters 25(4), 377–389 (2004)CrossRefMATHGoogle Scholar
  18. 18.
    Fanany, M.I., Ohno, M., Kumazawa, I.: A Scheme for Reconstructing Face from Shading using Smooth Projected Polygon NN. In: ICIP, vol. II, pp. 305–308 (2002)Google Scholar
  19. 19.
    Smith, A.R., Blinn, J.F.: Blue Screen Matting. In: SIGGRAPH, pp. 259-268 (1996)Google Scholar
  20. 20.
    Chen, Z., Haykin, S.: On Different Facets of Regularization Theory. Neural Computation 14(12), 2791–2846 (2002)MATHCrossRefGoogle Scholar
  21. 21.
    Hearn, D., Baker, M.P.: Computer Graphics: C Version. Prentice Hall, Upper Saddle River (1998)Google Scholar
  22. 22.
    Wolff, B., Kurlander, D.J.: Ray Tracing with Polarization Parameters, IEEE Computer Graphics Appl. 10(6), 44–55 (1990)CrossRefGoogle Scholar
  23. 23.
    Romeo, F., Vincentelli, A.S.: A theoretical framework for Simulated Annealing. Algorithmica 6, 302–345 (1991)MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Mohamad Ivan Fanany
    • 1
  • Itsuo Kumazawa
    • 1
  1. 1.Imaging Science and EngineeringTokyo Institute of Technology 

Personalised recommendations