Material Classification Based on Training Data Synthesized Using a BTF Database

  • Michael Weinmann
  • Juergen Gall
  • Reinhard Klein
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8691)


To cope with the richness in appearance variation found in real-world data under natural illumination, we propose to synthesize training data capturing these variations for material classification. Using synthetic training data created from separately acquired material and illumination characteristics allows to overcome the problems of existing material databases which only include a tiny fraction of the possible real-world conditions under controlled laboratory environments. However, it is essential to utilize a representation for material appearance which preserves fine details in the reflectance behavior of the digitized materials. As BRDFs are not sufficient for many materials due to the lack of modeling mesoscopic effects, we present a high-quality BTF database with 22,801 densely measured view-light configurations including surface geometry measurements for each of the 84 measured material samples. This representation is used to generate a database of synthesized images depicting the materials under different view-light conditions with their characteristic surface geometry using image-based lighting to simulate the complexity of real-world scenarios. We demonstrate that our synthesized data allows classifying materials under complex real-world scenarios.


Material classification material database reflectance texture synthesis 

Supplementary material

978-3-319-10578-9_11_MOESM1_ESM.pdf (86.7 mb)
Electronic Supplementary Material (PDF 88,744 KB)


  1. 1.
    (November 2013),
  2. 2.
    Barron, J.T., Malik, J.: Shape, albedo, and illumination from a single image of an unknown object. In: CVPR, pp. 334–341 (2012)Google Scholar
  3. 3.
    Barron, J., Malik, J.: Intrinsic scene properties from a single rgb-d image. In: CVPR, pp. 17–24 (2013)Google Scholar
  4. 4.
    Ben-Artzi, A., Ramamoorthi, R., Agrawala, M.: Efficient shadows for sampled environment maps. J. Graphics Tools 11(1), 13–36 (2006)CrossRefGoogle Scholar
  5. 5.
    Bosch, A., Zisserman, A., Muñoz, X.: Scene classification via pLSA. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 517–530. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  7. 7.
    Caputo, B., Hayman, E., Mallikarjuna, P.: Class-specific material categorisation. In: ICCV, vol. 2, pp. 1597–1604 (2005)Google Scholar
  8. 8.
    Dana, K.J., van Ginneken, B., Nayar, S.K., Koenderink, J.J.: Reflectance and texture of real world surfaces. Tech. rep. (1996)Google Scholar
  9. 9.
    Dana, K.J., Nayar, S.K., Ginneken, B.V., Koenderink, J.J.: Reflectance and texture of real-world surfaces. In: CVPR, pp. 151–157 (1997)Google Scholar
  10. 10.
    Debevec, P.: Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: SIGGRAPH, pp. 189–198 (1998)Google Scholar
  11. 11.
    Enzweiler, M., Gavrila, D.M.: A mixed generative-discriminative framework for pedestrian classification. In: CVPR, pp. 1–8 (2008)Google Scholar
  12. 12.
    Filip, J., Haindl, M.: Bidirectional texture function modeling: A state of the art survey. IEEE Trans. Pattern Anal. Mach. Intell. 31, 1921–1940 (2009)CrossRefGoogle Scholar
  13. 13.
    Hayman, E., Caputo, B., Fritz, M., Eklundh, J.-O.: On the significance of real-world conditions for material classification. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 253–266. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  14. 14.
    Jegou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: CVPR, pp. 3304–3311 (2010)Google Scholar
  15. 15.
    Kaneva, B., Torralba, A., Freeman, W.T.: Evaluation of image features using a photorealistic virtual world. In: ICCV, pp. 2282–2289 (2011)Google Scholar
  16. 16.
    Li, W., Fritz, M.: Recognizing materials from virtual examples. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part IV. LNCS, vol. 7575, pp. 345–358. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  17. 17.
    Liu, C., Sharan, L., Adelson, E.H., Rosenholtz, R.: Exploring features in a bayesian framework for material recognition. In: CVPR, pp. 239–246 (2010)Google Scholar
  18. 18.
    Liu, C., Yang, G., Gu, J.: Learning discriminative illumination and filters for raw material classification with optimal projections of bidirectional texture functions. In: CVPR, pp. 1430–1437 (2013)Google Scholar
  19. 19.
    Oxholm, G., Nishino, K.: Shape and reflectance from natural illumination. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part I. LNCS, vol. 7572, pp. 528–541. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  20. 20.
    Perronnin, F., Dance, C.R.: Fisher kernels on visual vocabularies for image categorization. In: CVPR (2007)Google Scholar
  21. 21.
    Pishchulin, L., Jain, A., Wojek, C., Andriluka, M., Thormählen, T., Schiele, B.: Learning people detection models from few training samples. In: CVPR, pp. 1473–1480 (2011)Google Scholar
  22. 22.
    Reeves, W.T., Salesin, D.H., Cook, R.L.: Rendering antialiased shadows with depth maps. SIGGRAPH Comput. Graph. 21(4), 283–291 (1987)CrossRefGoogle Scholar
  23. 23.
    Ruiters, R., Schwartz, C., Klein, R.: Example-based interpolation and synthesis of bidirectional texture functions. Computer Graphics Forum (Proceedings of the Eurographics 2013) 32(2), 361–370 (2013)CrossRefGoogle Scholar
  24. 24.
    Schwartz, C., Weinmann, M., Ruiters, R., Klein, R.: Integrated high-quality acquisition of geometry and appearance for cultural heritage. In: The 12th International Symposium on Virtual Reality, Archeology and Cultural Heritage VAST 2011, pp. 25–32 (2011)Google Scholar
  25. 25.
    Sharan, L., Rosenholtz, R., Adelson, E.H.: Material perception: What can you see in a brief glance? Journal of Vision 8 (2009)Google Scholar
  26. 26.
    Sharan, L., Liu, C., Rosenholtz, R., Adelson, E.H.: Recognizing materials using perceptually inspired features. IJCV 103(3), 348–371 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Shotton, J., Fitzgibbon, A.W., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: CVPR, pp. 1297–1304 (2011)Google Scholar
  28. 28.
    Stark, M., Goesele, M., Schiele, B.: Back to the future: Learning shape models from 3d cad data. In: BMVC. pp. 106.1–106.11 (2010)Google Scholar
  29. 29.
    Targhi, A.T., Geusebroek, J.-M., Zisserman, A.: Texture classification with minimal training images. In: International Conference on Pattern Recognition, pp. 1–4 (2008)Google Scholar
  30. 30.
    Timofte, R., Van Gool, L.: A training-free classification framework for textures, writers, and materials. In: BMVC, pp. 1–12 (2012)Google Scholar
  31. 31.
    Vapnik, V.N.: The nature of statistical learning theory. Springer-Verlag New York, Inc., New York (1995)CrossRefzbMATHGoogle Scholar
  32. 32.
    Varma, M., Zisserman, A.: A statistical approach to material classification using image patch exemplars. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 2032–2047 (2009)CrossRefGoogle Scholar
  33. 33.
    Wang, J., Dana, K.J.: Hybrid textons: Modeling surfaces with reflectance and geometry. In: CVPR, vol. 1, pp. 372–378.Google Scholar
  34. 34.
    Wenzel, J.: Mitsuba renderer (2010),
  35. 35.
    Wong, T.-T., Heng, P.-A., Or, S.-H., Ng, W.-Y.: Image-based rendering with controllable illumination. In: EGWR, pp. 13–22 (1997)Google Scholar
  36. 36.
    Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object categories: A comprehensive study. IJCV 73(2), 213–238 (2007)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Michael Weinmann
    • 1
  • Juergen Gall
    • 1
  • Reinhard Klein
    • 1
  1. 1.Institute of Computer ScienceUniversity of BonnGermany

Personalised recommendations