View Dependent Surface Material Recognition

  • Stanislav Mikeš
  • Michal HaindlEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11844)


The paper presents a detailed study of surface material recognition dependence on the illumination and viewing conditions which is a hard challenge in a realistic scene interpretation. The results document sharp classification accuracy decrease when using usual texture recognition approach, i.e., small learning set size and the vertical viewing and illumination angle which is a very inadequate representation of the enormous material appearance variability. The visual appearance of materials is considered in the state-of-the-art Bidirectional Texture Function (BTF) representation and measured using the upper-end BTF gonioreflectometer. The materials in this study are sixty-five different wood species. The supervised material recognition uses the shallow convolutional neural network (CNN) for the error analysis of angular dependency. We propose a Gaussian mixture model-based method for robust material segmentation.



The Czech Science Foundation project GAČR 19-12340S supported this research.


  1. 1.
    TensorFlow. Technical report, Google AI.
  2. 2.
    Dana, K.J., van Ginneken, B., Nayar, S.K., Koenderink, J.J.: Reflectance and texture of real world surfaces. Technical report CUCS-048-96, Columbia University, December 1996Google Scholar
  3. 3.
    Dana, K.J., Nayar, S.K., van Ginneken, B., Koenderink, J.J.: Reflectance and texture of real-world surfaces. In: CVPR, pp. 151–157. IEEE Computer Society (1997)Google Scholar
  4. 4.
    Haindl, M., Mikeš, S.: Unsupervised texture segmentation using multispectral modelling approach. In: Tang, Y., Wang, S., Yeung, D., Yan, H., Lorette, G. (eds.) Proceedings of the 18th International Conference on Pattern Recognition, ICPR 2006, vol. II, pp. 203–206. IEEE Computer Society, Los Alamitos, August 2006Google Scholar
  5. 5.
    Haindl, M., Mikeš, S.: Texture segmentation benchmark. In: Lovell, B., Laurendeau, D., Duin, R. (eds.) Proceedings of the 19th International Conference on Pattern Recognition, ICPR 2008, pp. 1–4. IEEE Computer Society, Los Alamitos, December 2008Google Scholar
  6. 6.
    Haindl, M., Filip, J.: Visual Texture. ACVPR. Springer, London (2013). Scholar
  7. 7.
    Haindl, M., Mikeš, S.: A competition in unsupervised color image segmentation. Pattern Recogn. 57(9), 136–151 (2016)CrossRefGoogle Scholar
  8. 8.
    Haindl, M., Mikeš, S., Kudo, M.: Unsupervised surface reflectance field multi-segmenter. In: Azzopardi, G., Petkov, N. (eds.) CAIP 2015. LNCS, vol. 9256, pp. 261–273. Springer, Cham (2015). Scholar
  9. 9.
    Hayman, E., Caputo, B., Fritz, M., Eklundh, J.-O.: On the significance of real-world conditions for material classification. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 253–266. Springer, Heidelberg (2004). Scholar
  10. 10.
    Jehle, M., Sommer, C., Jähne, B.: Learning of optimal illumination for material classification. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) DAGM 2010. LNCS, vol. 6376, pp. 563–572. Springer, Heidelberg (2010). Scholar
  11. 11.
    Kampouris, C., Zafeiriou, S., Ghosh, A., Malassiotis, S.: Fine-grained material classification using micro-geometry and reflectance. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 778–792. Springer, Cham (2016). Scholar
  12. 12.
    Koudelka, M.L., Magda, S., Belhumeur, P.N., Kriegman, D.J.: Acquisition, compression, and synthesis of bidirectional texture functions. In: Texture 2003: Third International Workshop on Texture Analysis and Synthesis, Nice, France, pp. 59–64, October 2003Google Scholar
  13. 13.
    Krizhevsky, A.: Learning multiple layers of features from tiny images. Master’s thesis, University of Toronto, Canada (2009)Google Scholar
  14. 14.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  15. 15.
    Liu, C., Gu, J.: Discriminative illumination: per-pixel classification of raw materials based on optimal projections of spectral BRDF. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 86–98 (2014)CrossRefGoogle Scholar
  16. 16.
    Liu, C., Yang, G., Gu, J.: Learning discriminative illumination and filters for raw material classification with optimal projections of bidirectional texture functions. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2013Google Scholar
  17. 17.
    Lu, F., He, L., You, S., Chen, X., Hao, Z.: Identifying surface BRDF from a single 4-D light field image via deep neural network. IEEE J. Sel. Top. Sig. Process. 11(7), 1047–1057 (2017)CrossRefGoogle Scholar
  18. 18.
    Müller, G., Meseth, J., Sattler, M., Sarlette, R., Klein, R.: Acquisition, synthesis and rendering of bidirectional texture functions. In: Eurographics 2004, STAR - State of The Art Report, pp. 69–94. Eurographics Association (2004)Google Scholar
  19. 19.
    Ngan, A., Durand, F.: Statistical acquisition of texture appearance. In: Eurographics Symposium on Rendering. Eurographics (2006)Google Scholar
  20. 20.
    Pattanayak, S.: Pro Deep Learning with TensorFlow. Apress, New York (2017)CrossRefGoogle Scholar
  21. 21.
    Sattler, M., Sarlette, R., Klein, R.: Efficient and realistic visualization of cloth. In: Eurographics Symposium on Rendering, June 2003Google Scholar
  22. 22.
    Wang, J., Dana, K.: Relief texture from specularities. IEEE Trans. Pattern Anal. Mach. Intell. 28(3), 446–457 (2006)CrossRefGoogle Scholar
  23. 23.
    Wang, T.-C., Zhu, J.-Y., Hiroaki, E., Chandraker, M., Efros, A.A., Ramamoorthi, R.: A 4D light-field dataset and CNN architectures for material recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 121–138. Springer, Cham (2016). Scholar
  24. 24.
    Weinmann, M., Gall, J., Klein, R.: Material classification based on training data synthesized using a BTF database. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 156–171. Springer, Cham (2014). Scholar
  25. 25.
    Weinmann, M., Klein, R.: Material recognition for efficient acquisition of geometry and reflectance. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8927, pp. 321–333. Springer, Cham (2015). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.The Institute of Information Theory and Automation of the Czech Academy of SciencesPragueCzechia

Personalised recommendations