The Second Workshop on 3D Reconstruction Meets Semantics: Challenge Results Discussion

  • Radim TylecekEmail author
  • Torsten Sattler
  • Hoang-An Le
  • Thomas Brox
  • Marc Pollefeys
  • Robert B. Fisher
  • Theo Gevers
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11131)


This paper discusses a reconstruction challenge held as a part of the second 3D Reconstruction meets Semantics workshop (3DRMS). The challenge goals and datasets are introduced, including both synthetic and real data from outdoor scenes, here represented by gardens with a variety of bushes, trees, other plants and objects. Both qualitative and quantitative evaluation of the challenge participants’ submissions is given in categories of geometric and semantic accuracy. Finally, comparison of submitted results with baseline methods is given, showing a modest performance increase in some of the categories.


3D reconstruction Semantic segmentation Challenge Dataset 



The workshop, reconstruction challenge and acquisition of datasets was supported by EU project TrimBot2020.


  1. 1.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 2481–2495 (2017)CrossRefGoogle Scholar
  2. 2.
    Baslamisli, A.S., Groenestege, T.T., Das, P., Le, H.-A., Karaoglu, S., Gevers, T.: Joint learning of intrinsic images and semantic segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11210, pp. 289–305. Springer, Cham (2018). Scholar
  3. 3.
    Carvalho, M., Ferrera, M., Boulch, A., Moras, J., Saux, B.L., Trouvé-Peloux, P.: Co-learning of geometry and semantics for online 3D mapping. In: 3DRMS Workshop Challenge, ECCV (2018)Google Scholar
  4. 4.
    Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 (2017)
  5. 5.
    Cherabier, I., Schönberger, J.L., Oswald, M.R., Pollefeys, M., Geiger, A.: Learning priors for semantic 3D reconstruction. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11216, pp. 325–341. Springer, Cham (2018). Scholar
  6. 6.
    Eigen, D., Fergus, R.: Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In: Proceedings of ICCV, pp. 2650–2658 (2015)Google Scholar
  7. 7.
    Furgale, P., Rehder, J., Siegwart, R.: Unified temporal and spatial calibration for multi-sensor systems. In: International Conference on Intelligent Robots and Systems, pp. 1280–1286, November 2013Google Scholar
  8. 8.
    Geiger, A., Roser, M., Urtasun, R.: Efficient large-scale stereo matching. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010. LNCS, vol. 6492, pp. 25–38. Springer, Heidelberg (2011). Scholar
  9. 9.
    Häne, C., Zach, C., Cohen, A., Pollefeys, M.: Dense semantic 3D reconstruction. Trans. Pattern Anal. Mach. Intell. 39(9), 1730–1743 (2017)CrossRefGoogle Scholar
  10. 10.
    Haque, S.M., Arora, S., Babu, V.: 3D semantic reconstruction using class-specific models. In: 3DRMS Workshop Challenge, ECCV (2018)Google Scholar
  11. 11.
    Hazirbas, C., Ma, L., Domokos, C., Cremers, D.: FuseNet: incorporating depth into semantic segmentation via fusion-based CNN architecture. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10111, pp. 213–228. Springer, Cham (2017). Scholar
  12. 12.
    Ilha, G., Waszak, T., Pereira, F.I., Susin, A.A.: Lapsi-360. In: 3DRMS Workshop Challenge, ECCV (2018)Google Scholar
  13. 13.
    Kazhdan, M., Hoppe, H.: Screened poisson surface reconstruction. ACM Trans. Graph. (ToG) 32(3), 29 (2013)CrossRefGoogle Scholar
  14. 14.
    Le, H.A., Baslamisli, A.S., Mensink, T., Gevers, T.: Three for one and one for three: flow, segmentation, and surface normals. In: Proceedings of BMVC (2018)Google Scholar
  15. 15.
    Lorensen, W.E., Cline, H.E.: Marching cubes: a high resolution 3D surface construction algorithm. ACM SIGGRAPH Comput. Graph. 21, 163–169 (1987)CrossRefGoogle Scholar
  16. 16.
    Schönberger, J.L., Zheng, E., Frahm, J.-M., Pollefeys, M.: Pixelwise view selection for unstructured multi-view stereo. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 501–518. Springer, Cham (2016). Scholar
  17. 17.
    Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Proceedings of CVPR (2016)Google Scholar
  18. 18.
    Schöps, T., et al.: A multi-view stereo benchmark with high-resolution images and multi-camera videos. In: Proceedings of CVPR (2017)Google Scholar
  19. 19.
    Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings of CVPR, pp. 519–528. IEEE Computer Society, Washington (2006)Google Scholar
  20. 20.
    Tylecek, R., Fisher, R.B.: Consistent semantic annotation of outdoor datasets via 2D/3D label transfer. Sensors 18(7), 2249 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Radim Tylecek
    • 1
    Email author
  • Torsten Sattler
    • 2
  • Hoang-An Le
    • 3
  • Thomas Brox
    • 4
  • Marc Pollefeys
    • 2
    • 5
  • Robert B. Fisher
    • 1
  • Theo Gevers
    • 4
  1. 1.University of EdinburghEdinburghScotland
  2. 2.Department of Computer ScienceETH ZurichZurichSwitzerland
  3. 3.University of AmsterdamAmsterdamNetherlands
  4. 4.University of FreiburgFreiburg im BreisgauGermany
  5. 5.Software Development CentreMicrosoftZurichSwitzerland

Personalised recommendations