Advertisement

International Journal of Computer Vision

, Volume 120, Issue 2, pp 153–168 | Cite as

Large-Scale Data for Multiple-View Stereopsis

  • Henrik Aanæs
  • Rasmus Ramsbøl Jensen
  • George Vogiatzis
  • Engin Tola
  • Anders Bjorholm DahlEmail author
Article

Abstract

The seminal multiple-view stereo benchmark evaluations from Middlebury and by Strecha et al. have played a major role in propelling the development of multi-view stereopsis (MVS) methodology. The somewhat small size and variability of these data sets, however, limit their scope and the conclusions that can be derived from them. To facilitate further development within MVS, we here present a new and varied data set consisting of 80 scenes, seen from 49 or 64 accurate camera positions. This is accompanied by accurate structured light scans for reference and evaluation. In addition all images are taken under seven different lighting conditions. As a benchmark and to validate the use of our data set for obtaining reasonable and statistically significant findings about MVS, we have applied the three state-of-the-art MVS algorithms by Campbell et al., Furukawa et al., and Tola et al. to the data set. To do this we have extended the evaluation protocol from the Middlebury evaluation, necessitated by the more complex geometry of some of our scenes. The data set and accompanying evaluation framework are made freely available online. Based on this evaluation, we are able to observe several characteristics of state-of-the-art MVS, e.g. that there is a tradeoff between the quality of the reconstructed 3D points (accuracy) and how much of an object’s surface is captured (completeness). Also, several issues that we hypothesized would challenge MVS, such as specularities and changing lighting conditions did not pose serious problems. Our study finds that the two most pressing issues for MVS are lack of texture and meshing (forming 3D points into closed triangulated surfaces).

Keywords

Multiple-view stereopsis 3D data Benchmark Stereo Multi-view stereo 3D evaluation 

References

  1. Aanæs, H., Dahl, A., & Steenstrup Pedersen, K. (2012). Interesting interest points. IJCV, 97, 18–35.CrossRefGoogle Scholar
  2. Aanæs, H., & Dahl, A. (2015). Accuracy in robot generated image data sets. In Image analysis (pp. 472–479). Springer International Publishing.Google Scholar
  3. Anderson, T. (1984). An introduction to multivariate statistical analysis. Hoboken: Wiley.zbMATHGoogle Scholar
  4. Bailer, C., Finckh, M., Lensch, H. P. A. (2012). Scale robust multi view stereo. In ECCV (pp. 398–411). Springer.Google Scholar
  5. Beraldin, J. A., Gaiani, M. (2005). Evaluating the performance of close range 3D active vision systems for industrial design applications. In SPIE: Videometrics IX (Vol. 5665).Google Scholar
  6. Boehler, W., Vicent, B., Marbs, A. (2003). Investigating laser scanner accuracy. In CIPA.Google Scholar
  7. Boykov, Y., Kolmogorov, V. (2003). Computing geodesics and minimal surfaces via graph cuts. In ICCV (pp. 26–33).Google Scholar
  8. Campbell, N. D., Vogiatzis, G., Hernández, C., Cipolla, R. (2008). Using multiple hypotheses to improve depth-maps for multi-view stereo. In ECCV (pp. 766–779).Google Scholar
  9. Cook, R. L., & Torrance, K. E. (1981). A reflectance model for computer graphics. SIGGRAPH Computer Graphics, 15(3), 307–316.CrossRefGoogle Scholar
  10. Curless, B., Levoy, M. (1996). A volumetric method for building complex models from range images. In Conference on computer graphics and interactive iechniques, ACM (pp. 303–312).Google Scholar
  11. Fuhrmann, S., & Goesele, M. (2014). Floating scale surface reconstruction. ACM Transactions on Graphics (TOG), 33(4), 46.CrossRefGoogle Scholar
  12. Furukawa, Y., Curless, B., Seitz, S. M., Szeliski, R. (2010). Towards internet-scale multi-view stereo. In CVPR (pp. 1434–1441).Google Scholar
  13. Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multiview stereopsis. PAMI, 32(8), 1362–1376.CrossRefGoogle Scholar
  14. Goesele, M., Curless, B., Seitz, S. M. (2006). Multi-view stereo revisited. In CVPR (pp. 2402–2409).Google Scholar
  15. Hernández, C., Vogiatzis, G., Cipolla, R. (2007). Probabilistic visibility for multi-view stereo. In CVPR (pp. 1–8).Google Scholar
  16. Hiep, V. H., Keriven, R., Labatut, P., Pons, J. P. (2009). Towards high-resolution large-scale multi-view stereo. In PAMI (pp. 1430–1437).Google Scholar
  17. Hu, X., Mordohai, P. (2010). Evaluation of stereo confidence indoors and outdoors. In CVPR (pp. 1466–1473).Google Scholar
  18. Jensen, R., Dahl, A., Vogiatzis, G., Tola, E., Aanaes, H. (2014). Large scale multi-view stereopsis evaluation. In CVPR (pp. 406–413).Google Scholar
  19. Kazhdan, M., Bolitho, M., Hoppe, H. (2006). Poisson surface reconstruction. In Eurographics symposium on geometry processing (pp. 61–70).Google Scholar
  20. Kim, S., Kim, S., Dahl, A., Conradsen, K., Jensen, R., Aanæ s, H. (2012). Multiple view stereo by reflectance modeling. In 2011 International conference on 3D imaging, modeling, processing, visualization and transmission.Google Scholar
  21. Klowsky, R., Kuijper, A., Goesele, M. (2012). Modulation transfer function of patch-based stereo systems. In CVPR (pp. 1386–1393).Google Scholar
  22. Kolev, K., Brox, T., & Cremers, D. (2012). Fast joint estimation of silhouettes and dense 3D geometry from multiple images. PAMI, 34(3), 493–505.CrossRefGoogle Scholar
  23. Labatut, P., Pons, J.P., Keriven, R. (2009). Robust and efficient surface reconstruction from range data. In Computer graphics forum (Vol. 28, pp. 2275–2290). Wiley Online Library.Google Scholar
  24. Liu, S., Cooper, D. B. (2011). A complete statistical inverse ray tracing approach to multi-view stereo. In CVPR (pp. 913–920).Google Scholar
  25. Luhmann, T. (2008). Comparison and verification of optical 3-d surface measurement systems. In International archives photogrammetry, remote sensing spatial information science (Vol. 37).Google Scholar
  26. Møller, B., Balslev, I., & Krüger, N. (2013). An automatic evaluation procedure for 3D scanners in robotics applications. IEEE Sensors Journal, 13(2), 870–878.CrossRefGoogle Scholar
  27. Mücke, P., Klowsky, R., Goesele, M. (2011). Surface reconstruction from multi-resolution sample points. In VMV (pp. 105–112). Citeseer.Google Scholar
  28. Nair, R., Meister, S., Lambers, M., Balda, M., Hofmann, H., Kolb, A., Kondermann, D., Jähne, B. (2013). Ground truth for evaluating time of flight imaging. In Time-of-flight and depth imaging (Vol. 8200, pp. 52–74). Springer.Google Scholar
  29. Newcombe, R.A., Lovegrove, S.J., Davison, A.J. (2011). Dtam: Dense tracking and mapping in real-time. In ICCV (pp. 2320–2327).Google Scholar
  30. Salvi, J., Fernandez, S., Pribanic, T., & Llado, X. (2010). A state of the art in structured light patterns for surface profilometry. Pattern Recognition, 43(8), 2666–2680.CrossRefzbMATHGoogle Scholar
  31. Salvi, J., Pages, J., & Batlle, J. (2004). Pattern codification strategies in structured light systems. Pattern Recognition, 37(4), 827–849.CrossRefzbMATHGoogle Scholar
  32. Scharstein, D., Szeliski, R. (2003). High-accuracy stereo depth maps using structured light. In CVPR (Vol. 1, pp. I–195).Google Scholar
  33. Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In CVPR (Vol. 1, pp. 519–528).Google Scholar
  34. Shalom, S., Shamir, A., Zhang, H., Cohen-Or, D. (2010). Cone carving for surface reconstruction. In ACM transactions on graphics (TOG), ACM (Vol. 29, p. 150).Google Scholar
  35. Strecha, C., von Hansen, W., Van Gool, L., Fua, P., Thoennessen, U. (2008). On benchmarking camera calibration and multi-view stereo for high resolution imagery. In CVPR (pp. 1–8).Google Scholar
  36. Tola, E., Strecha, C., & Fua, P. (2012). Efficient large-scale multi-view stereo for ultra high-resolution image sets. Machine Vision and Applications, 23(5), 903–920.CrossRefGoogle Scholar
  37. Tukey, J. W. (1977). Exploratory data analysis (1st ed.). Reading, MA: Pearson-Addison-Wesley.zbMATHGoogle Scholar
  38. Vogiatzis, G., Hernández, C., Torr, P. H. S., & Cipolla, R. (2007). Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency. PAMI, 29(12), 2241–2246.CrossRefGoogle Scholar
  39. Wendel, A., Maurer, M., Graber, G., Pock, T., Bischof, H. (2012). Dense reconstruction on-the-fly. In CVPR (pp. 1450–1457).Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Henrik Aanæs
    • 1
  • Rasmus Ramsbøl Jensen
    • 1
  • George Vogiatzis
    • 2
  • Engin Tola
    • 3
  • Anders Bjorholm Dahl
    • 1
    Email author
  1. 1.Technical University of DenmarkLyngbyDenmark
  2. 2.Aston UniversityBirminghamEngland
  3. 3.AurvisAnkaraTurkey

Personalised recommendations