Skip to main content

Large-Scale Data for Multiple-View Stereopsis

Abstract

The seminal multiple-view stereo benchmark evaluations from Middlebury and by Strecha et al. have played a major role in propelling the development of multi-view stereopsis (MVS) methodology. The somewhat small size and variability of these data sets, however, limit their scope and the conclusions that can be derived from them. To facilitate further development within MVS, we here present a new and varied data set consisting of 80 scenes, seen from 49 or 64 accurate camera positions. This is accompanied by accurate structured light scans for reference and evaluation. In addition all images are taken under seven different lighting conditions. As a benchmark and to validate the use of our data set for obtaining reasonable and statistically significant findings about MVS, we have applied the three state-of-the-art MVS algorithms by Campbell et al., Furukawa et al., and Tola et al. to the data set. To do this we have extended the evaluation protocol from the Middlebury evaluation, necessitated by the more complex geometry of some of our scenes. The data set and accompanying evaluation framework are made freely available online. Based on this evaluation, we are able to observe several characteristics of state-of-the-art MVS, e.g. that there is a tradeoff between the quality of the reconstructed 3D points (accuracy) and how much of an object’s surface is captured (completeness). Also, several issues that we hypothesized would challenge MVS, such as specularities and changing lighting conditions did not pose serious problems. Our study finds that the two most pressing issues for MVS are lack of texture and meshing (forming 3D points into closed triangulated surfaces).

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Notes

  1. In a few of the extreme positions, the robot shaded a few of the LEDs.

  2. In the online data set, \(360^\circ \) scans of some models are included by combining four scans. In these cases we only included one data set into the evaluation, in order to avoid biasing the data set unnecessarily.

  3. This standardization of the Poisson reconstruction parameters was done after the preliminary version of this work (Jensen et al. 2014), which is why there is a slight discrepancy between the result of this paper and the preliminary version.

  4. http://roboimagedata.compute.dtu.dk/

References

  • Aanæs, H., Dahl, A., & Steenstrup Pedersen, K. (2012). Interesting interest points. IJCV, 97, 18–35.

    Article  Google Scholar 

  • Aanæs, H., & Dahl, A. (2015). Accuracy in robot generated image data sets. In Image analysis (pp. 472–479). Springer International Publishing.

  • Anderson, T. (1984). An introduction to multivariate statistical analysis. Hoboken: Wiley.

    MATH  Google Scholar 

  • Bailer, C., Finckh, M., Lensch, H. P. A. (2012). Scale robust multi view stereo. In ECCV (pp. 398–411). Springer.

  • Beraldin, J. A., Gaiani, M. (2005). Evaluating the performance of close range 3D active vision systems for industrial design applications. In SPIE: Videometrics IX (Vol. 5665).

  • Boehler, W., Vicent, B., Marbs, A. (2003). Investigating laser scanner accuracy. In CIPA.

  • Boykov, Y., Kolmogorov, V. (2003). Computing geodesics and minimal surfaces via graph cuts. In ICCV (pp. 26–33).

  • Campbell, N. D., Vogiatzis, G., Hernández, C., Cipolla, R. (2008). Using multiple hypotheses to improve depth-maps for multi-view stereo. In ECCV (pp. 766–779).

  • Cook, R. L., & Torrance, K. E. (1981). A reflectance model for computer graphics. SIGGRAPH Computer Graphics, 15(3), 307–316.

    Article  Google Scholar 

  • Curless, B., Levoy, M. (1996). A volumetric method for building complex models from range images. In Conference on computer graphics and interactive iechniques, ACM (pp. 303–312).

  • Fuhrmann, S., & Goesele, M. (2014). Floating scale surface reconstruction. ACM Transactions on Graphics (TOG), 33(4), 46.

    Article  Google Scholar 

  • Furukawa, Y., Curless, B., Seitz, S. M., Szeliski, R. (2010). Towards internet-scale multi-view stereo. In CVPR (pp. 1434–1441).

  • Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multiview stereopsis. PAMI, 32(8), 1362–1376.

    Article  Google Scholar 

  • Goesele, M., Curless, B., Seitz, S. M. (2006). Multi-view stereo revisited. In CVPR (pp. 2402–2409).

  • Hernández, C., Vogiatzis, G., Cipolla, R. (2007). Probabilistic visibility for multi-view stereo. In CVPR (pp. 1–8).

  • Hiep, V. H., Keriven, R., Labatut, P., Pons, J. P. (2009). Towards high-resolution large-scale multi-view stereo. In PAMI (pp. 1430–1437).

  • Hu, X., Mordohai, P. (2010). Evaluation of stereo confidence indoors and outdoors. In CVPR (pp. 1466–1473).

  • Jensen, R., Dahl, A., Vogiatzis, G., Tola, E., Aanaes, H. (2014). Large scale multi-view stereopsis evaluation. In CVPR (pp. 406–413).

  • Kazhdan, M., Bolitho, M., Hoppe, H. (2006). Poisson surface reconstruction. In Eurographics symposium on geometry processing (pp. 61–70).

  • Kim, S., Kim, S., Dahl, A., Conradsen, K., Jensen, R., Aanæ s, H. (2012). Multiple view stereo by reflectance modeling. In 2011 International conference on 3D imaging, modeling, processing, visualization and transmission.

  • Klowsky, R., Kuijper, A., Goesele, M. (2012). Modulation transfer function of patch-based stereo systems. In CVPR (pp. 1386–1393).

  • Kolev, K., Brox, T., & Cremers, D. (2012). Fast joint estimation of silhouettes and dense 3D geometry from multiple images. PAMI, 34(3), 493–505.

    Article  Google Scholar 

  • Labatut, P., Pons, J.P., Keriven, R. (2009). Robust and efficient surface reconstruction from range data. In Computer graphics forum (Vol. 28, pp. 2275–2290). Wiley Online Library.

  • Liu, S., Cooper, D. B. (2011). A complete statistical inverse ray tracing approach to multi-view stereo. In CVPR (pp. 913–920).

  • Luhmann, T. (2008). Comparison and verification of optical 3-d surface measurement systems. In International archives photogrammetry, remote sensing spatial information science (Vol. 37).

  • Møller, B., Balslev, I., & Krüger, N. (2013). An automatic evaluation procedure for 3D scanners in robotics applications. IEEE Sensors Journal, 13(2), 870–878.

    Article  Google Scholar 

  • Mücke, P., Klowsky, R., Goesele, M. (2011). Surface reconstruction from multi-resolution sample points. In VMV (pp. 105–112). Citeseer.

  • Nair, R., Meister, S., Lambers, M., Balda, M., Hofmann, H., Kolb, A., Kondermann, D., Jähne, B. (2013). Ground truth for evaluating time of flight imaging. In Time-of-flight and depth imaging (Vol. 8200, pp. 52–74). Springer.

  • Newcombe, R.A., Lovegrove, S.J., Davison, A.J. (2011). Dtam: Dense tracking and mapping in real-time. In ICCV (pp. 2320–2327).

  • Salvi, J., Fernandez, S., Pribanic, T., & Llado, X. (2010). A state of the art in structured light patterns for surface profilometry. Pattern Recognition, 43(8), 2666–2680.

    Article  MATH  Google Scholar 

  • Salvi, J., Pages, J., & Batlle, J. (2004). Pattern codification strategies in structured light systems. Pattern Recognition, 37(4), 827–849.

    Article  MATH  Google Scholar 

  • Scharstein, D., Szeliski, R. (2003). High-accuracy stereo depth maps using structured light. In CVPR (Vol. 1, pp. I–195).

  • Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In CVPR (Vol. 1, pp. 519–528).

  • Shalom, S., Shamir, A., Zhang, H., Cohen-Or, D. (2010). Cone carving for surface reconstruction. In ACM transactions on graphics (TOG), ACM (Vol. 29, p. 150).

  • Strecha, C., von Hansen, W., Van Gool, L., Fua, P., Thoennessen, U. (2008). On benchmarking camera calibration and multi-view stereo for high resolution imagery. In CVPR (pp. 1–8).

  • Tola, E., Strecha, C., & Fua, P. (2012). Efficient large-scale multi-view stereo for ultra high-resolution image sets. Machine Vision and Applications, 23(5), 903–920.

    Article  Google Scholar 

  • Tukey, J. W. (1977). Exploratory data analysis (1st ed.). Reading, MA: Pearson-Addison-Wesley.

    MATH  Google Scholar 

  • Vogiatzis, G., Hernández, C., Torr, P. H. S., & Cipolla, R. (2007). Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency. PAMI, 29(12), 2241–2246.

    Article  Google Scholar 

  • Wendel, A., Maurer, M., Graber, G., Pock, T., Bischof, H. (2012). Dense reconstruction on-the-fly. In CVPR (pp. 1450–1457).

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Anders Bjorholm Dahl.

Additional information

Communicated by Edmond Boyer.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Aanæs, H., Jensen, R.R., Vogiatzis, G. et al. Large-Scale Data for Multiple-View Stereopsis. Int J Comput Vis 120, 153–168 (2016). https://doi.org/10.1007/s11263-016-0902-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-016-0902-9

Keywords

  • Multiple-view stereopsis
  • 3D data
  • Benchmark
  • Stereo
  • Multi-view stereo
  • 3D evaluation