Abstract
The seminal multiple-view stereo benchmark evaluations from Middlebury and by Strecha et al. have played a major role in propelling the development of multi-view stereopsis (MVS) methodology. The somewhat small size and variability of these data sets, however, limit their scope and the conclusions that can be derived from them. To facilitate further development within MVS, we here present a new and varied data set consisting of 80 scenes, seen from 49 or 64 accurate camera positions. This is accompanied by accurate structured light scans for reference and evaluation. In addition all images are taken under seven different lighting conditions. As a benchmark and to validate the use of our data set for obtaining reasonable and statistically significant findings about MVS, we have applied the three state-of-the-art MVS algorithms by Campbell et al., Furukawa et al., and Tola et al. to the data set. To do this we have extended the evaluation protocol from the Middlebury evaluation, necessitated by the more complex geometry of some of our scenes. The data set and accompanying evaluation framework are made freely available online. Based on this evaluation, we are able to observe several characteristics of state-of-the-art MVS, e.g. that there is a tradeoff between the quality of the reconstructed 3D points (accuracy) and how much of an object’s surface is captured (completeness). Also, several issues that we hypothesized would challenge MVS, such as specularities and changing lighting conditions did not pose serious problems. Our study finds that the two most pressing issues for MVS are lack of texture and meshing (forming 3D points into closed triangulated surfaces).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
In a few of the extreme positions, the robot shaded a few of the LEDs.
In the online data set, \(360^\circ \) scans of some models are included by combining four scans. In these cases we only included one data set into the evaluation, in order to avoid biasing the data set unnecessarily.
This standardization of the Poisson reconstruction parameters was done after the preliminary version of this work (Jensen et al. 2014), which is why there is a slight discrepancy between the result of this paper and the preliminary version.
References
Aanæs, H., Dahl, A., & Steenstrup Pedersen, K. (2012). Interesting interest points. IJCV, 97, 18–35.
Aanæs, H., & Dahl, A. (2015). Accuracy in robot generated image data sets. In Image analysis (pp. 472–479). Springer International Publishing.
Anderson, T. (1984). An introduction to multivariate statistical analysis. Hoboken: Wiley.
Bailer, C., Finckh, M., Lensch, H. P. A. (2012). Scale robust multi view stereo. In ECCV (pp. 398–411). Springer.
Beraldin, J. A., Gaiani, M. (2005). Evaluating the performance of close range 3D active vision systems for industrial design applications. In SPIE: Videometrics IX (Vol. 5665).
Boehler, W., Vicent, B., Marbs, A. (2003). Investigating laser scanner accuracy. In CIPA.
Boykov, Y., Kolmogorov, V. (2003). Computing geodesics and minimal surfaces via graph cuts. In ICCV (pp. 26–33).
Campbell, N. D., Vogiatzis, G., Hernández, C., Cipolla, R. (2008). Using multiple hypotheses to improve depth-maps for multi-view stereo. In ECCV (pp. 766–779).
Cook, R. L., & Torrance, K. E. (1981). A reflectance model for computer graphics. SIGGRAPH Computer Graphics, 15(3), 307–316.
Curless, B., Levoy, M. (1996). A volumetric method for building complex models from range images. In Conference on computer graphics and interactive iechniques, ACM (pp. 303–312).
Fuhrmann, S., & Goesele, M. (2014). Floating scale surface reconstruction. ACM Transactions on Graphics (TOG), 33(4), 46.
Furukawa, Y., Curless, B., Seitz, S. M., Szeliski, R. (2010). Towards internet-scale multi-view stereo. In CVPR (pp. 1434–1441).
Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multiview stereopsis. PAMI, 32(8), 1362–1376.
Goesele, M., Curless, B., Seitz, S. M. (2006). Multi-view stereo revisited. In CVPR (pp. 2402–2409).
Hernández, C., Vogiatzis, G., Cipolla, R. (2007). Probabilistic visibility for multi-view stereo. In CVPR (pp. 1–8).
Hiep, V. H., Keriven, R., Labatut, P., Pons, J. P. (2009). Towards high-resolution large-scale multi-view stereo. In PAMI (pp. 1430–1437).
Hu, X., Mordohai, P. (2010). Evaluation of stereo confidence indoors and outdoors. In CVPR (pp. 1466–1473).
Jensen, R., Dahl, A., Vogiatzis, G., Tola, E., Aanaes, H. (2014). Large scale multi-view stereopsis evaluation. In CVPR (pp. 406–413).
Kazhdan, M., Bolitho, M., Hoppe, H. (2006). Poisson surface reconstruction. In Eurographics symposium on geometry processing (pp. 61–70).
Kim, S., Kim, S., Dahl, A., Conradsen, K., Jensen, R., Aanæ s, H. (2012). Multiple view stereo by reflectance modeling. In 2011 International conference on 3D imaging, modeling, processing, visualization and transmission.
Klowsky, R., Kuijper, A., Goesele, M. (2012). Modulation transfer function of patch-based stereo systems. In CVPR (pp. 1386–1393).
Kolev, K., Brox, T., & Cremers, D. (2012). Fast joint estimation of silhouettes and dense 3D geometry from multiple images. PAMI, 34(3), 493–505.
Labatut, P., Pons, J.P., Keriven, R. (2009). Robust and efficient surface reconstruction from range data. In Computer graphics forum (Vol. 28, pp. 2275–2290). Wiley Online Library.
Liu, S., Cooper, D. B. (2011). A complete statistical inverse ray tracing approach to multi-view stereo. In CVPR (pp. 913–920).
Luhmann, T. (2008). Comparison and verification of optical 3-d surface measurement systems. In International archives photogrammetry, remote sensing spatial information science (Vol. 37).
Møller, B., Balslev, I., & Krüger, N. (2013). An automatic evaluation procedure for 3D scanners in robotics applications. IEEE Sensors Journal, 13(2), 870–878.
Mücke, P., Klowsky, R., Goesele, M. (2011). Surface reconstruction from multi-resolution sample points. In VMV (pp. 105–112). Citeseer.
Nair, R., Meister, S., Lambers, M., Balda, M., Hofmann, H., Kolb, A., Kondermann, D., Jähne, B. (2013). Ground truth for evaluating time of flight imaging. In Time-of-flight and depth imaging (Vol. 8200, pp. 52–74). Springer.
Newcombe, R.A., Lovegrove, S.J., Davison, A.J. (2011). Dtam: Dense tracking and mapping in real-time. In ICCV (pp. 2320–2327).
Salvi, J., Fernandez, S., Pribanic, T., & Llado, X. (2010). A state of the art in structured light patterns for surface profilometry. Pattern Recognition, 43(8), 2666–2680.
Salvi, J., Pages, J., & Batlle, J. (2004). Pattern codification strategies in structured light systems. Pattern Recognition, 37(4), 827–849.
Scharstein, D., Szeliski, R. (2003). High-accuracy stereo depth maps using structured light. In CVPR (Vol. 1, pp. I–195).
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In CVPR (Vol. 1, pp. 519–528).
Shalom, S., Shamir, A., Zhang, H., Cohen-Or, D. (2010). Cone carving for surface reconstruction. In ACM transactions on graphics (TOG), ACM (Vol. 29, p. 150).
Strecha, C., von Hansen, W., Van Gool, L., Fua, P., Thoennessen, U. (2008). On benchmarking camera calibration and multi-view stereo for high resolution imagery. In CVPR (pp. 1–8).
Tola, E., Strecha, C., & Fua, P. (2012). Efficient large-scale multi-view stereo for ultra high-resolution image sets. Machine Vision and Applications, 23(5), 903–920.
Tukey, J. W. (1977). Exploratory data analysis (1st ed.). Reading, MA: Pearson-Addison-Wesley.
Vogiatzis, G., Hernández, C., Torr, P. H. S., & Cipolla, R. (2007). Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency. PAMI, 29(12), 2241–2246.
Wendel, A., Maurer, M., Graber, G., Pock, T., Bischof, H. (2012). Dense reconstruction on-the-fly. In CVPR (pp. 1450–1457).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Edmond Boyer.
Rights and permissions
About this article
Cite this article
Aanæs, H., Jensen, R.R., Vogiatzis, G. et al. Large-Scale Data for Multiple-View Stereopsis. Int J Comput Vis 120, 153–168 (2016). https://doi.org/10.1007/s11263-016-0902-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-016-0902-9