Advertisement

A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields

  • Katrin Honauer
  • Ole Johannsen
  • Daniel Kondermann
  • Bastian Goldluecke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10113)

Abstract

In computer vision communities such as stereo, optical flow, or visual tracking, commonly accepted and widely used benchmarks have enabled objective comparison and boosted scientific progress.

In the emergent light field community, a comparable benchmark and evaluation methodology is still missing. The performance of newly proposed methods is often demonstrated qualitatively on a handful of images, making quantitative comparison and targeted progress very difficult. To overcome these difficulties, we propose a novel light field benchmark. We provide 24 carefully designed synthetic, densely sampled 4D light fields with highly accurate disparity ground truth. We thoroughly evaluate four state-of-the-art light field algorithms and one multi-view stereo algorithm using existing and novel error measures.

This consolidated state-of-the art may serve as a baseline to stimulate and guide further scientific progress. We publish the benchmark website http://www.lightfield-analysis.net, an evaluation toolkit, and our rendering setup to encourage submissions of both algorithms and further datasets.

Keywords

Ground Truth Light Field Algorithm Performance Occlusion Area Occlusion Boundary 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgment

This work was supported by the ERC Starting Grant “Light Field Imaging and Analysis” (LIA 336978, FP7-2014), the Heidelberg Collaboratory for Image Processing (Institutional Strategy ZUK49, Measure 6.4) and the AIT Vienna, Austria.

Supplementary material

416261_1_En_2_MOESM1_ESM.pdf (7.7 mb)
Supplementary material 1 (pdf 7932 KB)

References

  1. 1.
    Levoy, M.: Light fields and computational imaging. Computer 39, 46–55 (2006)CrossRefGoogle Scholar
  2. 2.
    Tao, M., Hadap, S., Malik, J., Ramamoorthi, R.: Depth from combining defocus and correspondence using light-field cameras. In: Proceedings of the International Conference on Computer Vision (2013)Google Scholar
  3. 3.
    Wanner, S., Goldluecke, B.: Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36, 606–619 (2014)CrossRefGoogle Scholar
  4. 4.
    Heber, S., Pock, T.: Shape from light field meets robust PCA. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 751–767. Springer, Cham (2014). doi: 10.1007/978-3-319-10599-4_48 Google Scholar
  5. 5.
    Jeon, H., Park, J., Choe, G., Park, J., Bok, Y., Tai, Y., Kweon, I.: Accurate depth map estimation from a lenslet light field camera. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  6. 6.
    Johannsen, O., Sulc, A., Goldluecke, B.: What sparse light field coding reveals about scene structure. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3262–3270 (2016)Google Scholar
  7. 7.
    Wang, T., Efros, A., Ramamoorthi, R.: Occlusion-aware depth estimation using light-field cameras. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3487–3495 (2015)Google Scholar
  8. 8.
    Wilburn, B., Joshi, N., Vaish, V., Talvala, E.V., Antunez, E., Barth, A., Adams, A., Horowitz, M., Levoy, M.: High performance imaging using large camera arrays. ACM Trans. Graph. (TOG) 24, 765–776 (2005). ACM. http://lightfield.stanford.edu/ CrossRefGoogle Scholar
  9. 9.
    Marwah, K., Wetzstein, G., Bando, Y., Raskar, R.: Compressive light field photography using overcomplete dictionaries and optimized projections. ACM Trans. Graph. (Proc. SIGGRAPH) 32, 1–11 (2013). http://web.media.mit.edu/ gordonw/SyntheticLightFields/index.php CrossRefMATHGoogle Scholar
  10. 10.
    Mousnier, A., Vural, E., Guillemot, C.: Partial light field tomographic reconstruction from a fixed-camera focal stack. arXiv preprint arXiv:1503.01903 (2015). https://www.irisa.fr/temics/demos/lightField/index.html
  11. 11.
    Kim, C., Zimmer, H., Pritch, Y., Sorkine-Hornung, A., Gross, M.H.: Scene reconstruction from high spatio-angular resolution light fields. ACM Trans. Graph. 32, 73:1–73:12 (2013). https://www.disneyresearch.com/project/lightfields/ MATHGoogle Scholar
  12. 12.
    Rerabek, M., Ebrahimi, T.: New light field image dataset. In: 8th International Conference on Quality of Multimedia Experience (QoMEX). Number EPFL-CONF-218363 (2016)Google Scholar
  13. 13.
    Wanner, S., Meister, S., Goldluecke, B.: Datasets and benchmarks for densely sampled 4D light fields. In: Vision, Modelling and Visualization (VMV) (2013)Google Scholar
  14. 14.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002)CrossRefMATHGoogle Scholar
  15. 15.
    Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33783-3_44 CrossRefGoogle Scholar
  16. 16.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 3354–3361 (2012)Google Scholar
  17. 17.
    Kristan, M., Matas, J., Leonardis, A., Felsberg, M., Cehovin, L., Fernandez, G., Vojir, T., Hager, G., Nebehay, G., Pflugfelder, R.: The visual object tracking VOT2015 challenge results. In: Proceedings of the ICCV, pp. 1–23 (2015)Google Scholar
  18. 18.
    Kristan, M., Pflugfelder, R., Leonardis, A., Matas, J., Porikli, F., Čehovin, L., Nebehay, G., Fernandez, G., Vojir, T.: The VOT2013 challenge: overview and additional results (2014)Google Scholar
  19. 19.
    Scharstein, D., Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nešić, N., Wang, X., Westling, P.: High-resolution stereo datasets with subpixel-accurate ground truth. In: Jiang, X., Hornegger, J., Koch, R. (eds.) GCPR 2014. LNCS, vol. 8753, pp. 31–42. Springer, Cham (2014). doi: 10.1007/978-3-319-11752-2_3 Google Scholar
  20. 20.
    Kristan, M., Matas, J., Leonardis, A., Vojir, T., Pflugfelder, R., Fernandez, G., Nebehay, G., Porikli, F., Cehovin, L.: A novel performance evaluation methodology for single-target trackers (2015)Google Scholar
  21. 21.
    Honauer, K., Maier-Hein, L., Kondermann, D.: The HCI stereo metrics: geometry-aware performance analysis of stereo algorithms. In: Proceedings of the ICCV, pp. 2120–2128 (2015)Google Scholar
  22. 22.
    Menze, M., Geiger, A.: Object scene flow for autonomous vehicles. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  23. 23.
    Zendel, O., Murschitz, M., Humenberger, M., Herzner, W.: CV-HAZOP: introducing test data validation for computer vision. In: Proceedings of the ICCV (2015)Google Scholar
  24. 24.
    Haeusler, R., Kondermann, D.: Synthesizing real world stereo challenges. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 164–173. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40602-7_17 CrossRefGoogle Scholar
  25. 25.
    Kondermann, D., Nair, R., Honauer, K., Krispin, K., Andrulis, J., Brock, A., Güssefeld, B., Rahimimoghaddam, M., Hofmann, S., Brenner, C., Jähne, B.: The HCI benchmark suite: stereo and flow ground truth with uncertainties for urban autonomous driving. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition Workshops (2016)Google Scholar
  26. 26.
    Perazzi, F., Pont-Tuset, J., McWilliams, B., Gool, L.V., Gross, M., Sorkine-Hornung, A.: A benchmark dataset and evaluation methodology for video object segmentation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  27. 27.
    Meister, S., Kondermann, D.: Real versus realistically rendered scenes for optical flow evaluation. In: 14th ITG Conference on Electronic Media Technology, pp. 1–6 (2011)Google Scholar
  28. 28.
    Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D., Brox, T.: Flownet: learning optical flow with convolutional networks, pp. 2758–2766 (2015)Google Scholar
  29. 29.
    Blender Online Community: Blender - a 3D modelling and rendering package (2016)Google Scholar
  30. 30.
    Wanner, S., Goldluecke, B.: Reconstructing reflective and transparent surfaces from epipolar plane images. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 1–10. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40602-7_1 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Katrin Honauer
    • 1
  • Ole Johannsen
    • 2
  • Daniel Kondermann
    • 1
  • Bastian Goldluecke
    • 2
  1. 1.HCIHeidelberg UniversityHeidelbergGermany
  2. 2.University of KonstanzKonstanzGermany

Personalised recommendations