Skip to main content
SpringerLink
Log in

Real-time adaptive content retargeting for live multi-view capture and light field display

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The discrete nature of multiprojector light field displays results in aliasing when rendering scene points at depths outside the supported depth of field causing visual discomfort. We propose an efficient on-the-fly content-aware real-time depth retargeting algorithm for live 3D light field video to increase the quality of visual perception on a cluster-driven multiprojector light field display. The proposed algorithm is embedded in an end-to-end real-time system capable of capturing and reconstructing light field from multiple calibrated cameras on a full horizontal parallax light field display. By automatically detecting salient regions of a scene, we solve an optimization to derive a non-linear operator to fit the whole scene within the comfortable viewing range of the light field display. We evaluate the effectiveness of our approach on synthetic and real world scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Agus, M., Gobbetti, E., Iglesias Guitián, J.A., Marton, F., Pintore, G.: GPU accelerated direct volume rendering on an interactive light field display. Comput. Graph. Forum 27(2), 231–240 (2008)

    Article  Google Scholar 

  2. Agus, M., Gobbetti, E., Jaspe Villanueva, A., Pintore, G., Pintus, R.: Automatic geometric calibration of projector-based light-field displays. In: Proc. Eurovis Short Papers, pp. 1–5 (2013)

  3. Balogh, T., Kovács, P.: Real-time 3D light field transmission. In: SPIE, vol. 7724, p. 5 (2010)

  4. Birklbauer, C., Bimber, O.: Light-field retargeting. Comput. Graph. Forum 31(2pt1), 295–303 (2012)

    Article  Google Scholar 

  5. Chai, J.X., Tong, X., Chan, S.C., Shum, H.Y.: Plenoptic sampling. In: Proceedings of SIGGRAPH, pp. 307–318 (2000)

  6. Didyk, P., Ritschel, T., Eisemann, E., Myszkowski, K., Seidel, H.P.: A perceptual model for disparity. ACM Trans. Graph. 30(4), 96:1–96:10 (2011)

  7. Graf, D., Panozzo, D., Sorkine-Hornung, O.: Mobile image retargeting. In: Proceedings of VMV (2013)

  8. Iglesias Guitián, J., Gobbetti, E., Marton, F.: View-dependent exploration of massive volumetric models on large scale light field displays. Vis. Comput. 26(6–8), 1037–1047 (2010)

    Article  Google Scholar 

  9. Jones, A., McDowall, I., Yamada, H., Bolas, M.T., Debevec, P.E.: Rendering for an interactive 360 degree light field display. ACM Trans. Graph 26(3), 40 (2007)

    Article  Google Scholar 

  10. Kim, C., Hornung, A., Heinzle, S., Matusik, W., Gross, M.: Multi-perspective stereoscopy from light fields. ACM Trans. Graph. 30(6), 190:1–190:10 (2011)

    Google Scholar 

  11. Kraevoy, V., Sheffer, A., Shamir, A., Cohen-Or, D.: Non-homogeneous resizing of complex models. ACM Trans. Graph. 27(5), 1–9 (2008)

  12. Kubota, A., Smolic, A., Magnor, M., Tanimoto, M., Chen, T., Zhang, C.: Multiview imaging and 3DTV. IEEE Signal Process. Mag. 24(6), 10 (2007)

    Article  Google Scholar 

  13. Kunita, Y., Ueno, M., Tanaka, K.: Layered probability maps: basic framework and prototype system. In: Proceedings of VRST, pp. 181–188 (2006)

  14. Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., Gross, M.: Nonlinear disparity mapping for stereoscopic 3d. ACM Trans. Graph. 29(4), 75:1–75:10 (2010)

    Article  Google Scholar 

  15. Marton, F., Gobbetti, E., Bettio, F., Iglesias Guitián, J., Pintus, R.: A real-time coarse-to-fine multiview capture system for all-in-focus rendering on a light-field display. In: Proceedings of 3DTV-CON, pp. 1–4 (2011)

  16. Masia, B., Wetzstein, G., Aliaga, C., Raskar, R., Gutierrez, D.: Display adaptive 3d content remapping. Comput. Graph. 37(8), 983–996 (2013)

    Article  Google Scholar 

  17. Masia, B., Wetzstein, G., Didyk, P., Gutierrez, D.: A survey on computational displays: pushing the boundaries of optics, computation, and perception. Comput. Graph. 37(8), 1012–1038 (2013)

    Article  Google Scholar 

  18. Matusik, W., Pfister, H.: 3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. ACM Trans. Graph. 23(3), 814–824 (2004)

    Article  Google Scholar 

  19. Smolic, A.: 3D video and free viewpoint video-from capture to display. Pattern Recogn. 44(9), 1958–1968 (2010)

  20. Taguchi, Y., Koike, T., Takahashi, K., Naemura, T.: TransCAIP: a live 3D TV system using a camera array and an integral photography display with interactive control of viewing parameters. IEEE Trans. Vis. Comput. Graph. 15, 841–852 (2009)

    Article  Google Scholar 

  21. Taguchi, Y., Takahashi, K., Naemura, T.: Real-time all-in-focus video-based rendering using a network camera array. In: Proceedings of 3DPVT, pp. 241–244 (2008)

  22. Yang, R., Huang, X., Li, S., Jaynes, C.: Toward the light field display: autostereoscopic rendering via a cluster of projectors. IEEE Trans. Vis. Comput. Graph. 14, 84–96 (2008)

    Article  Google Scholar 

  23. Yang, R., Welch, G., Bishop, G.: Real-time consensus-based scene reconstruction using commodity graphics hardware. In: Proceedings of Pacific Conference on Computer Graphics and Applications, pp. 225–234 (2002)

  24. Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Proceedings of ECCV, pp. 151–158 (1994)

Download references

Acknowledgments

This research has received funding from the DIVA Marie Curie Action of the People programme of the EU FP7/2007- 2013/ Program under REA Grant agreement 290227. The support of the TAMOP-4.2.1.B-11/2/KMR-2011-0002 is kindly acknowledged.

Author information

Authors and Affiliations

  1. Pázmány Péter Catholic University, Práter utca 50/a, Budapest, 1083, Hungary

    Vamsi Kiran Adhikarla

  2. CRS4, Pula, Italy

    Fabio Marton & Enrico Gobbetti

  3. Holografika, Budapest, Hungary

    Vamsi Kiran Adhikarla & Tibor Balogh

Authors
  1. Vamsi Kiran Adhikarla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fabio Marton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tibor Balogh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Enrico Gobbetti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vamsi Kiran Adhikarla.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 21330 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Adhikarla, V.K., Marton, F., Balogh, T. et al. Real-time adaptive content retargeting for live multi-view capture and light field display. Vis Comput 31, 1023–1032 (2015). https://doi.org/10.1007/s00371-015-1127-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-015-1127-6

Keywords

Search

Navigation