Skip to main content
SpringerLink
Log in

Interactive 3D video editing

  • Special Issue Paper
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We present a generic and versatile framework for interactive editing of 3D video footage. Our framework combines the advantages of conventional 2D video editing with the power of more advanced, depth-enhanced 3D video streams. Our editor takes 3D video as input and writes both 2D or 3D video formats as output. Its underlying core data structure is a novel 4D spatio-temporal representation which we call the video hypervolume. Conceptually, the processing loop comprises three fundamental operators: slicing, selection, and editing. The slicing operator allows users to visualize arbitrary hyperslices from the 4D data set. The selection operator labels subsets of the footage for spatio-temporal editing. This operator includes a 4D graph-cut based algorithm for object selection. The actual editing operators include cut & paste, affine transformations, and compositing with other media, such as images and 2D video. For high-quality rendering, we employ EWA splatting with view-dependent texturing and boundary matting. We demonstrate the applicability of our methods to post-production of 3D video.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bajaj, C.L., Pascucci, V., Rabbiolo, G., Schikorc, D.: Hypervolume visualization: a challenge in simplicity. In: Proc. IEEE/ACM Symposium on Volume Visualization 1998, pp. 95–102 (1998)

  2. Bennett, E.P., McMillan, L.: Proscenium: a framework for spatio-temporal video editing. In: Proc. ACM Multimedia 2003, pp. 177–184 (2003)

  3. Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(9), 1124–1137 (2004)

    Article  Google Scholar 

  4. Buehler, C., Bosse, M., McMillan, L., Gortler, S., Cohen, M.: Unstructured lumigraph rendering. In: Proc. ACM SIGGRAPH 2001, pp. 425–432 (2001)

  5. Carranza, J., Theobalt, C., Magnor, M.A., Seidel, H.P.: Free-viewpoint video of human actors. ACM Transactions on Graphics 22(3), 569–577 (2003)

    Article  Google Scholar 

  6. Cheung, G., Baker, S., Hodgins, J., Kanade, T.: Markerless human motion transfer. In: Proc. International Symposium on 3D Data Processing, Visualization and Transmission 2004, pp. 373–378 (2004)

  7. Fels, S., Mase, K.: Interactive video cubism. In: Proc. Workshop on New Paradigms in Information Visualization and Manipulation 99, pp. 78–82 (1999)

  8. Klein, A.W., Sloan, P.P.J., Finkelstein, A., Cohen, M.F.: Stylized video cubes. In: Proc. ACM SIGGRAPH Symposium on Computer Animation 2002 (2002)

  9. Li, Y., Sun, J., Shum, H.Y.: Video object cut and paste. ACM Transactions on Graphics 24(3), 595–600 (2005)

    Article  Google Scholar 

  10. Li, Y., Sun, J., Tang, C.K., Shum, H.Y.: Lazy snapping. ACM Transactions on Graphics 23(3), 303–308 (2004)

    Article  Google Scholar 

  11. Matusik, W., Buehler, C., Raskar, R., Gortler, S.J., McMillan, L.: Image-based visual hulls. In: Proc. ACM SIGGRAPH 2000, pp. 369–374 (2000)

  12. Nievergelt, J., Hinterberger, H., Sevcik, K.C.: The grid file: An adaptable, symmetric multikey file structure. ACM Transactions on Database Systems 9(1), 38–71 (1984)

    Article  Google Scholar 

  13. Pasko, A., Adzhiev, V., Schmitt, B., Schlick, C.: Constructive hypervolume modeling. Graphical Models 64(2) (2002)

  14. Pauly, M., Gross, M.: Spectral processing of point-sampled geometry. In: Proc. ACM SIGGRAPH 2001, pp. 379–386 (2001)

  15. Pauly, M., Gross, M., Kobbelt, L.: Efficient simplification of point-sampled geometry. In: Proc. IEEE Visualization 2002, pp. 163–170 (2002)

  16. Pfister, H., Zwicker, M., van Baar, J., Gross, M.: Surfels: surface elements as rendering primitives. In: Proc. ACM SIGGRAPH 2000, pp. 335–342 (2000)

  17. Rander, P., Narayanan, P., Kanade, T.: Virtualized reality: Constructing time-varying virtual worlds from real events. In: Proc. IEEE Visualization 1997, pp. 277–283 (1997)

  18. Rother, C., Kolmogorov, V., Blake, A.: “GrabCut” – interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics 23(3), 309–314 (2004)

    Article  Google Scholar 

  19. Sadlo, F., Weyrich, T., Peikert, R., Gross, M.: A practical structured light acquisition system for point-based geometry and texture. In: Eurographics Symposium on Point-Based Graphics 2005, pp. 89–98 (2005)

  20. Snavely, N., Zitnick, L., Kang, S.B., Cohen, M.: Stylizing 2.5-d video. In: Proc. International Symposium on Non-Photorealistic Animation and Rendering 2006, pp. 63–69 (2006)

  21. Theobalt, C., Ahmed, N., de Aguiar, E., Ziegler, G., Lensch, H., Magnor, M.A., Seidel, H.P.: Joint motion and reflectance capture for creating relightable 3D videos. Research Report MPI-I-2005-4-004, Max-Planck-Institut für Informatik (2005)

  22. Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. IEEE Transactions on Pattern Analysis and Machine Intelligence 2005 27(3), 475–480 (2005)

    Article  Google Scholar 

  23. Wang, J., Bhat, P., Colburn, R.A., Agrawala, M., Cohen, M.F.: Interactive video cutout. ACM Transactions on Graphics 24(3), 585–594 (2005)

    Article  Google Scholar 

  24. Waschbüsch, M., Würmlin, S., Cotting, D., Sadlo, F., Gross, M.: Scalable 3D video of dynamic scenes. The Visual Computer 21(8–10), 629–638 (2005)

    Article  Google Scholar 

  25. Weyrich, T., Pauly, M., Keiser, R., Heinzle, S., Scandella, S., Gross, M.: Post-processing of scanned 3D surface data. In: Eurographics Symposium on Point-Based Graphics 2004, pp. 85–94 (2004)

  26. Williams, L.: Casting curved shadows on curved surfaces. In: Proc. ACM SIGGRAPH 1978, pp. 270–274 (1978)

  27. Woodring, J., Wang, C., Shen, H.W.: High dimensional direct rendering of time-varying volumetric data. In: Proc. IEEE Visualization 2003, pp. 417–424 (2003)

  28. Würmlin, S., Lamboray, E., Gross, M.: 3D video fragments: Dynamic point samples for real-time free-viewpoint video. Computers & Graphics 28(1), 3–14 (2004)

    Article  Google Scholar 

  29. Würmlin, S., Lamboray, E., Staadt, O.G., Gross, M.H.: 3D video recorder. In: Proc. Pacific Graphics 2002, pp. 325–334 (2002)

  30. Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. ACM Transactions on Graphics 23(3), 600–608 (2004)

    Article  Google Scholar 

  31. Zwicker, M., Pauly, M., Knoll, O., Gross, M.: Pointshop 3D: an interactive system for point-based surface editing. ACM Transactions on Graphics 21(3), 322–329 (2002)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Graphics Laboratory, ETH Zürich, Zürich, Switzerland

    Michael Waschbüsch, Stephan Würmlin & Markus Gross

Authors
  1. Michael Waschbüsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephan Würmlin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Markus Gross
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Waschbüsch.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Waschbüsch, M., Würmlin, S. & Gross, M. Interactive 3D video editing. Visual Comput 22, 631–641 (2006). https://doi.org/10.1007/s00371-006-0053-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-006-0053-z

Keywords

Search

Navigation