Skip to main content
Book cover

Computer Vision in Sports pp 133–160Cite as

Geometry Reconstruction of Players for Novel-View Synthesis of Sports Broadcasts

  • 2078 Accesses

Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)

Abstract

In this chapter, we present two methods for geometric reconstruction of players in standard sports broadcasts specifically designed to enable the broadcast director to generate novel views from locations where there is no physical camera (novel-view synthesis). This will significantly broaden the creative freedom of the director greatly enhancing the viewing experience. First, we propose a data-driven method based on multiview body pose estimation. This method can operate in uncontrolled environments with loosely calibrated and low resolution cameras and without restricting assumptions on the family of possible poses or motions. Second, we propose a scalable top-down patch-based method that reconstructs the geometry of the players adaptively based on the amount of detail available in the video streams. These methods are complementary to each other and together provides a more complete set of tools for novel-view synthesis for sport broadcasts.

Keywords

  • Optical Flow
  • Feature Match
  • Calibration Error
  • Temporal Coherence
  • Visual Hull

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-09396-3_7
  • Chapter length: 28 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   129.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-09396-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   169.99
Price excludes VAT (USA)
Hardcover Book
USD   169.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 7.1
Fig. 7.2
Fig. 7.3
Fig. 7.4
Fig. 7.5
Fig. 7.6
Fig. 7.7
Fig. 7.8
Fig. 7.9
Fig. 7.10
Fig. 7.11
Fig. 7.12
Fig. 7.13
Fig. 7.14
Fig. 7.15
Fig. 7.16
Fig. 7.17
Fig. 7.18
Fig. 7.19
Fig. 7.20

References

  1. Hilton A, Guillemaut J, Kilner J, Grau O, Thomas G (2011) 3D-TV production from conventional cameras for sports broadcast. IEEE Trans Broadcast 57(2):462–476

    CrossRef  Google Scholar 

  2. LiberoVision. www.liberovision.com

  3. Kuster K, Bazin J-C, Martin T, Oztireli C, Popa T, Gross M (2014) Spatio-temporal geometry fusion for multiple hybrid cameras using moving least squares surfaces. In: Eurographics

    Google Scholar 

  4. Guillemaut J-Y, Kilner J, Hilton A (2009) Robust graph-cut scene segmentation and reconstruction for free-viewpoint video of complex dynamic scenes. In: ICCV

    Google Scholar 

  5. Germann M, Popa T, Ziegler R, Keiser R, Gross M (2011) Space-time body pose estimation in uncontrolled environments. In: 3DIMPVT

    Google Scholar 

  6. Thomas GA (2006) Real-time camera pose estimation for augmenting sports scenes. In: CVMP

    Google Scholar 

  7. Hayashi K, Saito H (2006) Synthesizing free-viewpoint images from multiple view videos in Soccer stadium. In: CGIV

    Google Scholar 

  8. Germann M, Hornung A, Keiser R, Ziegler R, Würmlin S, Gross M (2010) Articulated billboards for video-based rendering. In: Eurographics

    Google Scholar 

  9. Germann M, Popa T, Keiser R, Ziegler R, Gross M (2012) Novel-view synthesis of outdoor sport events using an adaptive view-dependent geometry. In: Computer graphics forum (Proceedings of the Eurographics)

    Google Scholar 

  10. Moeslund TB, Granum E (2001) A survey of computer vision-based human motion capture. In: CVIU

    Google Scholar 

  11. Moeslund TB, Hilton A, Krüger V (2006) A survey of advances in vision-based human motion capture and analysis. In: CVIU

    Google Scholar 

  12. Vicon (2010). http://www.vicon.com

  13. Ballan L, Cortelazzo GM (2008) Marker-less motion capture of skinned models in a four camera set-up using optical flow and silhouettes. In: 3DPVT

    Google Scholar 

  14. Choi C, Baek S-M, Lee S (2008) Real-time 3d object pose estimation and tracking for natural landmark based visual servo. In: IROS

    Google Scholar 

  15. Theobalt C, de Aguiar E, Magnor MA, Theisel H, Seidel H-P (2004) Marker-free kinematic skeleton estimation from sequences of volume data. In: VRST

    Google Scholar 

  16. de Aguiar E, Stoll C, Theobalt C, Ahmed N, Seidel H-P, Thrun S (2008) Performance capture from sparse multi-view video. In: SIGGRAPH

    Google Scholar 

  17. Vlasic D, Baran I, Matusik W, Popović J (2008) Articulated mesh animation from multi-view silhouettes. In: SIGGRAPH

    Google Scholar 

  18. Mori G (2005) Guiding model search using segmentation. In: ICCV

    Google Scholar 

  19. Ferrari V, Marin-Jimenez M, Zisserman A (2008) Progressive search space reduction for human pose estimation. In: CVPR

    Google Scholar 

  20. Efros AA, Berg AC, Mori G, Malik J (2003) Recognizing action at a distance. In: ICCV

    Google Scholar 

  21. Germann M, Hornung A, Keiser R, Ziegler R, Würmlin S, Gross M (2010) Articulated billboards for video-based rendering. In: Eurographics

    Google Scholar 

  22. CGLM Database (2010). http://mocap.cs.cmu.edu

  23. Bouguet J-Y (1999) Pyramidal implementation of the Lucas Kanade feature tracker: description of the algorithm. Technical report, Intel Corporation, Microprocessor Research Labs

    Google Scholar 

  24. Schreiner J, Asirvatham A, Praun E, Hoppe H (2004) Inter-surface mapping. In: SIGGRAPH

    Google Scholar 

  25. Mor J (1978) The Levenberg-Marquardt algorithm: implementation and theory. Lecture notes in mathematics, vol 630

    Google Scholar 

  26. Laurentini A (1994) The visual hull concept for Silhouette-based image understanding. PAMI 16(2):150–162

    CrossRef  Google Scholar 

  27. Matusik V, Buehler C, Raskar R, Gortler S, McMillan L (2000) Image-based visual hulls. In: SIGGRAPH

    Google Scholar 

  28. Li M, Magnor M, Seidel H-P (2004) A hybrid hardware-accelerated algorithm for high quality rendering of visual hulls. In: Graphics interface

    Google Scholar 

  29. Grau O, Thomas GA, Hilton A, Kilner J, Starck J (2007) A Robust free-viewpoint video system for sport scenes. In: 3DTV

    Google Scholar 

  30. Petit B, Lesage JD, Menier C, Allard J, Franco JS, Raffin B, Boyer E, Faure F (2010) Multicamera real-time 3D modeling for telepresence and remote collaboration. Int J Digit Multi Broadcast

    Google Scholar 

  31. Franco J-S, Boyer E (2009) Efficient polyhedral modelling from silhouettes. PAMI 31(3):414–427

    Google Scholar 

  32. Guillemaut J-Y, Hilton A (2011) Joint multi-layer segmentation and reconstruction for free-viewpoint video applications. IJCV 93(1):73–100

    CrossRef  Google Scholar 

  33. Inamoto N, Saito S (2002) Intermediate view generation of soccer scene from multiple videos. In: ICPR

    Google Scholar 

  34. Matusik W, Pfister H (2004) 3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. In: SIGGRAPH

    Google Scholar 

  35. Bradley D, Popa T, Sheffer A, Heidrich W, Boubekeur T (2008) Markerless garment capture. In: SIGGRAPH

    Google Scholar 

  36. Ballan L, Brostow GJ, Puwein J, Pollefeys M (2010) Unstructured video-based rendering: interactive exploration of casually captured videos. In: SIGGRAPH

    Google Scholar 

  37. Fraundorfer F, Schindler K, Bischof H (2006) Piecewise planar scene reconstruction from sparse correspondences. Image Vis Comput 24(4):395–406

    CrossRef  Google Scholar 

  38. Stich T, Linz C, Albuquerque G, Magnor M (2008) View and time interpolation in image space. In: Pacific graphics

    Google Scholar 

  39. Sudipta SN, Steedly D, Szeliski R (2009) Piecewise planar stereo for image-based rendering. In: ICCV

    Google Scholar 

  40. Gallup D, Frahm J-M, Pollefeys M (2010) Piecewise planar and non-planar stereo for urban scene reconstruction. In: CVPR

    Google Scholar 

  41. Tola E, Lepetit V, Fua P (2010) Daisy: an efficient dense descriptor applied to wide baseline stereo. PAMI 32(5):815–830

    CrossRef  Google Scholar 

  42. Fleuret F, Berclaz J, Lengagne R, Fua P (2007) Multi-camera people tracking with a probabilistic occupancy map. PAMI 30(2):267–282

    CrossRef  Google Scholar 

  43. Zach C, Pock T, Bischof H (2007) A globally optimal algorithm for Robust TV-L1 range image integration. In: ICCV

    Google Scholar 

  44. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24:381–395

    CrossRef  MathSciNet  Google Scholar 

  45. Barnes C, Shechtman E, Goldman DB, Finkelstein A (2010) The generalized PatchMatch correspondence algorithm. In: ECCV

    Google Scholar 

  46. Buehler C, Bosse M, McMillan L, Gortler S, Cohen M (2001) Unstructured Lumigraph rendering. In: SIGGRAPH

    Google Scholar 

Download references

Acknowledgments

The image data is courtesy of Teleclub and LiberoVision.

Author information

Authors and Affiliations

  1. Concordia University, Montreal, Canada

    Tiberiu Popa

  2. ETH Zurich, Zurich, Switzerland

    Marcel Germann & Markus Gross

  3. Vizrt Switzerland, Nyon, Switzerland

    Remo Ziegler & Richard Keiser

Authors
  1. Tiberiu Popa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcel Germann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Remo Ziegler
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Richard Keiser
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Markus Gross
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tiberiu Popa .

Editor information

Editors and Affiliations

  1. Aalborg University, Aalborg, Denmark

    Thomas B. Moeslund

  2. BBC R&D, London, United Kingdom

    Graham Thomas

  3. University of Surrey, Guildford, Surrey, United Kingdom

    Adrian Hilton

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Popa, T., Germann, M., Ziegler, R., Keiser, R., Gross, M. (2014). Geometry Reconstruction of Players for Novel-View Synthesis of Sports Broadcasts. In: Moeslund, T., Thomas, G., Hilton, A. (eds) Computer Vision in Sports. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-09396-3_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-09396-3_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-09395-6

  • Online ISBN: 978-3-319-09396-3

  • eBook Packages: Computer ScienceComputer Science (R0)