Skip to main content
SpringerLink
Log in
Book cover

Asian Conference on Computer Vision

ACCV 2018: Computer Vision – ACCV 2018 Workshops pp 494–509Cite as

Markerless Augmented Advertising for Sports Videos

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11367))

Abstract

Markerless augmented reality can be a challenging computer vision task, especially in live broadcast settings and in the absence of information related to the video capture such as the intrinsic camera parameters. This typically requires the assistance of a skilled artist, along with the use of advanced video editing tools in a post-production environment. We present an automated video augmentation pipeline that identifies textures of interest and overlays an advertisement onto these regions. We constrain the advertisement to be placed in a way that is aesthetic and natural. The aim is to augment the scene such that there is no longer a need for commercial breaks. In order to achieve seamless integration of the advertisement with the original video we build a 3D representation of the scene, place the advertisement in 3D, and then project it back onto the image plane. After successful placement in a single frame, we use homography-based, shape-preserving tracking such that the advertisement appears perspective correct for the duration of a video clip. The tracker is designed to handle smooth camera motion and shot boundaries.

Supported by the Institute for Pure and Applied Mathematics (IPAM) at the University of California Los Angeles, GumGum Inc. and U.S. National Science Foundation Grant DMS-0931852.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://youtu.be/ugZ-08c6IWY.

References

  1. Abadi, M., et al.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/, tensorflow.org

  2. Alcantarilla, P.F., Bartoli, A., Davison, A.J.: KAZE features. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 214–227. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33783-3_16

    Chapter  Google Scholar 

  3. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008). https://doi.org/10.1016/j.cviu.2007.09.014

    Article  Google Scholar 

  4. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986). https://doi.org/10.1109/TPAMI.1986.4767851

    Article  Google Scholar 

  5. Chang, C.H., Hsieh, K.Y., Chiang, M.C., Wu, J.L.: Virtual spotlighted advertising for tennis videos. J. Visual Commun. Image Represent. 21, 595–612 (2010)

    Article  Google Scholar 

  6. Chang, C.H., Hsieh, K.Y., Chung, M.C., Wu, J.L.: Visa: virtual spotlighted advertising. In: Proceedings of the 16th ACM International Conference on Multimedia, pp. 837–840 (2008). https://doi.org/10.1145/1459359.1459500

  7. Collobert, R., Kavukcuoglu, K., Farabet, C.: Torch7: a matlab-like environment for machine learning. In: BigLearn, NIPS Workshop (2011)

    Google Scholar 

  8. Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  9. Duda, R.O., Hart, P.E.: Use of the hough transformation to detect lines and curves in pictures. Commun. ACM 15(1), 11–15 (1972). https://doi.org/10.1145/361237.361242

    Article  Google Scholar 

  10. Durrant-whyte, H., Bailey, T.: Simultaneous localization and mapping: Part i. IEEE Robot. Autom. Mag. 13, 99–110 (2006). https://doi.org/10.1109/MRA.2006.1638022

    Article  Google Scholar 

  11. Eaton, J.W.: GNU Octave Manual. Network Theory Limited, Bristol (2002)

    Google Scholar 

  12. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results (2012). http://host.robots.ox.ac.uk/pascal/VOC/voc2012/

  13. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981). https://doi.org/10.1145/358669.358692

    Article  MathSciNet  Google Scholar 

  14. Guzzo, N.: Michigan15 (2016). https://flic.kr/p/QeVPEJ. Accessed 21 Sept 2018

  15. Han, J., de With, P.H.N.: 3-D camera modeling and its applications in sports broadcast video analysis. In: Sebe, N., Liu, Y., Zhuang, Y., Huang, T.S. (eds.) MCAM 2007. LNCS, vol. 4577, pp. 434–443. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73417-8_52

    Chapter  Google Scholar 

  16. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, New York (2003)

    MATH  Google Scholar 

  17. Kalman, R.: A new approach to linear filtering and prediction problems. J. Basic Eng. (ASME) 82D, 35–45 (1960)

    Article  Google Scholar 

  18. Li, B., Peng, K., Ying, X., Zha, H.: Simultaneous vanishing point detection and camera calibration from single images. In: Bebis, G., et al. (eds.) ISVC 2010. LNCS, vol. 6454, pp. 151–160. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-17274-8_15

    Chapter  Google Scholar 

  19. Li, Y., Wan, K.W., Yan, X., Xu, C.: Real time advertisement insertion in baseball video based on advertisement effect. In: Proceedings of the 13th Annual ACM International Conference on Multimedia, pp. 343–346 (2005). https://doi.org/10.1145/1101149.1101221

  20. Li, Z., Snavely, N.: Megadepth: learning single-view depth prediction from internet photos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    Google Scholar 

  21. Liu, H., Qiu, X., Huang, Q., Jiang, S., Xu, C.: Advertise gently - in-image advertising with low intrusiveness. In: 16th IEEE International Conference on Image Processing (ICIP), pp. 3105–3108 (2009)

    Google Scholar 

  22. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004). https://doi.org/10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  23. Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence. IJCAI 1981, vol. 2, pp. 674–679. Morgan Kaufmann Publishers Inc., San Francisco (1981). http://dl.acm.org/citation.cfm?id=1623264.1623280

  24. Medioni, G., Guy, G., Rom, H., François, A.: Real-time billboard substitution in a video stream. In: De Natale, F., Pupolin, S. (eds.) Multimedia Communications, pp. 71–84. Springer London (1999). https://doi.org/10.1007/978-1-4471-0859-7_6

    Chapter  Google Scholar 

  25. Mei, T., Guo, J., Hua, X.S., Liu, F.: Adon: toward contextual overlay in-video advertising. Multimedia Syst. 16(4–5), 335–344 (2010)

    Article  Google Scholar 

  26. Mei, T., Hua, X.S., Li, S.: Contextual in-image advertising. In: Proceedings of the 16th ACM International Conference on Multimedia, pp. 439–448. ACM (2008). https://doi.org/10.1145/1459359.1459418

  27. Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  28. Sturm, P., Triggs, B.: A factorization based algorithm for multi-image projective structure and motion. In: Buxton, B., Cipolla, R. (eds.) ECCV 1996. LNCS, vol. 1065, pp. 709–720. Springer, Heidelberg (1996). https://doi.org/10.1007/3-540-61123-1_183

    Chapter  Google Scholar 

  29. Wan, K.W., Xu, C.: Automatic content placement in sports highlights. In: 2006 IEEE International Conference on Multimedia and Expo, pp. 1893–1896 (2006)

    Google Scholar 

  30. Xu, C., Wan, K.W., Bui, S.H., Tian, Q.: Implanting virtual advertisement into broadcast soccer video. In: Aizawa, K., Nakamura, Y., Satoh, S. (eds.) PCM 2004. LNCS, vol. 3332, pp. 264–271. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30542-2_33

    Chapter  Google Scholar 

  31. Yildrim, Y.: Shotdetection (2015). https://github.com/yasinyildirim/ShotDetection

  32. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  33. Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Scene parsing through ade20k dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  34. Zhou, B., et al.: Semantic understanding of scenes through the ade20k dataset. Int. J. Comput. Vis. (2018). https://doi.org/10.1007/s11263-018-1140-0

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Williams College, Williamstown, USA

    Hallee E. Wong

  2. University of California Los Angeles, Los Angeles, CA, USA

    Osman Akar

  3. Universidad de Guanaquatro, Guanajuato, Mexico

    Emmanuel Antonio Cuevas

  4. Imperial College London, London, UK

    Iuliana Tabian

  5. GumGum Inc., Santa Monica, USA

    Divyaa Ravichandran, Iris Fu & Cambron Carter

Authors
  1. Hallee E. Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Osman Akar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emmanuel Antonio Cuevas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Iuliana Tabian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Divyaa Ravichandran
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Iris Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Cambron Carter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hallee E. Wong .

Editor information

Editors and Affiliations

  1. School of Computer Science, University of Adelaide, Adelaide, Australia

    Gustavo Carneiro

  2. Data61, Commonwealth Scientific and Industrial Research Organization, Canberra, Australia

    Shaodi You

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wong, H.E. et al. (2019). Markerless Augmented Advertising for Sports Videos. In: Carneiro, G., You, S. (eds) Computer Vision – ACCV 2018 Workshops. ACCV 2018. Lecture Notes in Computer Science(), vol 11367. Springer, Cham. https://doi.org/10.1007/978-3-030-21074-8_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-21074-8_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-21073-1

  • Online ISBN: 978-3-030-21074-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation