Skip to main content
SpringerLink
Log in

ARStudio

A low-cost virtual studio based on Augmented Reality for video production

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Film, television, and Internet productions have generated content that combines real and virtual elements in the same scene. In a traditional production pipeline, virtual elements are inserted only in the post-production stage, and consequently, these elements are only visualized after every editing process is finalized. This approach can cause problems in situations such as ones where an actor acts alongside virtual characters, due to the lack of precise reference for gaze direction, for example. Such a problem, in the traditional production pipeline, can be identified only in the post-production stage. This can make production more expensive if there is a need to recruit the actors again and reassemble the infrastructure. Costs can also be avoided if part of the production team can follow along with certain retakes or even the entire production stage remotely. In addition, if the generated content can be quickly refined and made available to all the staff still in the production stage, the post-production stage can be considerably simplified. In this paper, we present a low-cost virtual studio called ARStudio that offers a set of functionalities similar to those of professional virtual studios. In addition, our system allows the production team to remotely follow the production stage and allows content refinements to be carried out quickly through tools, usually used in the post-production stage, that are integrated into the system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Barbosa ECB (2015) Uma metodologia para a geração de conteúdos digitais baseado na utilização de estúdios virtuais com realidade aumentada. Master’s thesis, Universidade Estadual Paulista

  2. Bergh VD, Lalioti V (1999) Software chroma keying in an immersive virtual environment. South African Computer Journal 24:155–162

    Google Scholar 

  3. Blonde L, Buck M, Galli R, Niem W, Paker Y, Schmidt W, Thomas G (1996) A virtual studio for live broadcasting: the Mona Lisa project. IEEE MultiMedia 3(2):18–29. https://doi.org/10.1109/93.502291

    Article  Google Scholar 

  4. Boutaba R, Ren NN, Rasheed Y, Leon-Garcia A (2002) Distributed video production: Tasks, architecture and qos provisioning. Multimed Tools Appl 16 (1):99–136. https://doi.org/10.1023/A:1013245819429

    Article  MATH  Google Scholar 

  5. Cho H, Jung SU, Jee HK (2017) Real-time interactive AR system for broadcasting. In: 2017 IEEE Virtual Reality (VR). IEEE, pp 353–354. https://doi.org/10.1109/VR.2017.7892322

  6. De Gaspari T, Sementille AC, Vielmas DZ, Aguilar IA, Marar JaF (2014) ARSTUDIO: a virtual studio system with augmented reality features. In: Proceedings of the 13th ACM SIGGRAPH international conference on virtual-reality continuum and its applications in industry. VRCAI ’14. ACM, New York, pp 17–25. http://doi.acm.org/10.1145/2670473.2670491

  7. De Goussencourt T, Bertolino P (2015) Using the unity® game engine as a platform for advanced real time cinema image processing. In: 2015 IEEE international conference on image processing (ICIP), pp 4146–4149. https://doi.org/10.1109/ICIP.2015.7351586

  8. Epic Games (2019) Unreal engine. https://www.unrealengine.com/en-US. Accessed 14 May 2019

  9. Feng C, Feng T, Ye L (2019) When green screen meets panoramic videos: an interesting video combination framework. In: 2019 IEEE conference on multimedia information processing and retrieval (MIPR). IEEE, pp 480–485. https://doi.org/10.1109/MIPR.2019.00097

  10. Gibbs S, Arapis C, Breiteneder C, Lalioti V, Mostafawy S, Speier J (1998) Virtual studios: an overview. IEEE Multimedia 5(1):18–35. https://doi.org/10.1109/93.664740

    Article  Google Scholar 

  11. Hach T, Arias P, Bosch C, Montesa J, Gascó P (2017) Seamless 3D interaction of virtual and real objects in professional virtual studios. SMPTE Motion Imaging Journal 126(1):43–56. https://doi.org/10.5594/JMI.2016.2632398 https://doi.org/10.5594/JMI.2016.2632398

    Article  Google Scholar 

  12. Helzle V, Grau O, Knop T (2015) Digital representations of the real world: how to capture, model, and render visual reality. A K Peters/CRC Press, Boca Raton. chap Virtual Production

    Google Scholar 

  13. HIT Lab NZ (2018) ARToolkit. http://www.hitl.washington.edu/artoolkit/. Accessed 10 December 2018

  14. Hughes JF, van Dam A, McGuire M, Sklar DF, Foley JD, Feiner SK, Akeley K (2013) Computer graphics: principles and practice, 3rd edn. Addison-Wesley Professional, Boston

    Google Scholar 

  15. Industrial Light & Magic (2018) OpenEXR. https://www.openexr.com Accessed 3 December 2017

  16. Méndez R, Flores J, Castelló E, Viqueira JRR (2018) New distributed virtual tv set architecture for a synergistic operation of sensors and improved interaction between real and virtual worlds. Multimed Tools Appl 77(15):18999–19025. https://doi.org/10.1007/s11042-017-5353-y

    Article  Google Scholar 

  17. Microsoft (2018) Kinect tools and resources. https://developer.microsoft.com/en-us/windows/kinect/tools. Accessed 11 April 2018

  18. Natural Point (2018) OptiTrack. https://optitrack.com. Accessed 10 November 2018

  19. Northam L, Istead J, Kaplan CS (2012) A collaborative real time previsualization tool for video games and film. In: ACM SIGGRAPH 2012 posters. SIGGRAPH ’12. ACM, New York, pp 121:1–121:1. https://doi.org/10.1145/2342896.2343036

  20. Owen CB, Zhou J, Tang KH, Xiao F (2006) Handbook of video databases: Design and applications. CRC Press, Boca Raton. chap Augmented imagery for digital video applications

    Google Scholar 

  21. Owens J, Millerson G (2012) Television production. Focal Press, Waltham

    Book  Google Scholar 

  22. Pires K, Simon G (2015) Youtube live and twitch: a tour of user-generated live streaming systems. In: Proceedings of the 6th ACM multimedia systems conference. MMSys ’15. ACM, New York, pp 225–230. https://doi.org/10.1145/2713168.2713195

  23. PTC (2019) Vuforia developer portal. https://developer.vuforia.com/. Accessed 12 March 2019

  24. Reinhard E, Ward G, Pattanaik S, Debevec P (2006) The human visual system and {HDR} tone mapping. In: Reinhard E, Ward G, Pattanaik S, Debevec P (eds) High dynamic range imaging, the Morgan Kaufmann series in computer graphics. Morgan Kaufmann, San Francisco, pp 187–221. https://doi.org/10.1016/B978-012585263-0/50007-5

    Chapter  Google Scholar 

  25. Sanches SRR, Tokunaga DM, Silva VF, Sementille AC, Tori R (2012) Mutual occlusion between real and virtual elements in augmented reality based on fiducial markers. In: Proceedings of IEEE workshop on applications of computer vision – WACV 2012. IEEE Computer Society, Washington, pp 49–54. https://doi.org/10.1109/WACV.2012.6163037

  26. Sanches SRR, Silva VF, Nakamura R, Tori R (2013) Objective assessment of video segmentation quality for augmented reality. In: Proceedings of IEEE International Conference on Multimedia and Expo – ICME 2013. IEEE Computer Society, Washington, pp 1–6. https://doi.org/10.1109/ICME.2013.6607476

  27. Saraiji Y (2019) UnityCam. https://github.com/mrayy/UnityCam. Accessed 30 May 2019

  28. Shimoda S, Hayashi M, Kanatsugu Y (1989) New chroma-key imagining technique with hi-vision background. IEEE Trans Broadcast 35(4):357–361. https://doi.org/10.1109/11.40835

    Article  Google Scholar 

  29. Thomas G (2006) Mixed reality techniques for tv and their application for on-set and pre-visualization in film production. In: International workshop on mixed reality technology for filmmaking, pp 31–36

  30. Trottnow J, Götz K, Seibert S, Spielmann S, Helzle V, Einabadi F, Sielaff CKH, Grau O (2015) Intuitive virtual production tools for set and light editing. In: Proceedings of the 12th European conference on visual media production. CVMP ’15. ACM, New York, pp 6:1–6:8. https://doi.org/10.1145/2824840.2824851

  31. Ultrahaptics Ltd (2018) Leap motion. https://www.leapmotion.com. Accessed 5 October 2018

  32. Unity Technologies (2019) Unity. https://unity.com. Accessed 14 May 2019

  33. Wojdala A (1998) Challenges of virtual set technology. IEEE Multimedia 5(1):50–57. https://doi.org/10.1109/93.664742

    Article  Google Scholar 

  34. Wright S (2010) Digital compositing for film and video, 3rd edn. Focal Press, Waltham

    Google Scholar 

  35. Yang K, Park T (2015) K-motion: visualizing election information for live television broadcasts. Multimed Tools Appl 74(24):11631–11651. https://doi.org/10.1007/s11042-014-2253-2

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universidade Estadual Paulista “Júlio de Mesquita Filho”, Bauru, Brazil

    Ivan A. Aguilar & Antonio C. Sementille

  2. Universidade Tecnológica Federal do Paraná, Cornélio Procópio, Brazil

    Silvio R. R. Sanches

Authors
  1. Ivan A. Aguilar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio C. Sementille
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Silvio R. R. Sanches
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Silvio R. R. Sanches.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MKV 31.0 MB)

(MKV 35.7 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aguilar, I.A., Sementille, A.C. & Sanches, S.R.R. ARStudio. Multimed Tools Appl 78, 33899–33920 (2019). https://doi.org/10.1007/s11042-019-08064-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08064-4

Keywords

Search

Navigation