Skip to main content
SpringerLink
Log in

Harmonic rendering for visual coherence on mobile outdoor AR environment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Rapid developments in augmented reality (AR) and related technologies have led to increasing interest in immersive content. AR environments are created by combining virtual 3D models with a real-world video background. It is important to merge these two worlds seamlessly if users are to enjoy AR applications, but, all too often, the illumination and shading of virtual objects is not consider the real world lighting condition or does not match that of nearby real objects. In addition, visual artifacts produced when blending real and virtual objects further limit realism. In this paper, we propose a harmonic rendering technique that minimizes the visual discrepancy between the real and virtual environments to maintain visual coherence in outdoor AR. To do this, we introduce a method of estimating and approximating the Sun’s position and the sunlight direction to estimate the real sunlight intensity, as this is the most significant illumination source in outdoor AR and it provides a more realistic lighting environment for such content, reducing the mismatch between real and virtual objects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Boom BJ, Orts-Escolano S, Ning XX, McDonagh S, Sandilands P, Fisher RB (2017) Interactive light source position estimation for augmented reality with an rgb-d camera. Comput Anim Virt Worlds 28(1):e1686. https://doi.org/10.1002/cav.1686. https://onlinelibrary.wiley.com/doi/abs/10.1002/cav.1686. E1686 cav.1686

    Article  Google Scholar 

  2. Carmigniani J, Furht B, Anisetti M, Ceravolo P, Damiani E, Ivkovic M (2011) Augmented reality technologies, systems and applications. Multimed Tools Appl 51(1):341–377. https://doi.org/10.1007/s11042-010-0660-6

    Article  Google Scholar 

  3. Chatzopoulos D, Bermejo C, Huang Z, Hui P (2017) Mobile augmented reality survey: from where we are to where we go. IEEE Access 5:6917–6950. https://doi.org/10.1109/ACCESS.2017.2698164

    Article  Google Scholar 

  4. Farshad E, Oliver G (2015) Discrete light source estimation from light probes for photorealistic rendering. In: BMVC, pp 43–1

  5. Frahm JM, Koeser K, Grest D, Koch R (2005) Markerless augmented reality with light source estimation for direct illumination. In: Conference on visual media production CVMP. London, pp 211–220

  6. Höllerer T., Feiner S (2004) Mobile augmented reality. Telegeoinformatics: location-based computing and services. Taylor and Francis Books Ltd., London, p 21

    Google Scholar 

  7. Iqbal Y (1983) An introduction to solar radiation. Elsevier. https://doi.org/10.1016/B978-0-12-373750-2.X5001-0

  8. Jensen T, Andersen MS, Madsen CB (2006) Real-time image based lighting for outdoor augmented reality under dynamically changing illumination conditions. GRAPP 6:364–371

    Google Scholar 

  9. Kanbara M, Yokoya N (2004) Real-time estimation of light source environment for photorealistic augmented reality. In: Proceedings of the 17th international conference on pattern recognition, 2004. ICPR 2004., vol 2, pp 911–914. https://doi.org/10.1109/ICPR.2004.1334407

  10. Kang D, Kong P, Yoon K, Seo S (2015) Directional texture transfer for video. Multimed Tools Appl 74(1):245–258. https://doi.org/10.1007/s11042-013-1759-3

    Article  Google Scholar 

  11. Kang D, Tian F, Seo S (2017) Perceptually inspired real-time artistic style transfer for video stream. J Real-Time Image Proc 13(3):581–589. https://doi.org/10.1007/s11554-016-0612-0

    Article  Google Scholar 

  12. Kasapakis V, Gavalas D (2017) Occlusion handling in outdoors augmented reality games. Multimed Tools Appl 76(7):9829–9854. https://doi.org/10.1007/s11042-016-3581-1

    Article  Google Scholar 

  13. Kolivand H, Sunar MS (2014) Realistic real-time outdoor rendering in augmented reality. PloS one 9(9):e108,334

    Article  Google Scholar 

  14. Lab 1: Solar radiation & seasons. http://sites.gsu.edu/geog1112/solar-radiation-seasons/

  15. Liu Y, Qin X, Xu S, Nakamae E, Peng Q (2009) Light source estimation of outdoor scenes for mixed reality. Vis Comput 25(5):637–646. https://doi.org/10.1007/s00371-009-0342-4

    Article  Google Scholar 

  16. Mekni M, Lemieux A (2014) Augmented reality: applications, challenges and future trends. Appl Comput Sci, 205–214

  17. Muhammad K, Sajjad M, Mehmood I, Rho S, Baik SW (2018) Image steganography using uncorrelated color space and its application for security of visual contents in online social networks. Fut Gen Comput Syst 86:951–960. https://doi.org/10.1016/j.future.2016.11.029. http://www.sciencedirect.com/science/article/pii/S0167739X16306768

    Article  Google Scholar 

  18. Okura F, Akaguma T, Sato T, Yokoya N (2017) Addressing temporal inconsistency in indirect augmented reality. Multimed Tools Appl 76(2):2671–2695. https://doi.org/10.1007/s11042-015-3222-0

    Article  Google Scholar 

  19. Park Jh, Rho S, Jeong Cs (2014) Real-time robust 3d object tracking and estimation for surveillance system. Secur Commun Netw 7 (10):1599–1611. https://doi.org/10.1002/sec.722

    Article  Google Scholar 

  20. Park J, Seo BK, Park JI (2017) Binocular mobile augmented reality based on stereo camera tracking. J Real-Time Image Proc 13(3):571–580. https://doi.org/10.1007/s11554-016-0640-9

    Article  Google Scholar 

  21. Parra L, Sendra S, Jiménez JM, Lloret J (2016) Multimedia sensors embedded in smartphones for ambient assisted living and e-health. Multimed Tools Appl 75(21):13,271–13,297. https://doi.org/10.1007/s11042-015-2745-8

    Article  Google Scholar 

  22. Reda I, Andreas A (2004) Solar position algorithm for solar radiation applications. Solar Energy 76(5):577–589

    Article  Google Scholar 

  23. Sebillo M, Vitiello G, Paolino L, Ginige A (2016) Training emergency responders through augmented reality mobile interfaces. Multimed Tools Appl 75(16):9609–9622. https://doi.org/10.1007/s11042-015-2955-0

    Article  Google Scholar 

  24. Seo S, Lee H (2015) Pixel based stroke generation for painterly effect using maximum homogeneity neighbor filter. Multimed Tools Appl 74(10):3317–3328. https://doi.org/10.1007/s11042-013-1835-8

    Article  Google Scholar 

  25. Seo S, Kang D, Park S (2018) Real-time adaptable and coherent rendering for outdoor augmented reality. EURASIP J Image Video Process 2018(1):118. https://doi.org/10.1186/s13640-018-0357-8

    Article  Google Scholar 

  26. Shapley R, Enroth-Cugell C (1984) Visual adaptation and retinal gain controls. Progress Retinal Res 3:263–346

    Article  Google Scholar 

  27. Solar radiation & photosynthetically active radiation. https://www.fondriest.com/environmental-measurements/parameters/weather/photosynthetically-active-radiation/

  28. Sugano N, Kato H, Tachibana K (2003) The effects of shadow representation of virtual objects in augmented reality. In: The Second IEEE and ACM international symposium on mixed and augmented reality, 2003. Proceedings., pp 76–83. https://doi.org/10.1109/ISMAR.2003.1240690

  29. Sun position in c#. http://guideving.blogspot.kr/2010/08/sun-position-in-c.html

  30. van Krevelen DWF, Poelman R (2010) A survey of augmented reality technologies, applications and limitations. Int J Virt Real 9(2):1–20

    Article  Google Scholar 

  31. Yanli L, Xueying Q, Guanyu X, Qunsheng P (2010) A new approach to outdoor illumination estimation based on statistical analysis for augmented reality. Comput Anim Virt Worlds 21(3–4):321–330. https://doi.org/10.1002/cav.357. https://onlinelibrary.wiley.com/doi/abs/10.1002/cav.357

    Article  Google Scholar 

  32. Yu L, Ong SK, Nee AYC (2016) A tracking solution for mobile augmented reality based on sensor-aided marker-less tracking and panoramic mapping. Multimed Tools Appl 75(6):3199–3220. https://doi.org/10.1007/s11042-014-2430-3

    Article  Google Scholar 

Download references

Acknowledgements

This work has supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government(MSIT) (No. 2016R1D1A1B03935378) and supported by the Chung-Ang University Research Scholarship Grants in 2019

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Chung-Ang University, Seoul, Republic of Korea

    Suhwan Kwak & Jongin Choe

  2. School of Computer Art, College of Art and Technology, Chung-Ang University, Anseong, Republic of Korea

    Sanghyun Seo

Authors
  1. Suhwan Kwak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jongin Choe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sanghyun Seo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sanghyun Seo.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kwak, S., Choe, J. & Seo, S. Harmonic rendering for visual coherence on mobile outdoor AR environment. Multimed Tools Appl 79, 16141–16154 (2020). https://doi.org/10.1007/s11042-019-7628-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7628-y

Keywords

Search

Navigation