Advertisement

The Visual Computer

, Volume 26, Issue 6–8, pp 445–455 | Cite as

Real-time coherent stylization for augmented reality

  • Shandong Wang
  • Kangying Cai
  • Jian Lu
  • Xuehui Liu
  • Enhua Wu
Original Article

Abstract

This paper presents a new stylized augmented reality (AR) framework which can generate line drawing and abstracted shading styles. In comparison with the state-of-art work, our framework can significantly improve both the visual immersion of a single frame and the temporal coherence of augmented video streams in real time. In our framework, we first render virtual objects over the input camera images and then uniformly process the combined contents with stylization techniques. For generating line drawing stylization, we first propose a specially designed shading method to render the virtual objects, and then use an adapted Flow-based anisotropic Difference-of-Gaussion (FDoG) filter to yield the high-quality line drawing effect. For generating the abstracted stylization, a focus-guided diffusion filter and a soft color quantization operator are sequentially applied to the augmented image, and then the processed result is combined with the detected edges to produce the final abstraction effect. The presented algorithms are all sympathetic to highly parallel processing, allowing a real-time performance on contemporary graphics hardware.

Keywords

Stylization Augmented reality Line drawing Abstraction Coherent Real-time 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Cabral, B., Leedom, L.C.: Imaging vector fields using line integral convolution. In: SIGGRAPH ’93: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, pp. 263–270. ACM, New York (1993) CrossRefGoogle Scholar
  2. 2.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986) CrossRefGoogle Scholar
  3. 3.
    Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. In: SIGGRAPH ’07: ACM SIGGRAPH 2007 papers, p. 103. ACM, New York (2007) Google Scholar
  4. 4.
    Chen, J., Turk, G., MacIntyre, B.: Watercolor inspired non-photorealistic rendering for augmented reality. In: VRST ’08: Proceedings of the 2008 ACM Symposium on Virtual Reality Software and Technology, pp. 231–234. ACM, New York (2008) CrossRefGoogle Scholar
  5. 5.
    Cole, F., Sanik, K., DeCarlo, D., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S., Singh, M.: How well do line drawings depict shape?. In: SIGGRAPH ’09: ACM SIGGRAPH 2009 papers, pp. 1–9. ACM, New York (2009) Google Scholar
  6. 6.
    DeCarlo, D., Finkelstein, A., Rusinkiewicz, S.: Interactive rendering of suggestive contours with temporal coherence. In: NPAR ’04: Proceedings of the 3rd International Symposium on Non-photorealistic Animation and Rendering, pp. 15–145. ACM, New York (2004) CrossRefGoogle Scholar
  7. 7.
    DeCarlo, D., Finkelstein, A., Rusinkiewicz, S., Santella, A.: Suggestive contours for conveying shape. In: SIGGRAPH ’03: ACM SIGGRAPH 2003 Papers, pp. 848–855. ACM, New York (2003) CrossRefGoogle Scholar
  8. 8.
    DeCarlo, D., Rusinkiewicz, S.: Highlight lines for conveying shape. In: NPAR ’07: Proceedings of the 5th International Symposium on Non-photorealistic Animation and Rendering, pp. 63–70. ACM, New York (2007) Google Scholar
  9. 9.
    Fischer, J., Bartz, D.: Real-time cartoon-like stylization of AR video streams on the GPU. Technical Report WSI-2005-18, Wilhelm Schickard Institute for Computer Science, University of Tübingen (2005) Google Scholar
  10. 10.
    Fischer, J., Bartz, D.: Stylized augmented reality for improved immersion. In: VR ’05: Proceedings of the 2005 IEEE Conference on Virtual Reality, pp. 195–202, 325. IEEE Computer Society, Washington (2005) CrossRefGoogle Scholar
  11. 11.
    Fischer, J., Bartz, D., Straßer, W.: Artistic reality: Fast brush stroke stylization for augmented reality. In: VRST ’05: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp. 155–158. ACM, New York (2005) CrossRefGoogle Scholar
  12. 12.
    Fischer, J., Cunningham, D., Bartz, D., Wallraven, C., Bülthoff, H., Strasser, W.: Measuring the discernability of virtual objects in conventional and stylized augmented reality. In: Eurographics Symposium on Virtual Environments (EGVE), pp. 53–61 (2006) Google Scholar
  13. 13.
    Gooch, B., Reinhard, E., Gooch, A.: Human facial illustrations: Creation and psychophysical evaluation. ACM Trans. Graph. 23(1), 27–44 (2004) CrossRefGoogle Scholar
  14. 14.
    Haller, M.: Photorealism or/and non-photorealism in augmented reality. In: VRCAI ’04: Proceedings of the 2004 ACM SIGGRAPH International Conference on Virtual Reality Continuum and Its Applications in Industry, pp. 189–196. ACM, New York (2004) CrossRefGoogle Scholar
  15. 15.
    Hertzmann, A., Zorin, D.: Illustrating smooth surfaces. In: SIGGRAPH ’00: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 517–526. ACM/Addison-Wesley, New York (2000) CrossRefGoogle Scholar
  16. 16.
    Judd, T., Durand, F., Adelson, E.H.: Apparent ridges for line drawing. ACM Trans. Graph. 26(3), 19 (2007) CrossRefGoogle Scholar
  17. 17.
    Kang, H., Lee, S., Chui, C.K.: Coherent line drawing. In: NPAR ’07: Proceedings of the 5th International Symposium on Non-Photorealistic Animation and Rendering, pp. 43–50. ACM, New York (2007) Google Scholar
  18. 18.
    Kang, H., Lee, S., Chui, C.K.: Flow-based image abstraction. IEEE Trans. Vis. Comput. Graph. 15(1), 62–76 (2009) CrossRefGoogle Scholar
  19. 19.
    Kato, H., Billinghurst, M.: Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: IWAR ’99: Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality, p. 85 (1999) Google Scholar
  20. 20.
    Kolomenkin, M., Shimshoni, I., Tal, A.: Demarcating curves for shape illustration. ACM Trans. Graph. 27(5), 157 (2008) CrossRefGoogle Scholar
  21. 21.
    Lee, Y., Markosian, L., Lee, S., Hughes, J.F.: Line drawings via abstracted shading. In: SIGGRAPH ’07: ACM SIGGRAPH 2007 papers, p. 18. ACM, New York (2007) Google Scholar
  22. 22.
    Marr, D., Hildreth, E.: Theory of edge detection. Proc. R. Soc. Lond. B, Biol. Sci. 207(1167), 187–217 (1980) CrossRefGoogle Scholar
  23. 23.
    Ohtake, Y., Belyaev, A.G., Seidel, H.P.: Ridge-valley lines on meshes via implicit surface fitting. ACM Trans. Graph. 23(3), 609–612 (2004) CrossRefGoogle Scholar
  24. 24.
    Orzan, A., Bousseau, A., Barla, P., Thollot, J.: Structure-preserving manipulation of photographs. In: NPAR ’07: Proceedings of the 5th International Symposium on Non-Photorealistic Animation and Rendering, pp. 103–110. ACM, New York (2007) Google Scholar
  25. 25.
    Pham, T.Q., van Vliet, L.J.: Separable bilateral filtering for fast video preprocessing. In: ICME, pp. 454–457 (2005) Google Scholar
  26. 26.
    Saito, T., Takahashi, T.: Comprehensible rendering of 3-d shapes. In: SIGGRAPH ’90: Proceedings of the 17th Annual Conference on Computer Graphics and Interactive Techniques, pp. 197–206. ACM, New York (1990) CrossRefGoogle Scholar
  27. 27.
    Son, M., Kang, H., Lee, Y., Lee, S.: Abstract line drawings from 2d images. In: PG ’07: Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, pp. 333–342. IEEE Computer Society, Washington (2007) Google Scholar
  28. 28.
    Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: ICCV ’98: Proceedings of the Sixth International Conference on Computer Vision, pp. 839–846. IEEE Computer Society, Washington (1998) Google Scholar
  29. 29.
    Winnemöller, H., Olsen, S.C., Gooch, B.: Real-time video abstraction. In: SIGGRAPH ’06: ACM SIGGRAPH 2006 Papers, pp. 1221–1226. ACM, New York (2006) CrossRefGoogle Scholar
  30. 30.
    Zhao, H., Jin, X., Shen, J., Mao, X., Feng, J.: Real-time feature-aware video abstraction. Vis. Comput. 24(7–9), 727–734 (2008) CrossRefGoogle Scholar
  31. 31.
    Zhu, L., Wang, C., Zhu, G., Han, B., Wang, H., Huang, P., Wu, E.: Image spatial diffusion on GPUs. In: IEEE Asia Pacific Conference on Circuits and Systems (APCCAS 2008), pp. 610–613 (2008) Google Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • Shandong Wang
    • 1
    • 2
  • Kangying Cai
    • 3
  • Jian Lu
    • 1
  • Xuehui Liu
    • 1
  • Enhua Wu
    • 1
    • 4
  1. 1.State Key Lab. of CS, Inst. of SoftwareChinese Academy of SciencesBeijingChina
  2. 2.Graduate University of Chinese Academy of SciencesBeijingChina
  3. 3.Thomson C.R.BeijingChina
  4. 4.University of MacauMacauChina

Personalised recommendations