The Visual Computer

, Volume 33, Issue 11, pp 1385–1402 | Cite as

Sprite tree: an efficient image-based representation for networked virtual environments

  • Minhui Zhu
  • Géraldine Morin
  • Vincent Charvillat
  • Wei Tsang Ooi
Original Article
  • 249 Downloads

Abstract

We propose a new and efficient image-based representation for networked virtual environments, called the sprite tree. A sprite tree organizes multiple reference images efficiently and compactly for accelerating the rendering of complex virtual scenes. Using our basic construction and rendering methods, the results show that a sprite tree can efficiently organize the pixels from hundreds of distinctive reference images and accelerate the rendering of a complex scene. Furthermore, we propose the sprite view similarity to (i) largely reduce the lighting artifacts in the rendered images, and (ii) significantly reduce the redundancy and the tree size with little loss of the visual quality.

Keywords

3D image warping Image-based rendering Remote rendering Networked virtual environments 

Notes

Acknowledgments

We would like to thank Guillermo M. Leal Llaguno of Evolución Visual, Mexico for giving us permission to use the 3D model San Miguel in our work.

References

  1. 1.
    Aliaga, D.G.: Visualization of complex models using dynamic texture-based simplification. In: Proceedings of the 7th Conference on Visualization ’96, VIS ’96, pp. 101–ff (1996)Google Scholar
  2. 2.
    Bosc, E., Pepion, R., Le Callet, P., Koppel, M., Ndjiki-Nya, P., Pressigout, M., Morin, L.: Towards a new quality metric for 3-D synthesized view assessment. IEEE J. Sel. Top. Signal Process. 5(7), 1332–1343 (2011)CrossRefGoogle Scholar
  3. 3.
    Bouatouch, K., Point, G., Thomas, G.: A client-server approach to image-based rendering on mobile terminals. Research Report RR-5447, French Institute for Research in Computer Science and Automation (2005)Google Scholar
  4. 4.
    Chang, C.F., Bishop, G., Lastra, A.: LDI tree: a hierarchical representation for image-based rendering. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’99, pp. 291–298 (1999)Google Scholar
  5. 5.
    Chang, C.F., Ger, S.H.: Enhancing 3D graphics on mobile devices by image-based rendering. In: Proceedings of the Third IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing, PCM ’02, pp. 1105–1111 (2002)Google Scholar
  6. 6.
    Cheng, L., Bhushan, A., Pajarola, R., Zarki, M.E.: Real-time 3D graphics streaming using MPEG-4. In: Proceedings of the IEEE/ACM Workshop on Broadband Wireless Services and Applications, BroadWise ’04, pp. 1–16 (2004)Google Scholar
  7. 7.
    Cohen-Or, D., Noimark, Y., Zvi, T.: A server-based interactive remote walkthrough. Proceedings of the Sixth Eurographics Workshop on Multimedia 2001, 75–86 (2002)Google Scholar
  8. 8.
    Decaudin, P., Neyret, F.: Volumetric billboards. Comput. Graph. Forum 28(8), 2079–2089 (2009)Google Scholar
  9. 9.
    Décoret, X., Durand, F., Sillion, F.X., Dorsey, J.: Billboard clouds for extreme model simplification. ACM Trans. Graph. 22(3), 689–696 (2003)CrossRefGoogle Scholar
  10. 10.
    Decoret, X., Sillion, F., Schaufler, G., Dorsey, J.: Multi-layered impostors for accelerated rendering. Comput. Graph. Forum 18(3), 61–73 (1999)CrossRefGoogle Scholar
  11. 11.
    Ghiletiuc, J., Färber, M., Brüderlin, B.: Real-time remote rendering of large 3D models on smartphones using multi-layered impostors. In: Proceedings of the 6th International Conference on Computer Vision / Computer Graphics Collaboration Techniques and Applications, MIRAGE ’13, vol. 14, pp. 1–8 (2013)Google Scholar
  12. 12.
    Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The lumigraph. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’96, pp. 43–54 (1996)Google Scholar
  13. 13.
    Hoppe, H.: Progressive meshes. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’96, pp. 99–108 (1996)Google Scholar
  14. 14.
    Hudson, T.C., Mark, W.R.: Multiple image warping for remote display of rendered images. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA, Tech. rep. (1999)Google Scholar
  15. 15.
    Im, Y.H., Han, C.Y., Kim, L.S.: A method to generate soft shadows using a layered depth image and warping. IEEE Trans. Vis. Comput. Graph. 11(3), 265–272 (2005)CrossRefGoogle Scholar
  16. 16.
    Jeschke, S., Wimmer, M.: Textured depth meshes for real-time rendering of arbitrary scenes. In: Proceedings of the 13th Eurographics Workshop on Rendering, EGRW ’02, pp. 181–190. Eurographics Association (2002)Google Scholar
  17. 17.
    Lamberti, F., Sanna, A.: A streaming-based solution for remote visualization of 3D graphics on mobile devices. IEEE Trans. Vis. Comput. Graph. 13(2), 247–260 (2007)CrossRefGoogle Scholar
  18. 18.
    Lee, K., Hong, S., Kim, S.J., Rhee, I., Chong, S.: SLAW: self-similar least-action human walk. IEEE/ACM Trans. Netw. 20(2), 515–529 (2012)CrossRefGoogle Scholar
  19. 19.
    Levoy, M., Hanrahan, P.: Light field rendering. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’96, pp. 31–42 (1996)Google Scholar
  20. 20.
    Liang, H., Silva, R.N., Ooi, W.T., Motani, M.: Avatar mobility in user-created networked virtual worlds: measurements, analysis, and implications. Multimedia Tools Appl. 45(1–3), 163–190 (2009)CrossRefGoogle Scholar
  21. 21.
    Lischinski, D., Rappoport, A.: Image-based rendering for non-diffuse synthetic scenes. In: Proceedings of the Eurographics Workshop on Rendering Techniques ’98, pp. 301–314 (1998)Google Scholar
  22. 22.
    Ma, L., Duan, Q.: Image-based rendering of transparent object with caustic shadow. In: IEEE Youth Conference on Information, Computing and Telecommunication, YC-ICT ’09, pp. 538–541 (2009)Google Scholar
  23. 23.
    Maciel, P.W.C., Shirley, P.: Visual navigation of large environments using textured clusters. In: Proceedings of the 1995 Symposium on Interactive 3D graphics, I3D ’95, pp. 95–ff (1995)Google Scholar
  24. 24.
    Mark, W.R., McMillan, L., Bishop, G.: Post-rendering 3D warping. In: Proceedings of the 1997 Symposium on Interactive 3D graphics, I3D ’97, pp. 7–ff (1997)Google Scholar
  25. 25.
    McMillan, L., Bishop, G.: Plenoptic modeling: an image-based rendering system. In: Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’95, pp. 39–46 (1995)Google Scholar
  26. 26.
    McMillan Jr., L.: An image-based approach to three-dimensional computer graphics. Ph.D. thesis, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA (1997)Google Scholar
  27. 27.
    Noimark, Y., Cohen-Or, D.: Streaming scenes to MPEG-4 video-enabled devices. IEEE Comput. Graph. Appl. 23(1), 58–64 (2003)CrossRefGoogle Scholar
  28. 28.
    Phong, B.T.: Illumination for computer generated pictures. Commun ACM 18(6), 311–317 (1975)CrossRefGoogle Scholar
  29. 29.
    Popescu, V., Lastra, A., Aliaga, D., de Oliveira Neto, M.: Efficient warping for architectural walkthroughs using layered depth images. In: Proceedings of the Conference on Visualization ’98, VIS ’98, pp. 211–215 (1998)Google Scholar
  30. 30.
    Qu, H., Kaufman, A., Shao, R., Kumar, A.: A framework for sample-based rendering with o-buffers. In: Proceedings of the 14th IEEE Visualization 2003, VIS ’03, p. 58 (2003)Google Scholar
  31. 31.
    Schaufler, G., Stürzlinger, W.: A three dimensional image cache for virtual reality. Comput. Graph. Forum 15(3), 227–235 (1996)CrossRefGoogle Scholar
  32. 32.
    Shade, J., Gortler, S., He, L.w., Szeliski, R.: Layered depth images. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’98, pp. 231–242 (1998)Google Scholar
  33. 33.
    Shade, J., Lischinski, D., Salesin, D.H., DeRose, T., Snyder, J.: Hierarchical image caching for accelerated walkthroughs of complex environments. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’96, pp. 75–82 (1996)Google Scholar
  34. 34.
    Shen, S., Brouwers, N., Iosup, A., Epema, D.: Characterization of human mobility in networked virtual environments. In: Proceedings of Network and Operating System Support on Digital Audio and Video Workshop, NOSSDAV ’14, pp. 13–18 (2014)Google Scholar
  35. 35.
    Shen, S., Iosup, A.: Modeling avatar mobility of networked virtual environments. In: Proceedings of International Workshop on Massively Multiuser Virtual Environments, MMVE ’14, pp. 1–6 (2014)Google Scholar
  36. 36.
    Shi, S., Jeon, W.J., Nahrstedt, K., Campbell, R.H.: Real-time remote rendering of 3D video for mobile devices. In: Proceedings of the 17th ACM International Conference on Multimedia, MM ’09, pp. 391–400 (2009)Google Scholar
  37. 37.
    Shi, S., Kamali, M., Nahrstedt, K., Hart, J.C., Campbell, R.H.: A high-quality low-delay remote rendering system for 3D video. In: Proceedings of the International Conference on Multimedia, MM ’10, pp. 601–610 (2010)Google Scholar
  38. 38.
    Shi, S., Nahrstedt, K., Campbell, R.: A real-time remote rendering system for interactive mobile graphics. ACM Trans. Multimedia Comput. Commun. Appl. 8(3s), 1–20 (2012)CrossRefGoogle Scholar
  39. 39.
    Sillion, F., Drettakis, G., Bodelet, B.: Efficient impostor manipulation for real-time visualization of urban scenery. Comput. Graph. Forum 16(3), C207–C218 (1997)CrossRefGoogle Scholar
  40. 40.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  41. 41.
    Wilson, A., Manocha, D.: Simplifying complex environments using incremental textured depth meshes. In: ACM SIGGRAPH 2003 Papers, SIGGRAPH ’03, pp. 678–688 (2003)Google Scholar
  42. 42.
    Zhao, S., Ooi, W.T., Carlier, A., Morin, G., Charvillat, V.: 3D mesh preview streaming. In: Proceedings of the 4th ACM Multimedia Systems Conference, MMSys ’13, pp. 178–189 (2013)Google Scholar
  43. 43.
    Zhu, M., Mondet, S., Morin, G., Ooi, W.T., Cheng, W.: Towards peer-assisted rendering in networked virtual environments. In: Proceedings of the 19th ACM International Conference on Multimedia, MM ’11, pp. 183–192 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Minhui Zhu
    • 1
  • Géraldine Morin
    • 2
  • Vincent Charvillat
    • 2
  • Wei Tsang Ooi
    • 1
  1. 1.National University of SingaporeSingaporeSingapore
  2. 2.University of ToulouseToulouseFrance

Personalised recommendations