Skip to main content
SpringerLink
Log in

A hybrid remote rendering method for mobile applications

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

We present a hybrid remote rendering method for applications on mobile devices. In our remote rendering approach, we adopt a client-server model, where the server is responsible for rendering high-fidelity models, encoding the rendering results and sending them to the client, while the client renders low-fidelity models and overlays the high-fidelity frames received from the server on its rendering results. With this configuration, we are able to minimize the bandwidth requirements and interaction latency, since only key models are rendered in high-fidelity mode. We perform a quantitive analysis on the effectiveness of our method. Moreover, we conduct a user study on the subjective and objective effects of our method on the user experience. The results show that key model fidelity has a significant influence on the objective difficulty, while interaction latency plays an important role in the subjective difficulty. The results of the user study show how our method can benefit the users while minimizing resource requirements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. http://onlive.com

  2. https://www.nvidia.com/en-us/geforce/products/geforce-now/

  3. https://www.playstation.com/en-ca/explore/playstationnow/

  4. http://gaminganywhere.org

  5. https://www.ffmpeg.org

  6. http://store.steampowered.com/streaming/

References

  1. Bao P, Gourlay D (2006) A framework for remote rendering of 3-d scenes on limited mobile devices. IEEE Trans Multimed 8(2):382–389. https://doi.org/10.1109/TMM.2005.864337

    Article  Google Scholar 

  2. Bao P, Gourlay D, Li Y (2006) Deep compression of remotely rendered views. IEEE Trans Multimed 8(3):444–456. https://doi.org/10.1109/TMM.2006.870746

    Article  Google Scholar 

  3. Boukerche A, Pazzi RWN (2006) Remote rendering and streaming of progressive panoramas for mobile devices. In: Proceedings of the 14th annual ACM international conference on multimedia, multimedia ’06, pp 691–694

  4. Cecil J, Ramanathan P, Rahneshin V, Prakash A, Pirela-Cruz M (2013) Collaborative virtual environments for orthopedic surgery. In: 2013 IEEE international conference on automation science and engineering (CASE), pp 133–137. https://doi.org/10.1109/CoASE.2013.6654045

  5. Chang CF, Ger SH (2002) Enhancing 3d graphics on mobile devices by image-based rendering. In: Inproceedings of the 3rd IEEE pacific rim conference on multimedia: Advances in multimedia information processing, PCM ’02, pp 1105–1111

    Chapter  Google Scholar 

  6. Chang CL, Girod B (2004) Receiver-based rate-distortion optimized interactive streaming for scalable bitstreams of light fields. In: 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763), vol 3, pp 1623–1626. https://doi.org/10.1109/ICME.2004.1394561

  7. Chen KT, Chang YC, Hsu HJ, Chen DY, Huang CY, Hsu CH (2014) On the quality of service of cloud gaming systems. IEEE Trans Multimed 16 (2):480–495. https://doi.org/10.1109/TMM.2013.2291532

    Article  Google Scholar 

  8. Chen SE (1995) Quicktime vr: An image-based approach to virtual environment navigation. In: Proceedings of the 22Nd annual conference on computer graphics and interactive techniques. ACM, New York, pp 29–38. https://doi.org/10.1145/218380.218395

  9. Crassin C, Luebke D, Mara M, McGuire M, Oster B, Shirley P, Sloan PP, Wyman C (2015) CloudLight: A system for amortizing indirect lighting in real-time rendering. J Comput Graph Techniques (JCGT) 4(4):1–27. http://jcgt.org/published/0004/04/01/

    Google Scholar 

  10. Garland M, Heckbert PS (1997) Surface simplification using quadric error metrics. In: Proceedings of the 24th annual conference on computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., New York, pp 209–216. https://doi.org/10.1145/258734.258849

  11. Hemmati M, Javadtalab A, Nazari Shirehjini AA, Shirmohammadi S, Arici T (2013) Game as video: Bit rate reduction through adaptive object encoding. In: Proceeding of the 23rd ACM workshop on network and operating systems support for digital audio and video. ACM, New York, pp 7–12. https://doi.org/10.1145/2460782.2460784

  12. Hong HJ, Hsu CF, Tsai TH, Huang CY, Chen KT, Hsu CH (2015) Enabling adaptive cloud gaming in an open-source cloud gaming platform. IEEE Trans Circuits Syst Video Technol 25(12):2078–2091. https://doi.org/10.1109/TCSVT.2015.2450173

    Article  Google Scholar 

  13. Lamberti F, Sanna A (2007) A streaming-based solution for remote visualization of 3d graphics on mobile devices. IEEE Trans Vis Comput Graphics 13:247–260

    Article  Google Scholar 

  14. Levoy M (1995) Polygon-assisted jpeg and mpeg compression of synthetic images. In: Proceedings of the 22Nd annual conference on computer graphics and interactive techniques. ACM, New York, pp 21–28. https://doi.org/10.1145/218380.218392

  15. Liu C, Jia J, Zhang Q, Zhao L (2017) Lightweight WebSIM rendering framework based on cloud-baking. Proceedings of the 2017 ACM SIGSIM conference on principles of advanced discrete simulation - SIGSIM-PADS ’17 pp 221–229. http://dl.acm.org/citation.cfm?doid=3064911.3064933

  16. Liu Y, Wang S, Dey S (2014) Content-aware modeling and enhancing user experience in cloud mobile rendering and streaming. IEEE Trans Emerg Sel Topics Circuits Syst 4:43–56

    Article  Google Scholar 

  17. Lu Y, Li S, Shen H (2011) Virtualized screen: A third element for cloud-mobile convergence. IEEE Multimed 18:4–11

    Article  Google Scholar 

  18. Lu Y, Liu Y, Dey S (2017) Asymmetric and selective object rendering for optimized cloud mobile 3d display gaming user experience. Multimed Tools and Appl 76(18):18,291–18,320. https://doi.org/10.1007/s11042-016-3798-z

    Article  Google Scholar 

  19. Ma Z, Yue T, Cao X, Xu Y, Li X, Wang Y (2017) Interactive screen video streaming-based pervasive mobile workstyle. IEEE Trans Multimed 19(10):2322–2332. https://doi.org/10.1109/TMM.2017.2737944

    Article  Google Scholar 

  20. Noimark Y, Cohen-Or D (2003) Streaming scenes to mpeg-4 video-enabled devices. IEEE Comput Graph Appl 23(1):58–64. https://doi.org/10.1109/MCG.2003.1159614

    Article  Google Scholar 

  21. Shi S, Hsu CH (2015) A survey of interactive remote rendering systems. ACM Comput Surveys 47:57

    Article  Google Scholar 

  22. Shi S, Nahrstedt K, Campbell R (2012) A real-time remote rendering system for interactive mobile graphics. ACM Trans Multimed Comput Commun Appl (TOMCCAP) 8(46):43–56

    Google Scholar 

  23. Simoens P, Joveski B, Gardenghi L, Marshall IJ, Vankeirsbilck B, Mitrea M, Prêteux F, De Turck F, Dhoedt B (2012) Optimized mobile thin clients through a mpeg-4 bifs semantic remote display framework. Multimed Tools and Appl 61(2):447–470. https://doi.org/10.1007/s11042-011-0849-3

    Article  Google Scholar 

  24. Slivar I, Suznjevic M, Skorin-Kapov L (2015) The impact of video encoding parameters and game type on qoe for cloud gaming: A case study using the steam platform. In: 2015 7th international workshop on quality of multimedia experience (QoMEX), pp 1–6. https://doi.org/10.1109/QoMEX.2015.7148144

  25. Suznjevic M, Slivar I, Skorin-Kapov L (2016) Analysis and qoe evaluation of cloud gaming service adaptation under different network conditions: The case of nvidia geforce now. In: 2016 8th international conference on quality of multimedia experience (QoMEX), pp 1–6. https://doi.org/10.1109/QoMEX.2016.7498968

  26. Zhu W, Hou YT, Wang Y, Zhang YQ (1998) End-to-end modeling and simulation of mpeg-2 transport streams over atm networks with jitter. IEEE Trans Circuits Syst Video Technol 8(1):9–12. https://doi.org/10.1109/76.660823

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering and Computer Science, University of Ottawa, 800 King Edward Ave., Ottawa, ON, K1N 6N5, Canada

    Lu Sun, Hussein Al Osman & Jochen Lang

Authors
  1. Lu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hussein Al Osman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jochen Lang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jochen Lang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, L., Osman, H.A. & Lang, J. A hybrid remote rendering method for mobile applications. Multimed Tools Appl 79, 3333–3358 (2020). https://doi.org/10.1007/s11042-019-7306-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7306-0

Keywords

Search

Navigation