Advertisement

Multimedia Systems

, Volume 19, Issue 5, pp 455–474 | Cite as

Semantic multimedia remote display for mobile thin clients

  • B. Joveski
  • M. Mitrea
  • P. Simoens
  • I. J. Marshall
  • F. Prêteux
  • B. Dhoedt
Regular Paper

Abstract

Current remote display technologies for mobile thin clients convert practically all types of graphical content into sequences of images rendered by the client. Consequently, important information concerning the content semantics is lost. The present paper goes beyond this bottleneck by developing a semantic multimedia remote display. The principle consists of representing the graphical content as a real-time interactive multimedia scene graph. The underlying architecture features novel components for scene-graph creation and management, as well as for user interactivity handling. The experimental setup considers the Linux X windows system and BiFS/LASeR multimedia scene technologies on the server and client sides, respectively. The implemented solution was benchmarked against currently deployed solutions (VNC and Microsoft-RDP), by considering text editing and WWW browsing applications. The quantitative assessments demonstrate: (1) visual quality expressed by seven objective metrics, e.g., PSNR values between 30 and 42 dB or SSIM values larger than 0.9999; (2) downlink bandwidth gain factors ranging from 2 to 60; (3) real-time user event management expressed by network round-trip time reduction by factors of 4–6 and by uplink bandwidth gain factors from 3 to 10; (4) feasible CPU activity, larger than in the RDP case but reduced by a factor of 1.5 with respect to the VNC-HEXTILE.

Keywords

Semantic multimedia remote display Mobile thin client MPEG-4 multimedia scene (BiFS, LASeR) X Window System VNC-HEXTILE RDP 

Abbreviations

AAD

Absolute Average Difference

AJAX HttpRequest

Asynchronous JavaScript and XML HyperText Transfer Protocol Request

ALP

Appliance Link Protocol

AVC

Advanced Video Coding

BiFS

Binary Format for Scene

CPU

Central Processing Unit

CQ

Correlation Quality

ECMA

European Computer Manufacturer Association

FLV

FLash Video

GDI

Graphical Device Interface

HTML

HyperText Markup Language

IF

Image Fidelity

I/O

Input/Output

iOS

iPhone Operating System

ISO

International Organization for Standardization

LASeR

Lightweight Application Scene Representation

Mac OS

Apple Operating System

MPEG

Moving Picture Expert Group

NCC

Normalized Cross Correlation

OS

Operating System

PC

Personal Computer

PDA

Personal Digital Assistant

png

Portable Network Graphics

ppm

Portable Pixel Map

PSNR

Peak Signal-to-Noise Ratio

QoE

Quality of Experience

RDP

Remote Desktop Protocol

RFB

Remote FrameBuffer

RIM

Research in Motion

RTP

Real-time Transport Protocol

RTSP

Real-time Streaming Protocol

SC

Structural Content

SMIL

Synchronized Multimedia Integration Language

SVG

Scalable Vector Graphics

SWF

ShockWave Flash

TCP

Transmission Control Protocol

UDP

User Datagram Protocol

VDI

Virtual Desktop Interface

VM

Virtual Machine

VNC

Virtual Network Computing

VRML

Virtual Reality Modeling Language

Wi-Fi

Wireless Fidelity

xHTML

eXtensible HyperText Markup Language

XML

eXtensible markup language

XMT

eXtensible MPEG-4 textual

Notes

Acknowledgments

This research was initiated under the framework of the FP7 MobiThin project. The work was partly founded by the ITEA2 SPY project.

References

  1. 1.
    Schlosser, D., Binzenhofer, A., Staehle, B.: Performance comparison of windows-based thin-client architectures. In: 2007 Australasian Telecommunication Networks and Applications Conference, Christchurch, New Zealand, 2–5 Dec 2007Google Scholar
  2. 2.
    Simoens, P., Praet, P., Vankeirsbilck, B., De Wachter, J., Deboosere, L., De Turck, F., Dhoedt, B., Demeester, P.: Design and implementation of a hybrid remote display protocol to optimize multimedia experience on thin client devices. In: ATNAC 2008, 7–10 Dec 2008Google Scholar
  3. 3.
    X Window System description. http://www.x.org
  4. 4.
    VNC, Virtual Network Computing description. http://www.realvnc.com
  5. 5.
    NX, NoMachine description. http://www.nomachine.com/
  6. 6.
    RDP, Microsoft—Remote Desktop Protocol: basic connectivity and graphics remote specification. http://msdn.microsoft.com/en-us/library/cc240445
  7. 7.
    Tan, K.-J., Gong, J.-W., Wu, B.-T., Chang, D.-C., Li1, H.-Y., Hsiao, Y.-M., Chen, Y.-C., Lo, S.-W., Chu, Y.-S., Guo, J.-I.: A remote thin client system for real time multimedia streaming over VNC. In: ICME 2010, Singapore, 19–23 July 2010Google Scholar
  8. 8.
    Shen, H., Lu, Y., Wu, F., Li, S.: A high-performance remote computing platform. In: PerCom 2009, Galveston, Texas, 9–13 March 2009Google Scholar
  9. 9.
    Yasser, P.-F., Kambiz, A.-H., Alnuweiri, H.: Internet delivery of MPEG-4 object-based multimedia. In: IEEE MultiMedia, April–June 2004Google Scholar
  10. 10.
    Lethanhman, C., Isokawa, H., Kato, T.: Multipath data transmission for wireless thin clients. In: UBICOMM 2009, Sliema, Malta, 11–16 Oct 2009Google Scholar
  11. 11.
    Bannan, M., Kellogg, D.: All about android, Nielsen (2011)Google Scholar
  12. 12.
    Yan, L., Shipeng, L., Huifeng, S.: Virtualized screen: a third element for cloud mobile convergence. IEEE MultiMedia 18(2), 4–11 (2011)CrossRefGoogle Scholar
  13. 13.
    MPEG-4 BiFS standard specification, ISO/IEC JTC1/SC29/WG11 14496-11Google Scholar
  14. 14.
    Asadi, M.K., Dufourd, J.-C: Context-aware semantic adaptation of multimedia presentations. In: IEEE International Conference on Multimedia and Expo, 6–8 July 2005, Amsterdam, HollandGoogle Scholar
  15. 15.
    Izquierdo, E., Casas, J.R., Leonardi, R., Migliorati, P., O’Connor, N.E., Kompatsiaris, I., Strintzis, M.G.: Advanced content-based semantic scene analysis and information retrieval: the chema project. In: Workshop on Image Analysis for Multimedia Interactive Services, 9–11 April 2003, London, UKGoogle Scholar
  16. 16.
    Carroll, A., Heiser, G.: An analysis of power consumption in a smartphone. In: USENIX 2010, 22–25 June 2010Google Scholar
  17. 17.
  18. 18.
    Forecasts for smartphone operating system sales until 2015, Gartner. http://www.rossdawsonblog.com
  19. 19.
    Battista, S., Casalino, F., Lande, C.: MPEG-4: a multimedia standard for the third millennium, part 1. IEEE MultiMedia 6, 74–83 (1999)CrossRefGoogle Scholar
  20. 20.
    Battista, S., Casalino, F., Lande, C.: MPEG-4: a multimedia standard for the third millennium, part 2. IEEE MultiMedia 7, 76–84 (2000)CrossRefGoogle Scholar
  21. 21.
    MPEG-4 LASeR standard specification. ISO/IEC°JTC1/SC29/WG11 14496-20Google Scholar
  22. 22.
    Dufourd, J.-C., Avaro, O., Concolato, C.: An MPEG standard for rich media services. IEEE MultiMedia 12, 60–68 (2005)CrossRefGoogle Scholar
  23. 23.
    Adobe Flash. http://www.adobe.com/
  24. 24.
  25. 25.
    W3C, Scalable Vector Graphics Specification. http://www.w3.org/TR/2002/WD-SVG12-20021115
  26. 26.
    W3C, Timed Text Markup Language Specification. http://www.w3.org/TR/ttaf1-dfxp/
  27. 27.
    xHTML specification. http://www.w3.org/TR/xhtml-basic/
  28. 28.
    GZIP file format specification version 4.3, rfc 1952. http://tools.ietf.org/html/rfc1952
  29. 29.
    Liu, Z., Saifullah, Y., Greis, M., Sreemanthula, S.: HTTP compression technique. In: Proceedings of the 2005 IEEE Wireless Communications and Networking Conference, vol. 4, pp. 2495–2500, New Orleans, USA, 13–17 March 2005Google Scholar
  30. 30.
    Song, J., Lee, B.-D.: Mobile rich media technologies: current status and future directions. KSII Trans Internet Inf Syst 5(2):359–373 (2011)Google Scholar
  31. 31.
    Ecma-262, EcmaScript documentation. http://www.ecmascript.org/docs.php
  32. 32.
    Bruno, E.J.: Ajax: asynchronous JavaScript and XML. Dr. Dobb’s J. 31(2), 32–35 (2006)MathSciNetGoogle Scholar
  33. 33.
    Mitrea, M., Simoens, P., Joveski, B., Marshall, I.J., Tanguengayte, A., Preteux, F., Dhoed, B.: BiFS based approaches to remote display for mobile thin clients. Proc. SPIE 7444, 74440F (2009). doi: 10.1117/12.828152 CrossRefGoogle Scholar
  34. 34.
    Joveski, B., Mitrea, M., Preteux, F.: MPEG-4 LASeR-based thin client remote viewer. In: EUVIP2010—European Workshop on Visual Information Processing, Paris, July 2010Google Scholar
  35. 35.
    Joveski, B., Simoens, P., Gardenghi, L., Marshall, J., Mitrea, M., Vankeirsbilck, B., Prêteux, F., Dhoed, B.: Towards a multimedia remote viewer for mobile thin clients. Proc. SPIE 7881, 788102 (2011). doi: 10.1117/12.876279 CrossRefGoogle Scholar
  36. 36.
    GPAC—Open Source Multimedia Framework. http://gpac.sourceforge.net/index.php
  37. 37.
    Concolato, C., Le Feuvre, J., Moissinac, J.-C.: Design of an efficient scalable vector graphics player for constrained devices. IEEE Trans. Consumer Electron. 54(2), 895–903 (2008)CrossRefGoogle Scholar
  38. 38.
    Live555 Streaming Media, source code libraries. http://www.live555.com
  39. 39.
    UDP— User Datagram Protocol, RFC: 768. http://tools.ietf.org/html/rfc768
  40. 40.
    TCP—Transmission Control Protocol specification, RFC: 793. http://www.ietf.org/rfc/rfc793.txt
  41. 41.
  42. 42.
  43. 43.
    Hypertext Transfer Protocol specification, RFC2616. http://www.http-compression.com/rfc2616.txt
  44. 44.
  45. 45.
    Beg, M.S., Muslim, A., Chang, Y.C., Tang, T.F.: Performance evaluation of error resilient tools for MPEG-4 video transmission over a mobile channel. In: Conference on Personal Wireless Communications, 15–17 Dec 2002Google Scholar
  46. 46.
    Galluccio, L.: Transmission of adaptive MPEG video over time-varying wireless channels: modeling and performance evaluation. IEEE Trans. Wirel. Commun. 4(6), 2777–2788 (2005)CrossRefGoogle Scholar
  47. 47.
    Basso, A., Kim, Y.-J., Jiang, Z.: Performance evaluation of MPEG-4 video over realistic EDGE wireless networks. In: The 5th International Symposium on Wireless Personal Multimedia Communications, vol. 3, pp. 1118–1122 (2002)Google Scholar
  48. 48.
    gEdit official text editor for the GNOME desktop environment. http://projects.gnome.org/gedit/
  49. 49.
    Epiphany web browser for the GNOME desktop environment. http://projects.gnome.org/epiphany/
  50. 50.
    XRDP— Open source project for Remote Desktop Protocol for Linux version 0.4.2. http://sourceforge.net/projects/xrdp/files/xrdp/0.4.2/
  51. 51.
    Joveski, B., Gardenghi, L., Mitrea, M., Prêteux, F.: Towards collaborative MPEG-4 BiFS mobile thin remote viewer. In: 15th IEEE International Symposium on Consumer Electronics, ISCE2011, Singapore, 14–17 June 2011Google Scholar
  52. 52.
    Recommendation ITU-R BT.500: Methodology for the subjective assessment of the quality of television pictures, 09/2009, ITU-T Tutorial, Video Quality Experts Group (VQEG). Objective perceptual assessment of video quality: Full reference television (2004)Google Scholar
  53. 53.
    ISO ISO/IEC JTC1/SC29/WG11 MPEG2010/N11275 Dresden, D.E., Baroncini, V., Ohm, J.-R., Sullivan, G.J.: Report of Subjective Test Results of Responses to the Joint Call for Proposals (CfP) on Video Coding Technology for High Efficiency Video Coding (HEVC) (2010)Google Scholar
  54. 54.
    Rahmoune, A., Vandergheynst, P., Frossard, P.: Flexible motion-adaptive video coding with redundant expansions. IEEE Trans. Circuits Syst. Video Technol. 16(2), 178–190 (2006)CrossRefGoogle Scholar
  55. 55.
    Skodras, A., Christopoulos, C., Ebrahimi, T.: The JPEG 2000 still Image compression standard. IEEE Signal Process. Mag. 18(5), 36–58 (2001)CrossRefGoogle Scholar
  56. 56.
    Petrazzuoli, G., Cagnazzo, M., Pesquet-Popescu, B.: High order motion interpolation for side information improvement in DVC. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP, April 2010Google Scholar
  57. 57.
    Shiang, H.-P., van der Schaar, M.: Multi-user video streaming over multi-hop wireless networks: a distributed, cross-layer approach based on priority queuing. IEEE J. Sel. Areas Commun. 25(4), 770–785 (2007)CrossRefGoogle Scholar
  58. 58.
    De Simone, F., Naccari, M., Tagliasacchi, M., Dufaux, F., Tubaro, S., Ebrahimi, T.: Subjective quality assessment of H.264/AVC video streaming with packet losses. EURASIP J. Image Video Process. 2011, 190431 (2011)CrossRefGoogle Scholar
  59. 59.
    Fry, T.C.: Probability and Its Engineering Use. D van Nostrand, Princeton (1965)Google Scholar
  60. 60.
    Walpole, R.E., Myers, R.H.: Probability and Statistics for Engineers and Scientists, 4th edn. MacMillan Publishing, New York (1989)Google Scholar
  61. 61.
    Baliga, J., Ayre, R.W.A., Hinton, K., Tucker, R.S.: Green cloud computing: balancing energy in processing, storage, and transport. Proc. IEEE 99(1), 149–167 (2011)CrossRefGoogle Scholar
  62. 62.
    Marshall, I.J., Mitrea, M., Joveski, B., Gardenghi L., Preteux, F.: Requirements on MPEG Scene Technology for Collaborative Applications, ISO/IEC JTC 1/SC 29/WG 11, No: N12206, Torino, Italy, 18–23 July 2011Google Scholar
  63. 63.
    Marshall, I.J., Mitrea, M., Joveski, B., Gardenghi L., Preteux, F.: Context and Objectives for MPEG Scene Technology with Collaborative Applications, ISO/IEC JTC 1/SC 29/WG 11, No: N12207, Torino, Italy, 18–23 July 2011Google Scholar
  64. 64.
    Marshall, I.J., Mitrea, M., Joveski, B., Preteux, F.: Use Cases for Collaborative Applications, ISO/IEC JTC 1/SC 29/WG 11, No: N12208, Torino, Italy, 18–23 July 2011Google Scholar
  65. 65.
    Marshall, I.J., Mitrea, M., Joveski, B., Preteux, F.: Draft Call for Proposals for Scene Technologies for Collaborative Applications, ISO/IEC JTC 1/SC 29/WG 11, No: N12209, Torino, Italy, 18–23 July 2011Google Scholar
  66. 66.
    HTML5 working draft by W3C. http://www.w3.org/TR/html5/
  67. 67.
  68. 68.
  69. 69.
  70. 70.
  71. 71.
  72. 72.
  73. 73.
  74. 74.
    Govern Remote Control. http://www.pjtec.com/
  75. 75.
  76. 76.
  77. 77.
  78. 78.
  79. 79.
  80. 80.
  81. 81.
  82. 82.
  83. 83.
    Remote Desktop Manager. http://remotedesktopmanager.com/
  84. 84.
  85. 85.
  86. 86.
  87. 87.
  88. 88.
  89. 89.
  90. 90.
  91. 91.
  92. 92.
  93. 93.
  94. 94.
  95. 95.
  96. 96.
  97. 97.
  98. 98.
  99. 99.
  100. 100.
  101. 101.

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • B. Joveski
    • 6
  • M. Mitrea
    • 1
  • P. Simoens
    • 2
    • 3
  • I. J. Marshall
    • 4
  • F. Prêteux
    • 5
  • B. Dhoedt
    • 2
  1. 1.ARTEMIS Department, UMR 8145, Institut Mines-Telecom, Telecom SudParisEvry CedexFrance
  2. 2.Department of Information TechnologyGhent University-IBBTGhentBelgium
  3. 3.Department INWEGhent University CollegeGhentBelgium
  4. 4.Department of ResearchPrologueUlis (Les)France
  5. 5.Research DepartmentMINES ParisTechParis Cedex 06France
  6. 6. ARTEMIS Department, Institut Mines-Telecom, Telecom SudParisEvry CedexFrance

Personalised recommendations