Multimedia Systems

, Volume 19, Issue 5, pp 455–474 | Cite as

Semantic multimedia remote display for mobile thin clients

  • B. Joveski
  • M. Mitrea
  • P. Simoens
  • I. J. Marshall
  • F. Prêteux
  • B. Dhoedt
Regular Paper


Current remote display technologies for mobile thin clients convert practically all types of graphical content into sequences of images rendered by the client. Consequently, important information concerning the content semantics is lost. The present paper goes beyond this bottleneck by developing a semantic multimedia remote display. The principle consists of representing the graphical content as a real-time interactive multimedia scene graph. The underlying architecture features novel components for scene-graph creation and management, as well as for user interactivity handling. The experimental setup considers the Linux X windows system and BiFS/LASeR multimedia scene technologies on the server and client sides, respectively. The implemented solution was benchmarked against currently deployed solutions (VNC and Microsoft-RDP), by considering text editing and WWW browsing applications. The quantitative assessments demonstrate: (1) visual quality expressed by seven objective metrics, e.g., PSNR values between 30 and 42 dB or SSIM values larger than 0.9999; (2) downlink bandwidth gain factors ranging from 2 to 60; (3) real-time user event management expressed by network round-trip time reduction by factors of 4–6 and by uplink bandwidth gain factors from 3 to 10; (4) feasible CPU activity, larger than in the RDP case but reduced by a factor of 1.5 with respect to the VNC-HEXTILE.


Semantic multimedia remote display Mobile thin client MPEG-4 multimedia scene (BiFS, LASeR) X Window System VNC-HEXTILE RDP 



Absolute Average Difference

AJAX HttpRequest

Asynchronous JavaScript and XML HyperText Transfer Protocol Request


Appliance Link Protocol


Advanced Video Coding


Binary Format for Scene


Central Processing Unit


Correlation Quality


European Computer Manufacturer Association


FLash Video


Graphical Device Interface


HyperText Markup Language


Image Fidelity




iPhone Operating System


International Organization for Standardization


Lightweight Application Scene Representation

Mac OS

Apple Operating System


Moving Picture Expert Group


Normalized Cross Correlation


Operating System


Personal Computer


Personal Digital Assistant


Portable Network Graphics


Portable Pixel Map


Peak Signal-to-Noise Ratio


Quality of Experience


Remote Desktop Protocol


Remote FrameBuffer


Research in Motion


Real-time Transport Protocol


Real-time Streaming Protocol


Structural Content


Synchronized Multimedia Integration Language


Scalable Vector Graphics


ShockWave Flash


Transmission Control Protocol


User Datagram Protocol


Virtual Desktop Interface


Virtual Machine


Virtual Network Computing


Virtual Reality Modeling Language


Wireless Fidelity


eXtensible HyperText Markup Language


eXtensible markup language


eXtensible MPEG-4 textual



This research was initiated under the framework of the FP7 MobiThin project. The work was partly founded by the ITEA2 SPY project.


  1. 1.
    Schlosser, D., Binzenhofer, A., Staehle, B.: Performance comparison of windows-based thin-client architectures. In: 2007 Australasian Telecommunication Networks and Applications Conference, Christchurch, New Zealand, 2–5 Dec 2007Google Scholar
  2. 2.
    Simoens, P., Praet, P., Vankeirsbilck, B., De Wachter, J., Deboosere, L., De Turck, F., Dhoedt, B., Demeester, P.: Design and implementation of a hybrid remote display protocol to optimize multimedia experience on thin client devices. In: ATNAC 2008, 7–10 Dec 2008Google Scholar
  3. 3.
    X Window System description.
  4. 4.
    VNC, Virtual Network Computing description.
  5. 5.
    NX, NoMachine description.
  6. 6.
    RDP, Microsoft—Remote Desktop Protocol: basic connectivity and graphics remote specification.
  7. 7.
    Tan, K.-J., Gong, J.-W., Wu, B.-T., Chang, D.-C., Li1, H.-Y., Hsiao, Y.-M., Chen, Y.-C., Lo, S.-W., Chu, Y.-S., Guo, J.-I.: A remote thin client system for real time multimedia streaming over VNC. In: ICME 2010, Singapore, 19–23 July 2010Google Scholar
  8. 8.
    Shen, H., Lu, Y., Wu, F., Li, S.: A high-performance remote computing platform. In: PerCom 2009, Galveston, Texas, 9–13 March 2009Google Scholar
  9. 9.
    Yasser, P.-F., Kambiz, A.-H., Alnuweiri, H.: Internet delivery of MPEG-4 object-based multimedia. In: IEEE MultiMedia, April–June 2004Google Scholar
  10. 10.
    Lethanhman, C., Isokawa, H., Kato, T.: Multipath data transmission for wireless thin clients. In: UBICOMM 2009, Sliema, Malta, 11–16 Oct 2009Google Scholar
  11. 11.
    Bannan, M., Kellogg, D.: All about android, Nielsen (2011)Google Scholar
  12. 12.
    Yan, L., Shipeng, L., Huifeng, S.: Virtualized screen: a third element for cloud mobile convergence. IEEE MultiMedia 18(2), 4–11 (2011)CrossRefGoogle Scholar
  13. 13.
    MPEG-4 BiFS standard specification, ISO/IEC JTC1/SC29/WG11 14496-11Google Scholar
  14. 14.
    Asadi, M.K., Dufourd, J.-C: Context-aware semantic adaptation of multimedia presentations. In: IEEE International Conference on Multimedia and Expo, 6–8 July 2005, Amsterdam, HollandGoogle Scholar
  15. 15.
    Izquierdo, E., Casas, J.R., Leonardi, R., Migliorati, P., O’Connor, N.E., Kompatsiaris, I., Strintzis, M.G.: Advanced content-based semantic scene analysis and information retrieval: the chema project. In: Workshop on Image Analysis for Multimedia Interactive Services, 9–11 April 2003, London, UKGoogle Scholar
  16. 16.
    Carroll, A., Heiser, G.: An analysis of power consumption in a smartphone. In: USENIX 2010, 22–25 June 2010Google Scholar
  17. 17.
  18. 18.
    Forecasts for smartphone operating system sales until 2015, Gartner.
  19. 19.
    Battista, S., Casalino, F., Lande, C.: MPEG-4: a multimedia standard for the third millennium, part 1. IEEE MultiMedia 6, 74–83 (1999)CrossRefGoogle Scholar
  20. 20.
    Battista, S., Casalino, F., Lande, C.: MPEG-4: a multimedia standard for the third millennium, part 2. IEEE MultiMedia 7, 76–84 (2000)CrossRefGoogle Scholar
  21. 21.
    MPEG-4 LASeR standard specification. ISO/IEC°JTC1/SC29/WG11 14496-20Google Scholar
  22. 22.
    Dufourd, J.-C., Avaro, O., Concolato, C.: An MPEG standard for rich media services. IEEE MultiMedia 12, 60–68 (2005)CrossRefGoogle Scholar
  23. 23.
    Adobe Flash.
  24. 24.
  25. 25.
    W3C, Scalable Vector Graphics Specification.
  26. 26.
    W3C, Timed Text Markup Language Specification.
  27. 27.
    xHTML specification.
  28. 28.
    GZIP file format specification version 4.3, rfc 1952.
  29. 29.
    Liu, Z., Saifullah, Y., Greis, M., Sreemanthula, S.: HTTP compression technique. In: Proceedings of the 2005 IEEE Wireless Communications and Networking Conference, vol. 4, pp. 2495–2500, New Orleans, USA, 13–17 March 2005Google Scholar
  30. 30.
    Song, J., Lee, B.-D.: Mobile rich media technologies: current status and future directions. KSII Trans Internet Inf Syst 5(2):359–373 (2011)Google Scholar
  31. 31.
    Ecma-262, EcmaScript documentation.
  32. 32.
    Bruno, E.J.: Ajax: asynchronous JavaScript and XML. Dr. Dobb’s J. 31(2), 32–35 (2006)MathSciNetGoogle Scholar
  33. 33.
    Mitrea, M., Simoens, P., Joveski, B., Marshall, I.J., Tanguengayte, A., Preteux, F., Dhoed, B.: BiFS based approaches to remote display for mobile thin clients. Proc. SPIE 7444, 74440F (2009). doi: 10.1117/12.828152 CrossRefGoogle Scholar
  34. 34.
    Joveski, B., Mitrea, M., Preteux, F.: MPEG-4 LASeR-based thin client remote viewer. In: EUVIP2010—European Workshop on Visual Information Processing, Paris, July 2010Google Scholar
  35. 35.
    Joveski, B., Simoens, P., Gardenghi, L., Marshall, J., Mitrea, M., Vankeirsbilck, B., Prêteux, F., Dhoed, B.: Towards a multimedia remote viewer for mobile thin clients. Proc. SPIE 7881, 788102 (2011). doi: 10.1117/12.876279 CrossRefGoogle Scholar
  36. 36.
    GPAC—Open Source Multimedia Framework.
  37. 37.
    Concolato, C., Le Feuvre, J., Moissinac, J.-C.: Design of an efficient scalable vector graphics player for constrained devices. IEEE Trans. Consumer Electron. 54(2), 895–903 (2008)CrossRefGoogle Scholar
  38. 38.
    Live555 Streaming Media, source code libraries.
  39. 39.
    UDP— User Datagram Protocol, RFC: 768.
  40. 40.
    TCP—Transmission Control Protocol specification, RFC: 793.
  41. 41.
  42. 42.
  43. 43.
    Hypertext Transfer Protocol specification, RFC2616.
  44. 44.
  45. 45.
    Beg, M.S., Muslim, A., Chang, Y.C., Tang, T.F.: Performance evaluation of error resilient tools for MPEG-4 video transmission over a mobile channel. In: Conference on Personal Wireless Communications, 15–17 Dec 2002Google Scholar
  46. 46.
    Galluccio, L.: Transmission of adaptive MPEG video over time-varying wireless channels: modeling and performance evaluation. IEEE Trans. Wirel. Commun. 4(6), 2777–2788 (2005)CrossRefGoogle Scholar
  47. 47.
    Basso, A., Kim, Y.-J., Jiang, Z.: Performance evaluation of MPEG-4 video over realistic EDGE wireless networks. In: The 5th International Symposium on Wireless Personal Multimedia Communications, vol. 3, pp. 1118–1122 (2002)Google Scholar
  48. 48.
    gEdit official text editor for the GNOME desktop environment.
  49. 49.
    Epiphany web browser for the GNOME desktop environment.
  50. 50.
    XRDP— Open source project for Remote Desktop Protocol for Linux version 0.4.2.
  51. 51.
    Joveski, B., Gardenghi, L., Mitrea, M., Prêteux, F.: Towards collaborative MPEG-4 BiFS mobile thin remote viewer. In: 15th IEEE International Symposium on Consumer Electronics, ISCE2011, Singapore, 14–17 June 2011Google Scholar
  52. 52.
    Recommendation ITU-R BT.500: Methodology for the subjective assessment of the quality of television pictures, 09/2009, ITU-T Tutorial, Video Quality Experts Group (VQEG). Objective perceptual assessment of video quality: Full reference television (2004)Google Scholar
  53. 53.
    ISO ISO/IEC JTC1/SC29/WG11 MPEG2010/N11275 Dresden, D.E., Baroncini, V., Ohm, J.-R., Sullivan, G.J.: Report of Subjective Test Results of Responses to the Joint Call for Proposals (CfP) on Video Coding Technology for High Efficiency Video Coding (HEVC) (2010)Google Scholar
  54. 54.
    Rahmoune, A., Vandergheynst, P., Frossard, P.: Flexible motion-adaptive video coding with redundant expansions. IEEE Trans. Circuits Syst. Video Technol. 16(2), 178–190 (2006)CrossRefGoogle Scholar
  55. 55.
    Skodras, A., Christopoulos, C., Ebrahimi, T.: The JPEG 2000 still Image compression standard. IEEE Signal Process. Mag. 18(5), 36–58 (2001)CrossRefGoogle Scholar
  56. 56.
    Petrazzuoli, G., Cagnazzo, M., Pesquet-Popescu, B.: High order motion interpolation for side information improvement in DVC. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP, April 2010Google Scholar
  57. 57.
    Shiang, H.-P., van der Schaar, M.: Multi-user video streaming over multi-hop wireless networks: a distributed, cross-layer approach based on priority queuing. IEEE J. Sel. Areas Commun. 25(4), 770–785 (2007)CrossRefGoogle Scholar
  58. 58.
    De Simone, F., Naccari, M., Tagliasacchi, M., Dufaux, F., Tubaro, S., Ebrahimi, T.: Subjective quality assessment of H.264/AVC video streaming with packet losses. EURASIP J. Image Video Process. 2011, 190431 (2011)CrossRefGoogle Scholar
  59. 59.
    Fry, T.C.: Probability and Its Engineering Use. D van Nostrand, Princeton (1965)Google Scholar
  60. 60.
    Walpole, R.E., Myers, R.H.: Probability and Statistics for Engineers and Scientists, 4th edn. MacMillan Publishing, New York (1989)Google Scholar
  61. 61.
    Baliga, J., Ayre, R.W.A., Hinton, K., Tucker, R.S.: Green cloud computing: balancing energy in processing, storage, and transport. Proc. IEEE 99(1), 149–167 (2011)CrossRefGoogle Scholar
  62. 62.
    Marshall, I.J., Mitrea, M., Joveski, B., Gardenghi L., Preteux, F.: Requirements on MPEG Scene Technology for Collaborative Applications, ISO/IEC JTC 1/SC 29/WG 11, No: N12206, Torino, Italy, 18–23 July 2011Google Scholar
  63. 63.
    Marshall, I.J., Mitrea, M., Joveski, B., Gardenghi L., Preteux, F.: Context and Objectives for MPEG Scene Technology with Collaborative Applications, ISO/IEC JTC 1/SC 29/WG 11, No: N12207, Torino, Italy, 18–23 July 2011Google Scholar
  64. 64.
    Marshall, I.J., Mitrea, M., Joveski, B., Preteux, F.: Use Cases for Collaborative Applications, ISO/IEC JTC 1/SC 29/WG 11, No: N12208, Torino, Italy, 18–23 July 2011Google Scholar
  65. 65.
    Marshall, I.J., Mitrea, M., Joveski, B., Preteux, F.: Draft Call for Proposals for Scene Technologies for Collaborative Applications, ISO/IEC JTC 1/SC 29/WG 11, No: N12209, Torino, Italy, 18–23 July 2011Google Scholar
  66. 66.
    HTML5 working draft by W3C.
  67. 67.
  68. 68.
  69. 69.
  70. 70.
  71. 71.
  72. 72.
  73. 73.
  74. 74.
    Govern Remote Control.
  75. 75.
  76. 76.
  77. 77.
  78. 78.
  79. 79.
  80. 80.
  81. 81.
  82. 82.
  83. 83.
    Remote Desktop Manager.
  84. 84.
  85. 85.
  86. 86.
  87. 87.
  88. 88.
  89. 89.
  90. 90.
  91. 91.
  92. 92.
  93. 93.
  94. 94.
  95. 95.
  96. 96.
  97. 97.
  98. 98.
  99. 99.
  100. 100.
  101. 101.

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • B. Joveski
    • 6
  • M. Mitrea
    • 1
  • P. Simoens
    • 2
    • 3
  • I. J. Marshall
    • 4
  • F. Prêteux
    • 5
  • B. Dhoedt
    • 2
  1. 1.ARTEMIS Department, UMR 8145, Institut Mines-Telecom, Telecom SudParisEvry CedexFrance
  2. 2.Department of Information TechnologyGhent University-IBBTGhentBelgium
  3. 3.Department INWEGhent University CollegeGhentBelgium
  4. 4.Department of ResearchPrologueUlis (Les)France
  5. 5.Research DepartmentMINES ParisTechParis Cedex 06France
  6. 6. ARTEMIS Department, Institut Mines-Telecom, Telecom SudParisEvry CedexFrance

Personalised recommendations