Skip to main content
SpringerLink
Log in

A multi-screen refactoring system for video-centric web applications

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The ubiquity of web applications and the multiplication of personal devices are major factors for the increased demand for multi-screen applications. Multi-screen applications impose challenges on the application developer and designer especially if existing single-screen applications have to be transformed to the multi-screen environment. Challenges are related to the user interface division and distribution, layout adaptation, logic re-organization, runtime synchronization and adaptation to the underlying multi-screen platform. This paper faces these challenges and proposes an end-to-end refactoring system. The system allows the re-use of existing single-screen applications to automatically create multi-screen applications. The components of the multi-screen applications have their layout adapted to small and large devices and are ready to operate synchronously to provide a complementary usage experience. Our system is quantitatively evaluated on different sets of applications containing at least one video element and interactive content. Compared to a ground truth, our segmentation approach achieves an average recall of 78 %. Our layout refactoring approach reduces horizontal scrolling by 67 % on the tested applications. Finally, we evaluate the performance of the run-time behavior of one multi-screen application that is highly dynamic, in real physical environment. With a maximum total delay of 34 ms in a LAN, our solution is realistic.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. Web components, http://webcomponents.org/

  2. KnockOut, https://github.com/knockout/knockout

  3. See http://httparchive.org

  4. See http://www.w3.org/TR/html5/dom.html#content-categories

  5. See http://www.ebizmba.com/articles/video-websites

  6. See http://mediaelementjs.com/

  7. See http://www.videojs.com/

  8. See http://jplayer.org/

  9. See http://httparchive.org/interesting.php

  10. See http://download.tsi.telecom-paristech.fr/gpac/MSoS

References

  1. Bassbouss L, Tritschler M, Steglich S, Tanaka K, Miyazaki Y (2013) Towards a multi-screen application model for the web. In: IEEE 37th Annual Computer Software and Applications Conference Workshops, Japan, pp 528–533

  2. Cai D, Yu S, Wen JR, Ma WY (2003) Vips A vision-based page segmentation algorithm. Technical report, Microsoft MSR-TR-2003-79

  3. Chen J, Zhou B, Shi J, Zhang H, Fengwu Q (2001) Function-based object model towards website adaptation. In: Proceedings of the 10th International Conference on World Wide Web, WWW ’01, New York, pp 587–596

  4. Cheng B (2012) Virtual browser for enabling multi-device web applications. In: Proceedings of the Workshop on Multi-device App Middleware, Montreal, Quebec

  5. Dufourd JC, Tritschler M, Bassbouss L, Bouazizi R, Steglich S (2013) An open platform for multiscreen services. In: The 11th European Interactive TV conference EuroITV, Como, Italy

  6. Faraday P (2000) Visually critiquing web pages. In: Multimedia’99. Springer, pp 155–166

  7. Frain B (2012) Responsive web design with HTML5 and CSS3. Packt Publishing Ltd

  8. Ghiani G, Polet J, Antila V, Mäntyjärvi J (2013) Evaluating context-aware user interface migration in multi-device environments. J Ambient Intell Humanized Comput 6(2):259–277

    Article  Google Scholar 

  9. Google The multi-screen world study. https://www.thinkwithgoogle.com/research-studies/the-new-multi-screen-world-study.html

  10. Han R, Perret V, Naghshineh M (2000) Websplitter: A unifed xml framework for multi-device collaborative web browsing. In: Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, USA, pp 221–230

  11. Hartmann B, Beaudouin-Lafon M, Mackay WE (2013) Hydrascope Creating multi-surface meta-applications through view synchronization and input multiplexing. In: Proceedings of the 2nd ACM International Symposium on Pervasive Displays, PerDis ’13. ACM, New York, pp 43–48

  12. Heinrich M, Grüneberger FJ, Springer T, Gaedke M (2013) Exploiting annotations for the rapid development of collaborative web applications, International World Wide Web Conferences Steering Committee

  13. Kohlschütter C, Nejdl W (2008) A densitometric approach to web page segmentation. In: Proceedings of the 17th ACM conference on Information and knowledge management. ACM, pp 1173–1182

  14. Miller RB (1968) Response time in man-computer conversational transactions. In: Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, Part I, AFIPS ’68 (Fall, part I). ACM, New York, pp 267–277

  15. Mozilla Semantic video with popcorn.js, 2011. accessed 22 May 2014. [Online] . Available: http://popcornjs.org/demo/semantic-video

  16. Nebeling M, Speicher M, Norrie M (2013) Crowdadapt: Enabling crowdsourced web page adaptation for individual viewing conditions and preferences. ACM, New York, pp 23–32

    Google Scholar 

  17. Pelli D, Majaj N, Raizman N, Christian C, Kim E, Palomares M (2009) Grouping in object recognition: The role of a gestalt law in letter identification. Cogn Neuropsychol 26(1):36–49

    Article  Google Scholar 

  18. Pnueli A, Bergman R, Schein S, Barkol O (2009) Web page layout via visual segmentation HP Laboratories

  19. Sanoja A, Gançarski S (2014) Block-o-matic: A web page segmentation framework. In: 2014 International Conference on Multimedia Computing and Systems (ICMCS). IEEE, pp 595–600

  20. Sarkis M, Concolato C, Dufourd JC (2014) The virtual splitter: Refactoring web applications for the multiscreen environment. In: Proceedings of the 2014 ACM Symposium on Document Engineering, DocEng ’14. ACM, pp 139–142

  21. Sarkis M, Concolato C, Dufourd JC (2015) Msos: A multi-screen-oriented web page segmentation approach. In: Proceedings of the 2015 ACM Symposium on Document Engineering, DocEng ’15. ACM, pp 85–88

  22. Sun C, Xia S, Sun D, Chen D, Shen H, Cai W (2006) Transparent adaptation of single-user applications for multi-user real-time collaboration. ACM Trans Comput-Hum Interact 13(4):531–582

    Article  Google Scholar 

  23. Twitter Bootstrap framework for responsive web design. twitter.github.com/bootstrap

  24. Vadrevu S, Gelgi F, Davulcu H (2005) Semantic partitioning of web pages. In: Web Information Systems Engineering–WISE 2005. Springer, pp 107–118

  25. Weinstein R, Klein A, Kearney M, Delgado E, Komorosko A, Bidelman E, Irish P (2013) Mutation-summary. accessed 21 February 2014. [Online]. Available: http://code.google.com/p/mutation-summary/

  26. W3C recommendation (2010) mobile web application best practices. http://www.w3.org/TR/css3-mediaqueries/

  27. Yang J, Wigdor D (2014) Panelrama Enabling easy specification of cross-device web applications. In: Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems, CHI ’14. ACM, New York, pp 2783–2792

  28. Zorrilla M, Borch N, Daoust F, Erk A, Flórez J, Lafuente A (2015) A web-based distributed architecture for multi-device adaptation in media applications. Pers Ubiquit Comput 19(5-6):803–820

    Article  Google Scholar 

  29. Zorrilla M, Tamayo I, Martin A, Dominguez A (2015) User interface adaptation for multi-device web-based media applications. In: Broadband Multimedia Systems and Broadcasting (BMSB), 2015 IEEE International Symposium. pp 1–7

  30. Zurb Foundation framework for responsive web design. http://foundation.zurb.com/

Download references

Author information

Authors and Affiliations

  1. LTCI, Télécom ParisTech, Université Paris-Saclay, 75013, Paris, France

    Mira Sarkis, Cyril Concolato & Jean-Claude Dufourd

Authors
  1. Mira Sarkis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cyril Concolato
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean-Claude Dufourd
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mira Sarkis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sarkis, M., Concolato, C. & Dufourd, JC. A multi-screen refactoring system for video-centric web applications. Multimed Tools Appl 77, 1943–1970 (2018). https://doi.org/10.1007/s11042-017-4357-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4357-y

Keywords

Search

Navigation