Advertisement

Towards end-user development of distributed user interfaces

  • Audrey Sanctorum
  • Beat Signer
Long Paper

Abstract

Over the last decade, we have seen an increasing number of solutions for distributed user interfaces (DUIs). This paper provides a detailed overview of existing DUI approaches and classify the different solutions based on the granularity of the distributed UI components, the supported interaction space as well as their support for the distribution of state. After the analysis of existing solutions, we discuss a DUI scenario and derive a number of requirements for end-user DUI development. We propose an approach where users can author their customised user interfaces based on a hypermedia metamodel and the concept of active components. We further discuss possibilities for the configuration and sharing of customised distributed user interfaces by end users where the focus is on an authoring rather than a programming approach.

Keywords

Distributed user interfaces Cross-device interaction End-user development 

Notes

Acknowledgements

The research of Audrey Sanctorum is funded by a Ph.D. Grant of the Research Foundation Flanders (FWO).

References

  1. 1.
    Bader, T., Heck, A., Beyerer, J.: Lift-and-drop: crossing boundaries in a multi-display environment by airlift. In: Proceedings of AVI 2010, International Conference on Advanced Visual Interfaces, Rome, Italy, pp. 139–146 (2010).  https://doi.org/10.1145/1842993.1843019
  2. 2.
    Balme, L., Demeure, A., Barralon, N., Coutaz, J., Calvary, G.: CAMELEON-RT: a software architecture reference model for distributed, migratable, and plastic user interfaces. In: Proceedings of EUSAI 2004, Symposium on Ambient Intelligence, Eindhoven, The Netherlands, pp. 291–302 (2004).  https://doi.org/10.1007/978-3-540-30473-9_28
  3. 3.
    Bardram, J., Gueddana, S., Houben, S., Nielsen, S.: ReticularSpaces: activity-based computing support for physically distributed and collaborative smart spaces. In: Proceedings of CHI 2012, Conference on Human Factors in Computing Systems, Austin, USA, pp. 2845–2854 (2012).  https://doi.org/10.1145/2207676.2208689
  4. 4.
    Bardram, J., Houben, S., Nielsen, S., Gueddana, S.: The design and architecture of reticularspaces: an activity-based computing framework for distributed and collaborative smartspaces. In: Proceedings of EICS 2012, Symposium on Engineering Interactive Computing Systems, Copenhagen, Denmark, pp. 269–274 (2012).  https://doi.org/10.1145/2305484.2305529
  5. 5.
    Biehl, J.T., Bailey, B.P.: ARIS: an interface for application relocation in an interactive space. In: Proceedings of GI 2004, Conference on Graphics Interface, London, Canada, pp. 107–116 (2004).  https://doi.org/10.20380/GI2004.14
  6. 6.
    Biehl, J.T., Baker, W.T., Bailey, B.P., Tan, D.S., Inkpen, K.M., Czerwinski, M.: IMPROMPTU: a new interaction framework for supporting collaboration in multiple display environments and its field evaluation for co-located software development. In: Proceedings of CHI 2008, Conference on Human Factors in Computing Systems, Florence, Italy, pp. 939–948 (2008).  https://doi.org/10.1145/1357054.1357200
  7. 7.
    Chang, T., Li, Y.: Deep shot: a framework for migrating tasks across devices using mobile phone cameras. In: Proceedings of CHI 2011, Conference on Human Factors in Computing Systems, Vancouver, Canada, pp. 2163–2172 (2011).  https://doi.org/10.1145/1978942.1979257
  8. 8.
    Chi, P.P., Li, Y.: Weave: scripting cross-device wearable interaction. In: Proceedings of CHI 2015, Conference on Human Factors in Computing Systems, Seoul, Republic of Korea, pp. 3923–3932 (2015).  https://doi.org/10.1145/2702123.2702451
  9. 9.
    Coutaz, J., Caffiau, S., Demeure, A., Crowley, J.L.: Early lessons from the development of SPOK, an end-user development environment for smart homes. In: Proceedings of UbiComp 2014, International Joint Conference on Pervasive and Ubiquitous Computing, Seattle, USA, pp. 895–902 (2014).  https://doi.org/10.1145/2638728.2641559
  10. 10.
    Danado, J., Paternò, F.: Puzzle: a mobile application development environment using a jigsaw metaphor. J. Vis. Lang. Comput. 25(4), 297–315 (2014).  https://doi.org/10.1016/j.jvlc.2014.03.005 CrossRefGoogle Scholar
  11. 11.
    Demeure, A., Sottet, J., Calvary, G., Coutaz, J., Ganneau, V., Vanderdonckt, J.: The 4C reference model for distributed user interfaces. In: Proceedings of ICAS 2008, Conference on Autonomic and Autonomous Systems, Gosier, Guadeloupe, pp. 61–69 (2008).  https://doi.org/10.1109/ICAS.2008.34
  12. 12.
    Dietz, P.H., Leigh, D.: DiamondTouch: a multi-user touch technology. In: Proceedings of UIST 2001, Symposium on User Interface Software and Technology, Orlando, USA, pp. 219–226 (2001).  https://doi.org/10.1145/502348.502389
  13. 13.
    Elmqvist, N.: Distributed user interfaces: state of the art. In: Distributed User Interfaces: Designing Interfaces for the Distributed Ecosystem, pp. 1–12 (2011).  https://doi.org/10.1007/978-1-4471-2271-5_1
  14. 14.
    Everitt, K., Shen, C., Ryall, K., Forlines, C.: MultiSpace: enabling electronic document micro-mobility in table-centric, multi-device environments. In: Proceedings of TableTop 2006, Workshop on Horizontal Interactive Human–Computer Systems, Adelaide, Australia, pp. 27–34 (2006).  https://doi.org/10.1109/TABLETOP.2006.23
  15. 15.
    Frosini, L., Manca, M., Paternò, F.: A framework for the development of distributed interactive applications. In: Proceedings of EICS 2013, Symposium on Engineering Interactive Computing Systems, London, UK, pp. 249–254 (2013).  https://doi.org/10.1145/2480296.2480328
  16. 16.
    Geronimo, L.D., Husmann, M., Patel, A., Tuerk, C., Norrie, M.C.: CTAT: tilt-and-tap across devices. In: Proceedings of ICWE 2016, International Conference on Web Engineering, Lugano, Switzerland, pp. 96–113 (2016).  https://doi.org/10.1007/978-3-319-38791-8_6
  17. 17.
    Ghiani, G., Paternò, F., Spano, L.D.: Cicero designer: an environment for end-user development of multi-device museum guides. In: Proceedings of IS-EUD, Symposium on End-User Development, Siegen, Germany, pp. 265–274 (2009).  https://doi.org/10.1007/978-3-642-00427-8_15
  18. 18.
    Hamilton, P., Wigdor, D.J.: Conductor: enabling and understanding cross-device interaction. In: Proceedings of CHI 2014, Conference on Human Factors in Computing Systems, Toronto, Canada, pp. 2773–2782 (2014).  https://doi.org/10.1145/2556288.2557170
  19. 19.
    Han, R., Perret, V., Naghshineh, M.: WebSplitter: a unified XML framework for multi-device collaborative web browsing. In: Proceedings of CSCW 2000, Conference on Computer Supported Cooperative Work, Philadelphia, USA, pp. 221–230 (2000).  https://doi.org/10.1145/358916.358993
  20. 20.
    Holloway, S., Julien, C.: The case for end-user programming of ubiquitous computing environments. In: Proceedings of FoSER 2010, Workshop on Future of Software Engineering Research, Santa Fe, USA, pp. 167–172 (2010).  https://doi.org/10.1145/1882362.1882398
  21. 21.
    Humble, J., Crabtree, A., Hemmings, T., Åkesson, K., Koleva, B., Rodden, T., Hansson, P.: “Playing with the bits”: user-configuration of ubiquitous domestic environments. In: Proceedings of UbiComp 2003, Conference on Ubiquitous Computing, Seattle, USA, pp. 256–263 (2003).  https://doi.org/10.1007/978-3-540-39653-6_20
  22. 22.
    Husmann, M., Nebeling, M., Pongelli, S., Norrie, M.C.: MultiMasher: providing architectural support and visual tools for multi-device mashups. In: Proceedings of WISE 2014, International Conference on Web Information Systems Engineering, Thessaloniki, Greece, pp. 199–214 (2014).  https://doi.org/10.1007/978-3-319-11746-1_15
  23. 23.
    Johanson, B., Fox, A., Winograd, T.: The interactive workspaces project: experiences with ubiquitous computing rooms. IEEE Pervasive Comput. 1(2), 67–74 (2002).  https://doi.org/10.1109/MPRV.2002.1012339 CrossRefGoogle Scholar
  24. 24.
    Johanson, B., Hutchins, G., Winograd, T., Stone, M.C.: PointRight: experience with flexible input redirection in interactive workspaces. In: Proceedings of UIST 2002, Symposium on User Interface Software and Technology, Paris, France, pp. 227–234 (2002).  https://doi.org/10.1145/571985.572019
  25. 25.
    Leigh, S., Schoessler, P., Heibeck, F., Maes, P., Ishii, H.: THAW: tangible interaction with see-through augmentation for smartphones on computer screens. In: Proceedings of TEI 2015, International Conference on Tangible, Embedded, and Embodied Interaction, Stanford, USA, pp. 89–96 (2015).  https://doi.org/10.1145/2677199.2680584
  26. 26.
    Marquardt, N., Hinckley, K., Greenberg, S.: Cross-device interaction via micro-mobility and F-formations. In: Proceedings of UIST 2012, Symposium on User Interface Software and Technology, Cambridge, USA, pp. 13–22 (2012).  https://doi.org/10.1145/2380116.2380121
  27. 27.
    McCartney, P., Goldman, K.J., Saff, D.E.: EUPHORIA: end-user construction of direct manipulation user interfaces for distributed applications. Softw. Concepts Tools 16(4), 147–159 (1995).  https://doi.org/10.7936/K7PZ572S Google Scholar
  28. 28.
    Melchior, J.: Distributed user interfaces in space and time. In: Proceedings of EICS 2011, Symposium on Engineering Interactive Computing System, Pisa, Italy, pp. 311–314 (2011).  https://doi.org/10.1145/1996461.1996544
  29. 29.
    Melchior, J., Grolaux, D., Vanderdonckt, J., Roy, P.V.: A toolkit for peer-to-peer distributed user interfaces: concepts, implementation, and applications. In: Proceedings of EICS 2009, Symposium on Engineering Interactive Computing System, Pittsburgh, USA, pp. 69–78 (2009).  https://doi.org/10.1145/1570433.1570449
  30. 30.
    Nebeling, M., Mintsi, T., Husmann, M., Norrie, M.C.: Interactive development of cross-device user interfaces. In: Proceedings of CHI 2014, Conference on Human Factors in Computing Systems, Toronto, Canada, pp. 2793–2802 (2014).  https://doi.org/10.1145/2556288.2556980
  31. 31.
    Nebeling, M., Teunissen, E., Husmann, M., Norrie, M.C.: XDKinect: development framework for cross-device interaction using kinect. In: Proceedings of EICS 2014, Symposium on Engineering Interactive Computing Systems, Rome, Italy, pp. 65–74 (2014).  https://doi.org/10.1145/2607023.2607024
  32. 32.
    Nebeling, M., Zimmerli, C., Husmann, M., Simmen, D.E., Norrie, M.C.: Information concepts for cross-device applications. In: Proceedings of DUI 2013, Workshop on Distributed User Interfaces: Models, Methods and Tools, London, UK, pp. 14–17 (2013)Google Scholar
  33. 33.
    Paternò, F., Santoro, C.: A logical framework for multi-device user interfaces. In: Proceedings of EICS 2012, Symposium on Engineering Interactive Computing Systems, Copenhagen, Denmark, pp. 45–50 (2012).  https://doi.org/10.1145/2305484.2305494
  34. 34.
    Rädle, R., Jetter, H., Marquardt, N., Reiterer, H., Rogers, Y.: HuddleLamp: spatially-aware mobile displays for ad-hoc around-the-table collaboration. In: Proceedings of ITS 2014, International Conference on Interactive Tabletops and Surfaces, Dresden, Germany, pp. 45–54 (2014).  https://doi.org/10.1145/2669485.2669500
  35. 35.
    Rekimoto, J.: Pick-and-drop: a direct manipulation technique for multiple computer environments. In: Proceedings of UIST 1997, Symposium on User Interface Software and Technology, Banff, Canada, pp. 31–39 (1997).  https://doi.org/10.1145/263407.263505
  36. 36.
    Robertson, S.P., Wharton, C., Ashworth, C., Franzke, M.: Dual device user interface design: PDAs and interactive television. In: Proceedings of CHI 1996, Conference on Human Factors in Computing Systems, Vancouver, Canada, pp. 79–86(1996).  https://doi.org/10.1145/238386.238408
  37. 37.
    Sanctorum, A., Signer, B.: Towards user-defined cross-device interaction. In: Proceedings of DUI 2016, Workshop on Distributed User Interfaces, Lugano, Switzerland, pp. 179–187 (2016).  https://doi.org/10.1007/978-3-319-46963-8_17
  38. 38.
    Schreiner, M., Rädle, R., Jetter, H., Reiterer, H.: Connichiwa: A framework for cross-device web applications. In: Extended Abstracts of CHI, Conference on Human Factors in Computing Systems, Seoul, Republic of Korea, pp. 2163–2168 (2015).  https://doi.org/10.1145/2702613.2732909
  39. 39.
    Schreiner, M., Rädle, R., O’Hara, K., Reiterer, H.: Deployable cross-device experiences: proposing additional web standards. In: Proceedings of Cross-Surface 2015, Workshop on Interacting with Multi-Device Ecologies in the Wild, Madeira, Portugal, pp. 17–20 (2015).  https://doi.org/10.1145/2817721.2835067
  40. 40.
    Shen, C., Everitt, K., Ryall, K.: UbiTable: impromptu face-to-face collaboration on horizontal interactive surfaces. In: Proceedings of UbiComp 2003, International Conference on Ubiquitous Computing, Seattle, USA, pp. 281–288 (2003).  https://doi.org/10.1007/978-3-540-39653-6_22
  41. 41.
    Signer, B.: Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces, Second Edition, ISBN 978-3-8370-2713-6. Books on Demand GmbH (2017)Google Scholar
  42. 42.
    Signer, B., Norrie, M.C.: As we may link: a general metamodel for hypermedia systems. In: Proceedings of ER 2007, International Conference on Conceptual Modeling. Auckland, New Zealand (2007).  https://doi.org/10.1007/978-3-540-75563-0_25
  43. 43.
    Signer, B., Norrie, M.C.: A framework for developing pervasive cross-media applications based on physical hypermedia and active components. In: Proceedings of ICPCA 2008, International Conference on Pervasive Computing and Applications. Alexandria, Egypt (2008).  https://doi.org/10.1109/ICPCA.2008.4783676
  44. 44.
    Signer, B., Norrie, M.C.: Active components as a method for coupling data and services: a database-driven application development process. In: Proceedings of ICOODB 2009, International Conference on Object Databases. Zurich, Switzerland (2009).  https://doi.org/10.1007/978-3-642-14681-7_4
  45. 45.
    Signer, B., Norrie, M.C.: A model and architecture for open cross-media annotation and link services. Inf. Syst. 36(6), 538–550 (2011).  https://doi.org/10.1016/j.is.2010.08.002 CrossRefGoogle Scholar
  46. 46.
    Streitz, N.A., Geißler, J., Holmer, T., Konomi, S., Müller-Tomfelde, C., Reischl, W., Rexroth, P., Seitz, P., Steinmetz, R.: i-LAND: an interactive landscape for creativity and innovation. In: Proceeding of the CHI 1999, Conference on Human Factors in Computing Systems, Pittsburgh, USA, pp. 120–127 (1999).  https://doi.org/10.1145/302979.303010
  47. 47.
    Tandler, P., Prante, T., Müller-Tomfelde, C., Streitz, N.A., Steinmetz, R.: Connectables: dynamic coupling of displays for the flexible creation of shared workspaces. In: Proceedings of UIST 2001, Symposium on User Interface Software and Technology. Orlando, Florida (2001).  https://doi.org/10.1145/502348.502351
  48. 48.
    Tesoriero, R.: Distributing user interfaces. In: Proceedings of DUI 2014, Workshop on Distributed User Interfaces and Multimodal Interaction, Toulouse, France, pp. 1–10 (2014).  https://doi.org/10.1145/2677356.2677669
  49. 49.
    Villanueva, P.G., Tesoriero, R., Gallud, J.A.: Distributing web components in a display ecosystem using Proxywork. In: Proceedings of HCI 2013, Conference on Human Computer Interaction, London, UK, pp. 28:1–28:6 (2013)Google Scholar
  50. 50.
    Yang, J., Wigdor, D.: Panelrama: enabling easy specification of cross-device web applications. In: Proceedings of CHI 2014, Conference on Human Factors in Computing Systems, Toronto, Canada, pp. 2783–2792 (2014).  https://doi.org/10.1145/2556288.2557199
  51. 51.
    Zhang, T., Brügge, B.: Empowering the user to build smart home applications. In: Proceedings of ICOST 2004, International Conference on Smart Homes and Health Telematic, Singapore, Republic of Singapore, pp. 170–176 (2004)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Web and Information Systems Engineering LabVrije Universiteit BrusselBrusselsBelgium

Personalised recommendations