A Framework for Distributing and Migrating the User Interface in Web Apps
- 2.4k Downloads
Nowadays, the advent of mobile technologies with increasing functionality and computing power is changing the way people interact with their applications in more and more different contexts of use. This way, many traditional user interfaces are evolving towards “distributed” user ones, allowing that interaction elements can now be distributed among heterogeneous devices from different platforms. In this paper we present an HTTP-Based framework for generating and distributing UIs (User Interfaces) of custom applications, allowing device change with state preservation. We use a schema-based definition of DUIs (Distributed User Interfaces), allowing the specification of the elements to be distributed. The framework is based on open standards and supports any markup-based web language. We provide a graphic case of use implemented in HTML5.
KeywordsInteractive Session Proxy Server Color Palette Action Message Post Command
1 Introduction and Related Work
In a short period of time, the way people interact with computers has changed. The wide variety of devices that people can use today has an important effect on the way users interact with them: computers, tablets, smartphones, and so on. Particularly, new mobile devices provide ubiquitous access to information and services as well as the possibility of fulfill more and more desktop-related tasks with them.
These advances open up new possibilities for interaction, including the distribution of the User Interface (UI) among different devices. The UI can be divided, moved, copied or cloned among heterogeneous devices running the same or different operating systems, maintaining its current state. These new ways of handling the UI are considered under the emerging topics of Distributed User Interfaces (DUIs) and Migratory Interfaces (MIs). DUIs are related to the distribution of one or many elements, from one or many user interfaces, in order to support one or many users, to carry out one or many tasks, on one or many domains, in one or many contexts of use . Migratory user interfaces  are able to automatically move among diverse devices, allowing the users to continue in real-time their task after changing the device in use .
In this paper we propose a full client/server-based architecture to support DUIs that allows us to distribute the UI of any web-based application among different users in ubiquitous heterogeneous environments. The framework handles the registration process of applications and clients, as well as the distribution of the UIs and the communication between them. The state of an interactive session for each user is stored and saved between different devices. We use our formal description method for developing DUIs , our AUI model  and our schema-based approach to automatically construct a concrete DUI from an XML specification  as a basis for developing our framework.
Many research works have studied and proposed different implementations to the concepts of Distributed and Migratory User Interfaces. In  a software environment called Oz/Mozart supporting distribution is presented. It allows migrating windows and receiving events from the elements of an interface previously distributed. This is a non-HTTP-based approach and requires using non-standard web languages. In  an XML-based framework supporting collaborative web browsing is proposed. It uses a new XML language with specific tags in order to describe the UI, and XSL-T transformations to construct the HTML interface. A web page is split and then replicated to all the users. This approach needs a Java Applet at the client side. In  a framework for dynamically distribute UI’s among several devices is developed. It is based on an HTTP Interface Distribution Daemon (IDD) and RelaxNG schema language is used to describe the XHTML interface, defining constraints for each element, attribute and text values. The proposal requires implementing a new HTTP command in order to avoid server timeouts. Recently, in  a framework that supports user interface distribution in web-based and android devices is presented.
In  a partial migration web system is proposed. It uses different abstraction levels to define the UI and allows migration between big displays and mobile devices. Different modules translate the original interface into a new one suitable for the new device and a proxy server captures interaction between the browser and the original web site. In  a system for dynamic generation of web interfaces supporting migration between different platforms is proposed. The approach uses a proxy server again to adapt the content to the target platform.
The rest of the paper is organized as follows: First the basic components of the framework architecture, including elements and communication schema are discussed. Then, an example of application is described. Last section provides conclusions and further work.
2.1 Communication Model
Servlet commands to allow bi-directional communication between clients and applications.
The Servlet logs a client in the framework from a valid username and password. A unique client id is generated and then returned to the user in a cookie
Allows registering a new application. An app name and an app key are required. The Servlet looks for the application in the application.xml file. If all is correct, a unique application id is generated and then returned in a cookie
The Servlet unregister the specified client and then he/she is redirected to the login page
The Servlet unregister the application with the specified Id
Specifies the user sub-interface selected by the user. The body of the request includes all the required data. When the request is received, a new session starts and a new cookie id is sent to the user. Then, a new web page with the required sub-interface is generated, sent and rendered in the client device
When the user ends her/his session, the selected sub-interfaces are released, and then the user is redirected to the application selection page
Depending on each application, this command allows to specify different user interactions with the interface. The body of the request includes an XML document with the information needed for the Servlet to process the action. The actions are stored in the application message queue
If an application changes its state, a new event is generated and sent to the Servlet encapsulated in XML format. The event is stored in the message queue and sent to all the clients interested in such event in order to upgrade the state of their interface
This command allows both clients and application to ask the Servlet for incoming messages (actions and events)
Two different XML-based messages are used: “event” and “action”. When the user performs an action during an interactive session (e.g. clicking on a button) an action message is sent to the framework. Then, the message is forwarded to the application that has to execute the action at the server side. After that, an event message may be triggered by the application and then returned to the framework that will forward it to all the clients interested in a specific type of event. Thus, as a result of a client action over a device, an event may be triggered and sent to multiple devices, and the UIs of these devices updated accordingly. It should be noted that each event message is stored in a repository within the server, so that clients connecting later can apply for and update the status of their interface. This allows us to migrate the interface between different devices maintaining the application state.
Figure 1 shows the action/event communication model with applications on the left side and clients on the right side. The information flows between client interface and framework and between framework and clients. We also need a middleware layer composed by two new APIs: one at server-side, which acts as an intermediary between framework and applications and other one at client-side, acting as an intermediary between framework and clients.
2.2 Client Pull
Prior to the use of the framework, both clients and applications must be registered in the framework. Application registration is performed by means of an XML configuration file with a pre-defined schema specification called aplications.xml, and then stored in one of the framework folders. The schema specifies the services that an application provides. The file includes tags to specify the application folder and the sub-interfaces that can be distributed. A “state” attribute allows us to specify whether the sub-interface can or cannot be distributed to more than one user at the same time.
Another important file is globals.xml, including important configuration information for the Servlet, such as connection port, applications paths, names for the cookies and so on. Client registration is performed in a slightly different way: first, the end user connects to the framework through an URL and a list of applications is showed with available services. Second, the user selects an application sub-interface.
Then the framework runs an XML instance generator algorithm producing a valid XML instance (concrete DUI in a markup language like XHTML or HTML5), taking into account the constraints specified in the schema. Finally, the UI is sent to the client and rendered in her/his device. Each service is marked as “exclusive” or “collaborative” in the configuration file. If “collaborative” option is specified, then the sub-interface can be duplicated and used in a collaborative session among different users. If “exclusive” option is specified, the service is available only for one user. At low-level, we use an identification cookie that is used in subsequent requests in order to register a client. The client is validated by using a user name and a password. Then a session cookie is also generated and sent in order to link application and client. At the server side, each application is also provided with an identification cookie, which is sent to the framework in every new message.
3 DPICTIONARY: A Graphic Distributed Interface
Canvas: The cartoonist can draw on it, but it is also used as a viewer for the rest of users. It also includes a text box containing a word representing what the cartoonist has to draw.
Palette: This is a toolbar with some drawing tools that are distributed together: pencils, erasers, thickness, objects, colors, etc.
Players panel (Cartoonist): Includes the list of players and the answers. The cartoonist can select a player as a winner and the game ends.
Players panel (Player): This is the only interface available to the player role. It includes the canvas (a read only version), the current score label, a text box to write the answer and the list of answers of all the players (like the cartoonist’s one).
Figure 3 shows the interface for the cartoonist role with the canvas, the toolbar and the players panel. They can be displayed together of distributed between different devices. For instance, the tool palette could be displayed on a smartphone, the canvas on a tablet and the players panel on a PC. Although the player interface could be distributed equally, we provide all the elements of the interface together.
DPictionary implements several actions that user can perform: “Draw”, “Erase”, “Erase All”, “Change Object”, “Change Thickness” , “Change Color”, “New cartoon”, “Winner”, “Answer”, “New player”. Each action has its own XML schema grammar in order to specify all the required parameters so that the application can perform the action. All of them use the \(<\)action\(>\) tag with the “name” attribute. Although the current implementation already has several actions, it would be very easy to extend the application functionality extending the schema and adding new commands accordingly.
4 Conclusions and Further Work
Constraints related to the distribution process itself are defined through W3C Schema grammars. After the DUI has been defined, an XML instance generator algorithm generates a new XML instance (concrete DUI) in any markup-based language, taking into account the constraints specified in the schema. The framework allows device change with state preservation.
Our future work includes the definition of a metric that allows us to decide the most suitable distribution scheme. This metric will require the use of device profiles including the device features and the formal definition of the “optimal distribution” concept. Thus, the distribution of the elements could be decided automatically by the framework, depending on the device’s features.
This research is partially funded by the project 11859/2011 from Bancaja-UMH of Miguel Hernández University of Elche.
- 1.Vanderdonckt, J.: Distributed user interfaces: how to distribute user interface elements across users, platforms, and environments. Proceedings of X International Conference on Interaccion Persona-Ordenador (Interaccion 10) (2010)Google Scholar
- 2.Paterno, F.: User Interface design adaptation. In: Soegaard, M., Dam, R.F. (eds.) The Encyclopedia of Human-Computer Interaction, 2nd Ed. The Interaction Design Foundation, Aarhus. http://www.interaction-design.org/encyclopedia/user_interface_design_adaptation.html
- 3.Berti, S., Paternó, F., Santoro, C.: A taxonomy for migratory user interfaces. In: Gilroy, S.W., Harrison, M.D. (eds.) DSV-IS 2005. LNCS, vol. 3941, pp. 149–160. Springer, Heidelberg (2006) Google Scholar
- 4.Peñalver, A., López-Espín, J., Gallud, J., Lazcorreta, E., Botella, F.: Distributed user interfaces: specification of essential properties. In: Gallud, J.A., Tesoriero, R., Penichet, V.M. (eds.) Distributed User Interfaces. Human-Computer Interaction Series, pp. 13–21. Springer, London (2011)CrossRefGoogle Scholar
- 6.Peñalver, A., Botella, F., López-Espín, J., Gallud, J.: Defining distribution constraints in distributed user interfaces. J. Univers. Comput. Sci. 19, 831–850 (2013)Google Scholar
- 7.Grolaux, D., Van Roy, P., Vanderdonckt, J.: Migratable user interfaces: beyond migratory interfaces. In: Mobiquitous, pp. 422–430. IEEE Computer Society (2004)Google Scholar
- 8.Han, R., Perret, V., Naghshineh, M.: WebSplitter: a unified XML framework for multi-device collaborative Web browsing. In: Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, pp. 221–230 (2000)Google Scholar
- 10.Frosini, L., Paterno, F.: User interface distribution in multi-device and multi-user environments with dynamically migrating engines. In: Proceedings of Engineering Interactive Computing Systems (EICS2014), Rome, Italy (2014)Google Scholar
- 11.Ghiani, G., Patern, F., Santoro, C.: Partial Web interface migration. In: Proceedings of the International Conference on Advanced Visual Interfaces, Rome, Italy (2010)Google Scholar
- 12.Bandelloni, R., Mori, G., Patern, F.: Dynamic generation of web migratory interfaces. In: Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices and Services, New York, NY, USA (2005)Google Scholar