Keywords

1 Introduction

Inadequate user interface design is repeatedly reported as a root cause of many incidents in healthcare [1, 2], avionics [3], and other safety-critical domains [4]. Design and analysis of user interfaces often requires a multidisciplinary team of human factors specialists, engineers, and end users to validate requirements, specifications, and implementation details. Rigorous formal methods tools can enable early identification of potential design issues. State-of-the-art verification tools like PVS [5], however, generally have minimal front-ends that create barriers when formal methods experts need to work in a multidisciplinary team and engage with non-experts of formal methods technologies — e.g., to validate hypotheses included in the formal models, or to discuss formal analysis results.

The tool presented in this paper, PVSio-web, significantly reduces these barriers. PVSio-web is a web-based environment for modeling and prototyping interactive (human-computer) systems in PVS, and is particularly suitable for: validating hypotheses included in formal models and formal properties before starting the verification process; demonstrating formal analysis results to engineers and domain specialists in a way that is easy to comprehend; and enabling lightweight formal analysis of user interfaces based on user-centred design methods, such as user testing and expert walkthroughs of prototypes. PVSio-web can be freely downloaded with the latest version of PVS [6] or from our repository [7].

Related Work.  SCR [8] is a toolset for the analysis of system requirements and specifications. Using SCR, one can formally specify the behavior of a system, use visual front-ends for demonstrating the system behavior based on the specifications, and use a group of formal methods tools (including PVS) for the analysis of system properties. In contrast to our tool, SCR lacks specialized functionalities needed for the analysis of user interfaces, such as rapid generation of user interface prototypes, deployment of prototypes on mobile devices, and logging of user interactions. Simulink [9] is a de-facto standard environment for model-based design and analysis of dynamic systems. It provides a graphical model editor based on Statecharts, and functions for rapid generation of realistic prototypes. Unlike our tool, Simulink offers very limited functions for prototyping user interface designs. More importantly, its architecture is not open, preventing it from being used with PVS or other formal analysis tools. In [10], an approach to develop realistic device prototypes using graph models and interactive pictures is presented, but the approach is not supported by a development environment, and the prototypes are manually crafted. PetShop [11], IVY [12] and similar verification tools for formal analysis of user interfaces lack functions for rapid generation of realistic prototypes. Other verification tools like Bandera [13] and PVSioChecker [14] are not specialized for user interface analysis, and features such as rapid prototyping are out of their intended functionalities.

Fig. 1.
figure 1

Screenshots of the main tools provided by the PVSio-web environment.

Full size image

2 PVSio-web: System Overview and Applications

PVSio-web provides a formal methods based, sophisticated graphical front-end for modeling and prototyping interactive (human-computer) systems. It transforms the animation capabilities of PVS, and enables the user to rapidly generate realistic prototypes in two steps: first, a picture of the user interface is loaded into the tool; second, programmable areas are created over the picture and linked to the formal model specifying the human-system interaction. Programmable areas for input widgets (e.g., buttons) over the user interface picture define how user actions are translated into PVS expressions that can be animated within PVSio [15], the native animation environment of PVS. Programmable areas for output widgets (e.g., displays), on the other hand, define how results returned by PVSio are rendered into visual elements of the prototype, so the visual appearance of the prototype can closely resemble the appearance of the real system in the corresponding state.

Figure 1(a) is a screenshot of PVSio-web generating a prototype, where framed boxes are programmable overlay areas: thus, our tool embeds a script in the area over button “0” that translates click actions over the button into a PVS expression click_0(st), and evaluates this expression in PVSio. The function click_0 is defined in the PVS model of the system, st being the current model state. Our tool automatically keeps track of model states during interaction with the prototype, and seamlessly replaces st with the current model state. The overlay over the display region renders the value of PVS expressions returned by PVSio (here, as numbers).

Applications.  PVSio-web has been applied successfully in the analysis of commercial medical devices. Using PVSio-web, we have:

  • demonstrated previously undetected design issues in medical devices [16, 17],

  • validated requirements for medical devices [18–21], and

  • created training material [22] for device developers and users.

For example, the prototype shown in Fig. 1(a) is one of many that have been used to analyze real medical devices, here a drug infusion pump. Our analysis focused on the data entry defining how the infusion pump responds when the user enters configuration parameters, such as therapy data or patient data. The PVS model of the infusion pump’s user interface was obtained by translating the source-code implementation of its user interface software into a PVS theory. Using PVSio-web, we generated a realistic prototype based on the formal model, and used it to perform quick exploratory analyses of the model to understand how to best formalize human factors principles as PVS theorems. An example human factors principle is consistency, asserting that the same user actions should produce the same results in logically equivalent situations. We formalized this principle as a PVS theorem that checks whether the data entry interaction of the device consistently registered button clicks in all states. This theorem allowed us to discover previously undetected issues with the decimal point (full details of the analysis are in [16]). The same prototype was also used to demonstrate the identified design issues to regulators and real device users (nurses, doctors, medical device trainers), resulting in the recognition of the safety implications of these issues. This and other prototypes generated using PVSio-web are currently used in training material for hospitals [23], device manufacturers, and regulators to raise awareness about general user interface software issues [22].

3 The PVSio-web Architecture

The architecture of the latest release of PVSio-web is shown in Fig. 2. It follows the Model-View-Controller design pattern [24], creating a clear separation between modules responsible for the behavior of the prototype and those for its visual appearance. In particular, the behavior of a prototype is entirely specified using PVS executable models animated within PVSio. PVSio is used as is, without any modification that might compromise its correctness or sound integration with PVS.

PVSio-web provides multiple facets; that is, it combines different development environments specifically designed for different target users. One facet, Simulator View, is designed for domain specialists and end users: it includes only elements and functionalities for exploring the behavior of prototypes generated with PVSio-web. The other facets are designed for developers and formal methods specialists. They provide tools for creating the visual appearance of the prototype (Prototype Builder), and for editing the PVS model. Advanced PVS users can use the Model Editor for editing and type-checking PVS models; developers who are novice PVS users can use a visual model editor (Emucharts Editor) for developing formal models using a graphical notation based on Statecharts [25]. The facets work well together, allowing people with different background and expertise to work together with the same underlying formal models.

Fig. 2.
figure 2

PVSio-web architecture. Rectangular boxes represent the main modules of the tool; arrows between boxes represent use relationships between modules.

Full size image

\(\blacktriangleright \) The Simulator View handles the execution of prototypes developed within PVSio-web, and logs user interactions with the prototype. This module renders the visual elements of the prototype, and implements functions for detecting and logging user actions over input widgets. It also translates user actions performed on the input widgets into PVS expressions; triggers the evaluation of PVS expressions in PVSio; and renders PVS expressions returned by PVSio into visual elements of the prototype. Translation of user actions into PVS expressions, and rendering of PVS expressions into visual elements are performed in real time using template scripts created with Prototype Builder. Example prototypes executed within Simulator View are illustrated in [17, 22].

\(\blacktriangleright \) The Prototype Builder automates the generation of a prototype, providing a graphical environment with functions for defining the visual aspect of the prototype (typically, a picture) and for creating programmable overlay areas that enable interaction with the prototype. Overlay areas corresponding to input widgets define which user actions are recognized (e.g., press, release, click) and how these actions are translated into PVS expressions. The translation is performed using templates that map user actions to PVS functions on the basis of naming conventions. An example template is , which translates clicks performed by the user over a button into a PVS function that takes one parameter , representing the current model state. Areas corresponding to output widgets use string filters to extract the actual value of the widget from PVS expressions returned by PVSio.

\(\blacktriangleright \) The EmuCharts Editor implements a visual editor and code generators for creating executable formal models. With this module, developers can: define states by drawing labelled boxes; define transitions by drawing labelled arrows; define variables representing relevant characteristics of the system; and generate executable models from the visual diagram. Model generators employ constructs from languages supported by popular analysis tools and programming languages: state labels are translated into enumerated type constants; state variables are translated into fields of a record type defining the system state; state transitions are translated into transition functions over system states. Language constructs for checking well-formedness of the model are automatically embedded in the generated models. For example, the PVS model generator introduces subtyping [26] relations so that consistency and coverage of conditions can be checked with the PVS type-checker (e.g., see the PVS model snippet in Fig. 1(c)).

\(\blacktriangleright \) The Model Editor is a text editor for editing formal models, providing the typical functionalities of modern IDEs (syntax highlighting, autocomplete, search, etc.) as well as a file browser to perform operations on the file system.

\(\blacktriangleright \) The PVSio Wrapper spawns PVSio processes needed for model animation, and hides the native read-eval-print loop of PVSio behind an API implementing a standard observer pattern [27], with functions for sending commands to PVSio, and for receiving call-backs when PVSio returns a result. This module implements mechanisms to disable inappropriate configurations of our tool, e.g., it disallows spawning multiple concurrent instances of PVSio for demonstrating concurrent systems (the demonstration of such systems must be based on a PVS model that explicitly defines the concurrent behavior).

\(\blacktriangleright \) The Co-Simulator creates a communication infrastructure that enables exchange of simulation events and data between PVS models animated within PVSio and models animated within other simulation environments. This module is particularly useful for the development and evaluation of complex systems. In particular, a development team can employ different modeling and analysis tools for different parts of a complex system, while using the Co-Simulator to verify system properties in a coordinated simulation environment. Example prototypes using this module to perform co-simulation of PVS models and Simulink models are described in [28, 29].

4 Implementation

The core modules of PVSio-web are entirely implemented in JavaScript, which eases the deployment of PVSio-web to mobile devices (tablets, smartphones, etc.) allowing demonstrations to be given conveniently to domain experts. A client-server architecture is used, in which the server builds on NodeJS [30] and the client relies on the JavaScript engine of web browsers. Jison [31] is employed to automatically generate language parsers from production rules. Model generators use Handlebars [32] for generating formal specifications from model templates. PVSio-web 2.0 includes model generators for PVS, MAL [12], PIM [33], and VDM-SL [34]. PVSio-web text editors build on CodeMirror [35]. PVSio-web visual editors build on D3.js [36].

PVSio-web was first released in early 2013 [37]. The tool has been continuously extended with new features, and re-engineered to improve modularity and the overall code quality. JSLint [38] and Jasmine [39] are routinely used to ensure that our implementation is compliant with established coding standards and that the code is well-formed. The latest version of PVSio-web consists of 18,000 lines of JavaScript code.

5 Conclusions and Future Directions

PVSio-web shows it is possible and productive to make realistic user interfaces, with all the benefits of web access (mobility, platform independence), connected closely to formal methods tools. PVSio-web makes professional formal methods accessible to end users and others, as is required in best practice for user interface design. We believe that our tool has the potential to improve the development of safe and dependable device user interfaces, as it facilitates using formal methods practices in an area of product design that has typically not made use of this technology. The tool is gaining popularity: it was downloaded over 1,600 times in 2014 [40]; research groups are exploring applications of the tool to the analysis of commercial products in other application domains (e.g., Honeywell is using it to analyze flight decks [41]). PVSio-web has been successfully used in tutorials [42, 43] to explain the structure of PVS models, and the meaning of PVS theorems to researchers and students that were not familiar with formal methods. Other universities [44–47] are also using our tool as a basis for projects and student theses. Current and future development directions include improved support for advanced formal verification techniques. For example, we are developing a new visual front-end, Proof Explorer [48], to ease the demonstration of formal proofs, the generation of test cases from verification results, and the development of proof strategies specialized for the analysis of user interface software (example strategies are informally described in [21]). We are additionally developing model generators and co-simulators to link our tool with other popular formal methods tools, including SAL [49], KeYmaera [50], and Uppaal [51]. We are also regularly adding new case studies in medical and other domains, e.g., for avionics and aerospace.