Keywords

1 Introduction

Usability has been defined as the extent to which a player is able to understand, learn, and control the functions of a system [1]. More specifically, usability focuses on the ease of use of the independent functionalities within individual components of a game (e.g. functionality of displays, player input and usefulness of in-game feedback), making it a key feature of a system [2]. Usability in serious games is of particular importance; in accordance with cognitive load theory, the process of learning is hindered when an unusable interface demands cognitive resources that should be allocated to learning [2]. Moreover, usability can influence a variety of other outcomes, such as user satisfaction [3, 4], errors [4], and attitude [5, 6].

In order for a game to be usable, developers must consider human limits in terms of memory, perception, and attention, as well as anticipate probable user errors [7]. Thus, usability should allow users to successfully execute a given task as determined by salient objectives. Despite high performance, a system is unlikely to be utilized if users are generally dissatisfied with the mechanics or interface of the system [8]. It is important to quantify how efficiently players interact with the game to both validate gameplay and further reveal insight as to the mechanics underlying serious games. With this in mind, usability reflects the degree to which the system affords the user the ability to achieve objectives within a specific domain of use [9].

The last decade has seen a substantial rise in research on the emotional and affective aspects of the gaming experience. Traditional usability metrics are relevant, but must be adapted to the videogame context [10, 11]. Given the rise in interest in usability within the realm of HCI research, a variety of usability models and considerations of design principles have emerged with foci (e.g. emotion, affect, experience, pleasure, hedonic qualities) [12]. However, measures of these facets may not convey enough information about the efficacy of serious games.

2 Conceptual Basis of a New Approach Usability Evaluation

There are a variety of benefits to evaluating usability or user experience throughout the game development process [7]. Unfortunately, there are a multitude of shortcomings and challenges that inhibit the efficient and effective implementation of such evaluation. Most usability metrics are designed to evaluate traditional, task-oriented software. However, the nature of gameplay is quite different from that of traditional software [7]. Thus, this creates substantial difficulties for researchers investigating non-task-oriented software (i.e., the game industry). To overcome these difficulties, researchers have made efforts to adapt and create tools specific for the game industry. At this time, popular usability measures (e.g. efficiency) only convey how well a game is designed. These measures further fall short for the serious games designer. As such, it would be advantageous to utilize an underlying pedagogy to drive the game design process for designing serious games.

Why do people learn after playing a serious game versus a game for entertainment? Games for entertainment can be objective driven (think Call of Duty), but they are not designed to impart knowledge or skill outside of the game environment. Serious games, on the other hand, target the acquisition and transference of knowledge or skill outside the game environment. Serious games that achieve transference of knowledge do not work for the primary reason of having an eye-catching character or great theme music, rather it is the addition of an underlying pedagogy (activities that impart knowledge or skill) that is driving the learning process [13]. Therefore, serious games must successfully employ pedagogical methods and theories to increase the chance that knowledge is obtained [14]. In other words, in the field of clinical psychology, it is often said that treatment caused change; however, demonstrating cause does not explain why treatment produced change in behavior. Rather, it is a mechanism that explains how a treatment translates into events that lead to changes in behavior [15].

Given the educational nature of serious games, we suggest that usability in serious games must go above and beyond interface usability. This does not suggest that a sound underlying pedagogy will overcome all traditional UX principles; rather, despite the creation of a usable system, if the “player” does not evaluate the system as being usable can real transfer of knowledge occur? Thus, within the context of serious games that make use of model-based training, we suggest a need for a metric to assess whether a serious game integrates modeling in a user-friendly manner in serious games.

A variety of learning theories exist that have been applied to serious games [16]. Among the easiest and most effective pedagogical methods that can be implemented for serious games is behavior modeling. Modeling, as the pedagogical driver of learning and transference of knowledge, in the context described by Kazdin [15], is the mechanism of action that explains how a serious game translates into the transference of knowledge and skill. In order to effectively utilize modeling, we must first understand the history and principles of modeling.

2.1 Modeling as an Instructional Approach

Modeling has its roots in Albert Bandura’s version of Social Learning Theory [16], later renamed Social Cognitive Theory [17]. Bandura explains, “Learning would be exceedingly laborious, not to mention hazardous, if people had to rely solely on the effects of their own actions to inform them what to do. Fortunately, most human behavior is learned observationally through modeling: from observing others one forms an idea of how new behaviors are performed, and on later occasions this coded information serves as a guide for action.” [18, p.22] In other words, Bandura [17, 18] purported that the observer acquires an organized knowledge base, called a schema, of the modeled behavior. Later, the observer is able to drawn on the schema to execute novel behavior. With that said, it is essential to differentiate between acquisition and spontaneous performance of the observed behaviors [19].

How does modeling work? Modeling is comprised of salient behaviors to be learned, models exhibiting how to effectively execute these behaviors, opportunities for practice, feedback, and social reinforcement [18]. Specifically, Bandura [19] explains that modeling encompasses “various subsystems” [p.221]. These subsystems are: (1) attentional processes, (2) retention processes, (3) motoric reproduction processes, and (4) incentive or motivational processes.

  1. 1.

    Attentional processes explains that it is unlikely that an individual could replicate the modeled behavior, or responses, if the individual does not, “attend to, recognize, and differentiate the distinctive features of the model’s responses.” [19, p.222]. Serious games designers must create an environment that enables the observer, or trainees, to replicate the target behavior. The observer must be able to easily identify the model and view the demonstration of the target behavior [17, 18]. Superfluous stimuli should be kept to a minimum because it may flood the learners’ attention processes, thereby misdirecting resources. In other words, the trainee is not able to devote attention to the reference stimuli being targeted for learning.

  2. 2.

    Retention processes concerns the “…long-term retention of coded modeling events.” [19, p.222]. During exposure to the stimulus, the observer recodes, classifies, and reorganizes the observed elements of the target behavior into familiar and relevant representations. The translation into relevant representations allows the observer to create schemata that are more easily remembered. This is accomplished despite the lack of direct interaction with the target behavior. In other words, the observer has created a symbolic representation [19]. For these processes to occur, serious games designers must create powerful environments that enable the observer to attend to and reproduce thru covert rehearsal [19].

  3. 3.

    Motoric reproduction processes utilizes the symbolic representation that the observer has created to guide overt performance [19]. The motoric reproduction process takes the form of rehearsal. Whether mental or physical, rehearsal is critical for retention and improved performance; however, learners may be unable to reproduce observed behaviors. The capability of the observer to reproduce the symbolic representations of the modeled behavior serves as a guide to appropriate action. Serious games designers need to provide an environment that enables the observer to rehearse the target behavior because symbolic representation alone will not result in success [19]. Thus, it is the combination of the quality of the cognitive representation and the environment that dictates whether the learner is able to reproduce the modeled behavior [20].

  4. 4.

    Motivational or incentive processes determine whether the learner will willingly exhibit the modeled behavior [21, 22]. Incentive or motivational processes can affect learning, retention, and performance. The observer creates perceptions of either positive or negative outcomes regarding the implementation of the observed behavior. Depending on the observer’s conclusions regarding the outcome, the probability of the individual performing the behavior will either increase or decrease. Serious games designers need to create environments that foster motivation and self-efficacy. Bandura [22] illustrated that when the individual perceives the outcome of their behavior to produce positive incentives the observational learning promptly emerges in action.

2.2 Measuring Pedagogical Usability

The goal of a pedagogical usability metric is to evaluate the extent to which the system affords the player the opportunity to successfully complete each step of modeling. As such, each of the four subsystems of modeling as supported by theoretical and empirical work should be considered: (1) attentional processes, (2) retention processes, (3) motoric reproduction, and (4) motivational processes. Items should reflect the extent to which the model in the game was implemented into the game to accurately complete each stage of modeling.

3 Discussion

Usability testing is critical to the development of efficacious serious games. With the addition of measures that reflect underlying pedagogy, we purport that it is possible to improve the serious game design process, thereby increasing the efficacy of the learning tool. Serious games are unique in that they implement pedagogy to drive the learning process. Thus, the purpose of the present paper is to propose a theoretical basis for creating a measure of modeling in usability. The aim of the measure should be to identify the extent to which a serious game employs the pillars of modeling. We argue that games should be evaluated for the extent to which trainees have an understandable interaction with the system. More specifically, we suggest that games that utilize modeling must incorporate the four subsystems described by Bandura.

Our contribution is two-fold. First, we provide a novel notion of usability, which goes beyond traditional system usability by incorporating an evaluation of pedagogical usability. Secondly, we provide a theoretical basis for a measure that can be used at multiple points throughout the development process. The use of a measure that allows for multipoint assessment has the potential to save time and money, which is important considering the design process, is often costly. Moreover, we purport that this measure will allow designers and developers to pinpoint why a particular function does not successfully facilitate the learning process.

It is our hope to provide the serious games industry with a novel, useful, and practical measure of usability that can be utilized throughout the design process to increase the quality and efficacy of their product. By incorporating usability testing of the pedagogical approach underlying the training, the industry can ensure the production of well-designed games that have the potential to produce the anticipated learning outcomes.