Keywords

1 Introduction

User modeling is an important step in user-centered system design. User modeling provides an understanding of users and their behavior patterns that helps design or refine systems and system behavior [14]. Various disciplines of research and practice use different methods for user modeling, but at their core, user models represent synthesized information about user groups. In this paper, we present a case study of modeling users based on qualitative data collected through in-depth interviews. We illustrate the use of this modeling technique known as “persona” [10] with a case study centered around the development of an online interactive platform for knowledge mining and visualization, Deep Insights Anytime, Anywhere (DIA2).

DIA2 is an online platform for mining and visualizing the funding portfolio of the U.S. National Science Foundation (NSF) [22]. NSF is one of the major federal funding agencies in the U.S., with an annual budget of approximately 7 billion USD [27]. The funding rate for the NSF has been about 24 % in the fiscal years 2014 and 2015 [26]. The NSF funds more than 10,000 awards each year [26]. The DIA2 platform was created in response to a competitive call for proposals from an NSF program known at the time as Transforming Undergraduate Education in STEM (TUES) and is funded by the NSF [25]. The TUES program needed a way to understand, visualize and mine their own funding portfolio. Beginning with this need, our interdisciplinary team of researchers and designers created an online platform that would address this problem institution-wide, not only for the TUES program, even though that was our starting point.

DIA2 enables users to interact with data visualizations created from funding data provided by the NSF. In the process of doing so, users can explore, understand, and compare NSF’s funding portfolio, as well as identify researchers, NSF programs and program officers by topic, institution, and so on. A detailed description of the system is available in [15]. DIA2 is accessible to the public at www.dia2.org.

DIA2 serves two main user groups: internal NSF staff and external NSF audiences. The first development phase focused on the internal NSF audience. User research and modeling were conducted [34] and an alpha version of the system was developed and evaluated [15, 21, 24]. Once a satisfactory alpha version was launched, the design team proceeded to the second user group, external to the NSF [25]. While there are multiple groups external to the NSF such as journalists, congress and state representatives, etc., we prioritized faculty members who are looking to apply for funding from the NSF. Of these, we focused particularly on STEM faculty, who are the main audience for the TUES program that had funded this project. In this paper, we present our use of the user modeling technique known as persona to understand this user group, and we demonstrate the utility of personas for evaluating existing products and deriving new design requirements.

2 Related Research

2.1 Cooper’s Persona: User Research Analysis and Presentation Technique

Personas are an effective technique to understand users that was first introduced in the book The inmates are running the asylum by Alan Cooper [9]. In addition to understanding users, personas are a powerful tool for communication among members of the development team and externally with clients. Cooper developed the persona technique from his experience in the software industry using ethnographic user research to support his goal-directed design methodology [4, 11].

The main idea of persona technique is to conduct in-depth user research to understand users’ motivations and behavior patterns, and then create a representation of a composite user to represent the characteristics, goals, and behaviors of real users. Cooper [10] described the construction of persona in seven steps. The first step is to define observable aspects of focus - such as activities, attitudes, and skills. Second, identify the range of each aspect, and where to place each research participant in the range. Third, identify behavior patterns for each cluster of participants. Fourth, perform pattern analysis to reveal characteristics and goals for each cluster. Fifth, refine the analysis to ensure the correctness and the uniqueness of each persona. Sixth, collect more data and create a behavioral narrative for each persona. Seventh, compare and prioritize the personas.

2.2 Arguments for and Against the Utility of Personas

Research on the usage of personas has discussed issues such as personas’ accuracy of reflecting user data [8], the engagement of the development team [5, 16], and the appropriate settings for employing personas [32]. While personas are a helpful communication tool, [11, 17, 23], some questions have been raised about the impact of the technique on the end design result [4, 7, 23]. Design practitioners could be mislead and distracted by the abstraction of personas, especially if the personas are not based on in-depth research with users [23]. Blomkvist [4] argued that the persona technique in goal-directed design puts too much effort on the persona creation instead of the real design. Research has found, however, that, when used correctly, personas can be a very effective design tool, but superficial persona approximations that are not grounded in sufficient data merely support communication among designers about the over simplified users without the real consideration of user’s personalizing details [23].

2.3 Persona Usage in Industry and Research

A lot of research has supported the idea that personas are an efficient method for developing engagement and enhancing reality when refining techniques. Grudin and Pruitt [16] discussed that personas are a critical method to enhance the efficiency of using scenarios. They pointed out that persona is a medium for communication that helps designers focus on the most crucial aspects throughout the design process. Later research that reflected on the use of personas in the design process also found positive effects on communication and design process [13, 23, 30].

Personas are widely used in various areas of software design such as news website, business interactive services, and educational software [2, 11, 12, 19, 20]. However, in some cases, the modification of the original persona technique is required [1]. Putnam et al. [31] pointed out the use of personas to understand users in different cultural contexts with additional modification of the technique.

Dantin [12] introduced an alternative use of personas as a tool for evaluation. Dantin applied Nielsen’s heuristics [28] to the use of persona in order to evaluate the UI of educational applications. Dantin’s work indicated the possibility of the use of personas to improve existing design. In this paper, we also demonstrate how personas can be used for evaluating an existing product.

Although personas are commonly used in practice, Idoughi [19] reflected that there are too few implementation guidelines for design practitioners can follow. Previous studies also showed misunderstanding among practitioners about the concept of personas, which rendered the technique ineffective [23]. These conclusions point to the need for elucidating persona building and usage, a need we aim to address with this case study. The remainder of the paper explains the methods we used to collect in-depth qualitative data from a target user group, how we analyzed the data and summarized it into a persona, and how we used the persona to evaluate the existing product version and derive specific design requirements for the subsequent version.

3 Methodology

3.1 Participants

We selected a criterion sample [29] of interview participants who were part of the target audience for NSF, specifically, the TUES program. Therefore, all our participants had to be STEM faculty with an interest in education. We recruited participants at the annual convention of the American Society for Engineering Education, which attracts the particular audience we were targeting. Also, we recruited participants from publicly available lists of researchers who had received awards through the NSF TUES program. We conducted a total of 24 interviews with faculty members. Seven out of 24 participants were females. The participants’ age distribution is shown in Fig. 1. Of the participants who revealed highest degree information, 17 had a Ph.D., and one had a Master’s degree. Participants held a variety of academic ranks ranging from assistant professor (5), associate (4), full (7) and distinguished (1). Four participants reported holding administrative positions such as dean. Most participants (9) worked in engineering disciplines, followed by Technology (5) and Engineering Education (4).

Fig. 1.
figure 1

Interview participants’ age distribution

Full size image

3.2 Procedures

Interviews were conducted with the participants in one of the three ways: in person at the ASEE conference, in person on the researchers’ campus, and via Skype. On average, interviews were about 60 min long.

During the interview, the participants were first asked some general questions related to their background, work description, work context, and goals they want to achieve. After finishing gathering general information, the interview started to focus on questions regarding impact and diffusion of educational innovations. The questions were mainly targeted towards understanding their definition of “transformative ideas”, how and why they choose certain resources for course preparation, how they stay informed and connected with other professionals, their experience with NSF, and their process of submitting a grant proposal. Afterwards, we described what information DIA2 can provide and asked participants about their attitude towards DIA2. We also asked questions regarding the kind of information they would like to see in DIA2.

3.3 Data Analysis

Affinity diagramming was used to analyze the data. Affinity diagramming is a technique that enables teams to analyze qualitative data together. It requires organizing and grouping ideas and showing common issues and themes in a visual way [3, 18]. The 24 interviews were transcribed into plain text by an external transcription service. The DIA2 UX and development teams read through the transcripts and created work activity notes by recording each single idea on a sticky note. Then, the team members grouped and synthesized ideas into categories. Major findings were discussed and then summarized in the form of a persona. Even though the interview participants’ demographics varied, their thinking about issues at hand and their behavior patterns were consistent, which led the team to synthesize data from all 24 interviews into one persona. Had the team noticed remarkably different goals, ways of thinking, and behaviors, we would have had to create more than one persona (Fig. 2).

Fig. 2.
figure 2

Affinity diagramming

Full size image

4 Persona

According to Cooper [10], a persona is a user model derived from motivations and behaviors of real users. The persona encapsulates the synthesized user data into behavior patterns and presents them in the form of a composite character that designers can relate to. Personas support software design by allowing deep comprehension of the users during the design process [20] as well as assisting communications about the design among developers [17]. The persona technique enables designers to empathize with users to a larger extent than nameless, faceless data reports would [10].

In the previous phase of development, the researchers developed personas to represent user groups internal to the NSF [35]. An alpha version of DIA2 had already been launched and tested [15, 21, 24] based on design requirements extracted from the NSF personas. The aim of this phase of research was to further the design of DIA2 to serve the group of STEM academics - a group of target users external to the NSF [25]. The persona revealed specific behavior and motivation patterns of STEM academics, including their goals, needs, and challenges. Then, according to the persona, the researchers developed user scenarios, use cases, and specific design requirements. We present the persona, followed by the design insights derived from this particular persona, in order to illustrate the value of personas in design.

4.1 Persona: Michael Anderson

Figure 3 shows the short version, overview of the persona documentation. A persona report includes a brief overview like the one presented here and an in-depth report. The overview sheet can be easily remembered and even posted in visible locations around the designers’ workspace as a continuous reminder of target users and their needs. The persona overview includes a photo, a fictional but realistic full name, and demographics representative of the user group. It also includes a brief description of the persona that focuses on characteristics relevant to the product. A representative quote extracted from the data is also included. This ensures that designers are reminded of the users’ voice. The persona overview also includes brief pointers about the most relevant aspects relevant to the design - in this case, goals, needs, and challenges. The persona overview is accompanied by an in-depth report that provides further details about the persona’s goals, needs, motivations, behaviors, and challenges. The persona overview serves as a reminder and executive summary of the longer report.

Fig. 3.
figure 3

Persona overview

Full size image

Personas are a product of a design philosophy known as goal-directed design [10]. Goal-directed design is characterized by a focus on helping users achieve broad life and career goals, rather than automating small tasks. This is why the persona must include an understanding of users’ goals. In our report, the section on goals was followed by a description of activity and behavior patterns, which lead into a description of the persona’s needs and challenges. We present a summary of these insights in the following sections.

Goals.

Two major goals emerged from the interview data: impact and quality work. Michael Anderson’s primary goal is to make an impact. He divides this goal into 2 parts: Making an impact on students, and making an impact in his research field. A secondary and related goal for Michael is to produce quality work. Michael assesses the quality of his work by whether it is published and whether it results in research funding.

Activities.

Related to the notion of goals is the issue of what makes Michael feel successful. The top two activities that make him feel successful are research and teaching, followed by meeting and email at the very bottom. However, the activities that take Michael’s time are in precisely reverse order, as shown in Fig. 4.

Fig. 4.
figure 4

Inverse relationship between activities that define success and time spent

Full size image

Needs and Challenges.

Three main specific needs emerged from the research interviews: finding resources for teaching and research, identifying research collaborators, and securing research funding. For each one of these needs, the persona report includes a description of how Michael Anderson currently goes about accomplishing these tasks, and challenges he encounters while doing so. For example, when identifying research collaborators, Michael’s preference is to look within his own network and work with people he knows he can trust. If that is not possible, Michael looks for collaborators at the same institution or in close geographical proximity. He might ask colleagues for referrals or appeal to an administrator, such as a college dean, to help him identify people with the required expertise. Major difficulties emerged when discussing the identification of research collaborators. These were related to the fact that it is difficult to identify collaborators in different academic communities, as these communities tend to be fragmented into specific journals and conferences that Michael might not even know about. This information was very useful to the design team, because it pointed out to the importance of helping Michael reach across academic communities, but also enabling him to be more efficient at doing what he already does - looking for collaborators within his institution.

4.2 Scenarios and Use Cases

Scenarios are a design technique that uses storytelling to envision the use of a future system [6]. In goal-directed design, scenarios describe how the persona would use a future system to accomplish tasks related to their larger goals [10]. The designers use scenarios to understand how users would make use of the product in their natural context and to bridge the research-design gap by beginning to envision specific system features. The common use of the scenario is to guide the design before the creation of a prototype, and an evaluation such as cognitive walkthrough from the user’s perspective [23].

The DIA2 design team created user scenarios - stories showing how DIA2 can help Michael Anderson fulfill various tasks. The design team generated two main scenarios. The first scenario was the story of Michael Anderson using DIA2 to help target his funding request to NSF by identifying NSF programs and program officers who have funded research on his topic in the past. The second scenario was his use of DIA2 to help him identify collaborators. After that, DIA2 designer developed use cases, which were detailed step-by-step interactions in DIA2 systems that Michael Anderson might use the to accomplish each scenario. By building both scenarios and use cases, we developed a deeper understanding of how Michael Anderson would use DIA2 to achieve his goals. This process helped the design team better understand the user group, evaluate whether the persona would be able to accomplish these tasks on the existing version of DIA2, and identify new features that needed to be built in order to help Michael Anderson. In the following sections, we describe two user scenarios, use cases, and cognitive walkthroughs in detail for the purpose of presenting how we adapted developed persona into identifying design requirements.

User Scenarios.

Scenario 1: Research funding – identifying NSF programs and program officers. Michael has an idea for an NSF proposal, but he is not sure what program to submit it to, and what program officer to discuss his idea with. Michael launches DIA2 and begins a topic search in order to identify what NSF programs and officers have supported research on topics similar to his. The topic search helps him to quickly identify relevant program officers, programs, and even other researchers who have been funded in the same area. He is pleased to see that he can read abstracts of funded proposals in his area, but would like to have access to full proposal text, the publications that came out of those awards, and the involved researchers’ and program officers’ contact information.

Scenario 2: Identifying collaborators. Michael has an idea for an NSF proposal, but needs a collaborator to help him with a specific aspect of the grant – for example, social network analysis. He launches DIA2 and conducts a topic search on social network analysis. The first screen of the resulting search shows the principal investigators (PIs) and co-PIs who have worked on research related to this topic. He runs the cursor over the dots in the social network visualization to identify some of the names. He then looks at the table on the right-hand side and clicks on several names one by one in order to understand more about the work each person does by reading abstracts of the awards they have received. He would like, however, to see the full award text and resulting publications. He finally identifies a few researchers he would like to contact. He would like to have an easy way to get the people’s contact information directly from DIA2.

Use Cases.

Based on the scenarios, we created click-by-click stories describing the specific interactions Michael would have with the system in order to accomplish the specified tasks. These detailed descriptions were translated into sketches and wireframes for new system features.

We used the scenarios above for two purposes: to evaluate the current system and understand whether Michael would be able to accomplish the tasks given the current system features; and to derive design requirements for new features we would need to add in order to ensure that DIA2 serves the STEM faculty user group as well as the NSF staff user group. The procedures we used to accomplish these two goals follow the evaluation technique known as cognitive walkthrough, described next.

Cognitive Walkthrough.

The cognitive walkthrough is an inspection method for evaluating interfaces [32]. The cognitive walkthrough does not require user input. It is conducted by the design team, who “walks” through the interface as if they were the user attempting to accomplish a specific task. As every step (click), the design team asks a series of questions meant to ensure that the user would know what to do, that there are clear affordances to communicate to the user what to do, and that the user would get some confirmation that they are on the right path to accomplishing the task. The DIA2 team used several of the scenarios developed for Michael Anderson to “walk through” the system. As a result, we learned that the existing features would serve Michael well and help him accomplish the most important tasks. In addition to validating the existing version of the system, the results helped the team identify specific design requirements for features that would improve the system for the STEM faculty user group.

4.3 Design Requirements

We identified six main design requirements for improving DIA2 in order to better serve the STEM researcher user group.

  • Improve the social network visualization to make it easier for Michael to understand what it represents and how to use it to identify collaborators.

  • Include the ability to contact individuals from DIA2 and send requests for the full text of proposals.

  • Speak the user’s language (use fewer acronyms and avoid NSF jargon).

  • DIA2 should provide recommendations for individuals based on commonalities

    • Commonalities should be transparent (i.e. “X was suggested based on Y”)

    • Degrees of connection might be considered as a criterion for recommending individuals, such as:

      • People you know or have worked with

      • Research you’ve done or are doing

      • What neighborhoods, programs, or communities should (or do) users belong to?

  • Creation of individual profiles that allow customization of research interests so DIA2 can show research news related to those interests

  • Create a marketplace to enable researchers to identify collaborators (i.e. “Looking for collaborators with skills in X, Y, and Z”, as well as other postings)

In short, the persona helped the DIA2 team validate the existing design, as Michael Anderson would be able to accomplish the most important tasks on the existing version of the system. At the same time, we used the persona to help generate ideas for new features that would help Michael even more than the existing system version. The team proceeded to prioritize work and resources on the project in order to maximize benefit for both user groups.

5 Conclusion

In this paper, we illustrated the persona method from the field of human-computer interaction for modeling target users using qualitative research. We showed how we created a persona for a specific online tool, DIA2, based on qualitative data collected from 24 in-depth interviews. We presented the persona and explained the parts that went into our persona report, then we explained how the persona was useful to this project for both evaluation and further product refinement. While personas are used often in the field of human-computer interaction, it remains to be seen how the technique can be applied in different design and engineering fields. Future research can explore the use of personas, their advantages and disadvantages in other fields, as well as propose hybrid methods that can combine the depth of qualitative research with the breadth afforded by large amounts of quantitative data.