Designing for STEM Faculty: The Use of Personas for Evaluating and Improving Design
- 1.3k Downloads
We demonstrate in a case study how we used a qualitative method for user modeling, persona, to evaluate and refine the design of an interactive visual analytics tool. We explain the literature and our methods for building the persona, and its used to (a) evaluate existing design decisions; (b) make new design decisions in order to serve a specific user group, faculty in STEM. We present the results of 24 in-depth, qualitative interviews with STEM faculty. The interviews addressed topics such as daily work, sources of work satisfaction and success, work goals, activities, needs, and difficulties. The results provide an insight into the busy lives of STEM academics and can be useful to other efforts that aim to design for this user group. We discuss how we used these results, presented in the form of a persona, to evaluate existing design decisions and to create new features that would serve this audience.
KeywordsPersonas Scenarios Use cases User experience
User modeling is an important step in user-centered system design. User modeling provides an understanding of users and their behavior patterns that helps design or refine systems and system behavior . Various disciplines of research and practice use different methods for user modeling, but at their core, user models represent synthesized information about user groups. In this paper, we present a case study of modeling users based on qualitative data collected through in-depth interviews. We illustrate the use of this modeling technique known as “persona”  with a case study centered around the development of an online interactive platform for knowledge mining and visualization, Deep Insights Anytime, Anywhere (DIA2).
DIA2 is an online platform for mining and visualizing the funding portfolio of the U.S. National Science Foundation (NSF) . NSF is one of the major federal funding agencies in the U.S., with an annual budget of approximately 7 billion USD . The funding rate for the NSF has been about 24 % in the fiscal years 2014 and 2015 . The NSF funds more than 10,000 awards each year . The DIA2 platform was created in response to a competitive call for proposals from an NSF program known at the time as Transforming Undergraduate Education in STEM (TUES) and is funded by the NSF . The TUES program needed a way to understand, visualize and mine their own funding portfolio. Beginning with this need, our interdisciplinary team of researchers and designers created an online platform that would address this problem institution-wide, not only for the TUES program, even though that was our starting point.
DIA2 enables users to interact with data visualizations created from funding data provided by the NSF. In the process of doing so, users can explore, understand, and compare NSF’s funding portfolio, as well as identify researchers, NSF programs and program officers by topic, institution, and so on. A detailed description of the system is available in . DIA2 is accessible to the public at www.dia2.org.
DIA2 serves two main user groups: internal NSF staff and external NSF audiences. The first development phase focused on the internal NSF audience. User research and modeling were conducted  and an alpha version of the system was developed and evaluated [15, 21, 24]. Once a satisfactory alpha version was launched, the design team proceeded to the second user group, external to the NSF . While there are multiple groups external to the NSF such as journalists, congress and state representatives, etc., we prioritized faculty members who are looking to apply for funding from the NSF. Of these, we focused particularly on STEM faculty, who are the main audience for the TUES program that had funded this project. In this paper, we present our use of the user modeling technique known as persona to understand this user group, and we demonstrate the utility of personas for evaluating existing products and deriving new design requirements.
2 Related Research
2.1 Cooper’s Persona: User Research Analysis and Presentation Technique
Personas are an effective technique to understand users that was first introduced in the book The inmates are running the asylum by Alan Cooper . In addition to understanding users, personas are a powerful tool for communication among members of the development team and externally with clients. Cooper developed the persona technique from his experience in the software industry using ethnographic user research to support his goal-directed design methodology [4, 11].
The main idea of persona technique is to conduct in-depth user research to understand users’ motivations and behavior patterns, and then create a representation of a composite user to represent the characteristics, goals, and behaviors of real users. Cooper  described the construction of persona in seven steps. The first step is to define observable aspects of focus - such as activities, attitudes, and skills. Second, identify the range of each aspect, and where to place each research participant in the range. Third, identify behavior patterns for each cluster of participants. Fourth, perform pattern analysis to reveal characteristics and goals for each cluster. Fifth, refine the analysis to ensure the correctness and the uniqueness of each persona. Sixth, collect more data and create a behavioral narrative for each persona. Seventh, compare and prioritize the personas.
2.2 Arguments for and Against the Utility of Personas
Research on the usage of personas has discussed issues such as personas’ accuracy of reflecting user data , the engagement of the development team [5, 16], and the appropriate settings for employing personas . While personas are a helpful communication tool, [11, 17, 23], some questions have been raised about the impact of the technique on the end design result [4, 7, 23]. Design practitioners could be mislead and distracted by the abstraction of personas, especially if the personas are not based on in-depth research with users . Blomkvist  argued that the persona technique in goal-directed design puts too much effort on the persona creation instead of the real design. Research has found, however, that, when used correctly, personas can be a very effective design tool, but superficial persona approximations that are not grounded in sufficient data merely support communication among designers about the over simplified users without the real consideration of user’s personalizing details .
2.3 Persona Usage in Industry and Research
A lot of research has supported the idea that personas are an efficient method for developing engagement and enhancing reality when refining techniques. Grudin and Pruitt  discussed that personas are a critical method to enhance the efficiency of using scenarios. They pointed out that persona is a medium for communication that helps designers focus on the most crucial aspects throughout the design process. Later research that reflected on the use of personas in the design process also found positive effects on communication and design process [13, 23, 30].
Personas are widely used in various areas of software design such as news website, business interactive services, and educational software [2, 11, 12, 19, 20]. However, in some cases, the modification of the original persona technique is required . Putnam et al.  pointed out the use of personas to understand users in different cultural contexts with additional modification of the technique.
Dantin  introduced an alternative use of personas as a tool for evaluation. Dantin applied Nielsen’s heuristics  to the use of persona in order to evaluate the UI of educational applications. Dantin’s work indicated the possibility of the use of personas to improve existing design. In this paper, we also demonstrate how personas can be used for evaluating an existing product.
Although personas are commonly used in practice, Idoughi  reflected that there are too few implementation guidelines for design practitioners can follow. Previous studies also showed misunderstanding among practitioners about the concept of personas, which rendered the technique ineffective . These conclusions point to the need for elucidating persona building and usage, a need we aim to address with this case study. The remainder of the paper explains the methods we used to collect in-depth qualitative data from a target user group, how we analyzed the data and summarized it into a persona, and how we used the persona to evaluate the existing product version and derive specific design requirements for the subsequent version.
Interviews were conducted with the participants in one of the three ways: in person at the ASEE conference, in person on the researchers’ campus, and via Skype. On average, interviews were about 60 min long.
During the interview, the participants were first asked some general questions related to their background, work description, work context, and goals they want to achieve. After finishing gathering general information, the interview started to focus on questions regarding impact and diffusion of educational innovations. The questions were mainly targeted towards understanding their definition of “transformative ideas”, how and why they choose certain resources for course preparation, how they stay informed and connected with other professionals, their experience with NSF, and their process of submitting a grant proposal. Afterwards, we described what information DIA2 can provide and asked participants about their attitude towards DIA2. We also asked questions regarding the kind of information they would like to see in DIA2.
3.3 Data Analysis
According to Cooper , a persona is a user model derived from motivations and behaviors of real users. The persona encapsulates the synthesized user data into behavior patterns and presents them in the form of a composite character that designers can relate to. Personas support software design by allowing deep comprehension of the users during the design process  as well as assisting communications about the design among developers . The persona technique enables designers to empathize with users to a larger extent than nameless, faceless data reports would .
In the previous phase of development, the researchers developed personas to represent user groups internal to the NSF . An alpha version of DIA2 had already been launched and tested [15, 21, 24] based on design requirements extracted from the NSF personas. The aim of this phase of research was to further the design of DIA2 to serve the group of STEM academics - a group of target users external to the NSF . The persona revealed specific behavior and motivation patterns of STEM academics, including their goals, needs, and challenges. Then, according to the persona, the researchers developed user scenarios, use cases, and specific design requirements. We present the persona, followed by the design insights derived from this particular persona, in order to illustrate the value of personas in design.
4.1 Persona: Michael Anderson
Personas are a product of a design philosophy known as goal-directed design . Goal-directed design is characterized by a focus on helping users achieve broad life and career goals, rather than automating small tasks. This is why the persona must include an understanding of users’ goals. In our report, the section on goals was followed by a description of activity and behavior patterns, which lead into a description of the persona’s needs and challenges. We present a summary of these insights in the following sections.
Two major goals emerged from the interview data: impact and quality work. Michael Anderson’s primary goal is to make an impact. He divides this goal into 2 parts: Making an impact on students, and making an impact in his research field. A secondary and related goal for Michael is to produce quality work. Michael assesses the quality of his work by whether it is published and whether it results in research funding.
Needs and Challenges.
Three main specific needs emerged from the research interviews: finding resources for teaching and research, identifying research collaborators, and securing research funding. For each one of these needs, the persona report includes a description of how Michael Anderson currently goes about accomplishing these tasks, and challenges he encounters while doing so. For example, when identifying research collaborators, Michael’s preference is to look within his own network and work with people he knows he can trust. If that is not possible, Michael looks for collaborators at the same institution or in close geographical proximity. He might ask colleagues for referrals or appeal to an administrator, such as a college dean, to help him identify people with the required expertise. Major difficulties emerged when discussing the identification of research collaborators. These were related to the fact that it is difficult to identify collaborators in different academic communities, as these communities tend to be fragmented into specific journals and conferences that Michael might not even know about. This information was very useful to the design team, because it pointed out to the importance of helping Michael reach across academic communities, but also enabling him to be more efficient at doing what he already does - looking for collaborators within his institution.
4.2 Scenarios and Use Cases
Scenarios are a design technique that uses storytelling to envision the use of a future system . In goal-directed design, scenarios describe how the persona would use a future system to accomplish tasks related to their larger goals . The designers use scenarios to understand how users would make use of the product in their natural context and to bridge the research-design gap by beginning to envision specific system features. The common use of the scenario is to guide the design before the creation of a prototype, and an evaluation such as cognitive walkthrough from the user’s perspective .
The DIA2 design team created user scenarios - stories showing how DIA2 can help Michael Anderson fulfill various tasks. The design team generated two main scenarios. The first scenario was the story of Michael Anderson using DIA2 to help target his funding request to NSF by identifying NSF programs and program officers who have funded research on his topic in the past. The second scenario was his use of DIA2 to help him identify collaborators. After that, DIA2 designer developed use cases, which were detailed step-by-step interactions in DIA2 systems that Michael Anderson might use the to accomplish each scenario. By building both scenarios and use cases, we developed a deeper understanding of how Michael Anderson would use DIA2 to achieve his goals. This process helped the design team better understand the user group, evaluate whether the persona would be able to accomplish these tasks on the existing version of DIA2, and identify new features that needed to be built in order to help Michael Anderson. In the following sections, we describe two user scenarios, use cases, and cognitive walkthroughs in detail for the purpose of presenting how we adapted developed persona into identifying design requirements.
Scenario 1: Research funding – identifying NSF programs and program officers. Michael has an idea for an NSF proposal, but he is not sure what program to submit it to, and what program officer to discuss his idea with. Michael launches DIA2 and begins a topic search in order to identify what NSF programs and officers have supported research on topics similar to his. The topic search helps him to quickly identify relevant program officers, programs, and even other researchers who have been funded in the same area. He is pleased to see that he can read abstracts of funded proposals in his area, but would like to have access to full proposal text, the publications that came out of those awards, and the involved researchers’ and program officers’ contact information.
Scenario 2: Identifying collaborators. Michael has an idea for an NSF proposal, but needs a collaborator to help him with a specific aspect of the grant – for example, social network analysis. He launches DIA2 and conducts a topic search on social network analysis. The first screen of the resulting search shows the principal investigators (PIs) and co-PIs who have worked on research related to this topic. He runs the cursor over the dots in the social network visualization to identify some of the names. He then looks at the table on the right-hand side and clicks on several names one by one in order to understand more about the work each person does by reading abstracts of the awards they have received. He would like, however, to see the full award text and resulting publications. He finally identifies a few researchers he would like to contact. He would like to have an easy way to get the people’s contact information directly from DIA2.
Based on the scenarios, we created click-by-click stories describing the specific interactions Michael would have with the system in order to accomplish the specified tasks. These detailed descriptions were translated into sketches and wireframes for new system features.
We used the scenarios above for two purposes: to evaluate the current system and understand whether Michael would be able to accomplish the tasks given the current system features; and to derive design requirements for new features we would need to add in order to ensure that DIA2 serves the STEM faculty user group as well as the NSF staff user group. The procedures we used to accomplish these two goals follow the evaluation technique known as cognitive walkthrough, described next.
The cognitive walkthrough is an inspection method for evaluating interfaces . The cognitive walkthrough does not require user input. It is conducted by the design team, who “walks” through the interface as if they were the user attempting to accomplish a specific task. As every step (click), the design team asks a series of questions meant to ensure that the user would know what to do, that there are clear affordances to communicate to the user what to do, and that the user would get some confirmation that they are on the right path to accomplishing the task. The DIA2 team used several of the scenarios developed for Michael Anderson to “walk through” the system. As a result, we learned that the existing features would serve Michael well and help him accomplish the most important tasks. In addition to validating the existing version of the system, the results helped the team identify specific design requirements for features that would improve the system for the STEM faculty user group.
4.3 Design Requirements
Improve the social network visualization to make it easier for Michael to understand what it represents and how to use it to identify collaborators.
Include the ability to contact individuals from DIA2 and send requests for the full text of proposals.
Speak the user’s language (use fewer acronyms and avoid NSF jargon).
- DIA2 should provide recommendations for individuals based on commonalities
Commonalities should be transparent (i.e. “X was suggested based on Y”)
- Degrees of connection might be considered as a criterion for recommending individuals, such as:
People you know or have worked with
Research you’ve done or are doing
What neighborhoods, programs, or communities should (or do) users belong to?
Creation of individual profiles that allow customization of research interests so DIA2 can show research news related to those interests
Create a marketplace to enable researchers to identify collaborators (i.e. “Looking for collaborators with skills in X, Y, and Z”, as well as other postings)
In short, the persona helped the DIA2 team validate the existing design, as Michael Anderson would be able to accomplish the most important tasks on the existing version of the system. At the same time, we used the persona to help generate ideas for new features that would help Michael even more than the existing system version. The team proceeded to prioritize work and resources on the project in order to maximize benefit for both user groups.
In this paper, we illustrated the persona method from the field of human-computer interaction for modeling target users using qualitative research. We showed how we created a persona for a specific online tool, DIA2, based on qualitative data collected from 24 in-depth interviews. We presented the persona and explained the parts that went into our persona report, then we explained how the persona was useful to this project for both evaluation and further product refinement. While personas are used often in the field of human-computer interaction, it remains to be seen how the technique can be applied in different design and engineering fields. Future research can explore the use of personas, their advantages and disadvantages in other fields, as well as propose hybrid methods that can combine the depth of qualitative research with the breadth afforded by large amounts of quantitative data.
DIA2 research and design was supported by NSF award 1123108. The authors would like to thank the graduate students who were instrumental in the collection, analysis and reporting of data supporting this paper: Zhihua (Emma) Dong, Kanrawi Kitkhachonkunlaphat, Joshua Sarver, Rachel Whitson, and Liang Zhao.
- 2.Adlin, T., Pruitt, J., Goodwin, K., Hynes, C., McGrane, K., Rosenstein, A., Muller, M.J.: Putting personas to work. Paper presented at the CHI 2006 Extended Abstracts on Human Factors in Computing Systems (2006)Google Scholar
- 3.Beyer, H., Holtzblatt, K.: Contextual Design: Defining Customer-Centered Systems, p. 154. Elsevier, Amsterdam (1997)Google Scholar
- 4.Blomkvist, S.: The User as a Personality: A Reflection in the Theoretical and Practical Use of Personas in HCI Design (Technical report). Uppsala University, Uppsala (2007)Google Scholar
- 5.Blomquist, Å., Arvola, M.: Personas in action: ethnography in an interaction design team. In: Proceedings of the Second Nordic Conference on Human-Computer Interaction, pp. 197–200. ACM, October 2002Google Scholar
- 7.Chang, Y., Lim, Y., Stolterman, E.: Personas: from theory to practices. In: Proceedings of the 5th Nordic Conference on Human-Computer Interaction Building Bridges - NordiCHI 2008, p. 439 (2008). doi: 10.1145/1463160.1463214
- 8.Chapman, C.N., Milham, R.P.: The personas’ new clothes: methodological and practical arguments against a popular method. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 50, no. 5, pp. 634–636. SAGE Publications, October 2006Google Scholar
- 9.Cooper, A.: The Inmates Are Running the Asylum: [Why High-Tech Products Drive us Crazy and How to Restore the Sanity], vol. 261. Sams, Indianapolis (1999)Google Scholar
- 10.Cooper, A., Reimann, R., Cronin, D.: About Face: The Essentials of Interaction Design. Wiley Publishing Inc., Indianapolis (2007)Google Scholar
- 11.Cooper, A.: The Origin of Personas (2008). http://www.cooper.com/journal/2008/05/the_origin_of_personas. Retrieved
- 13.Dotan, A., Maiden, N., Lichtner, V., Germanovich, L.: Designing with only four people in mind? – a case study of using personas to redesign a work-integrated learning support system. In: Gross, T., Gulliksen, J., Kotzé, P., Oestreicher, L., Palanque, P., Prates, R.O., Winckler, M. (eds.) INTERACT 2009. LNCS, vol. 5727, pp. 497–509. Springer, Heidelberg (2009)CrossRefGoogle Scholar
- 16.Grudin, J., Pruitt, J.: Personas, participatory design and product development: an infrastructure for engagement. In: PDC, pp. 144–152, January 2002Google Scholar
- 18.Hartson, R., Pyla, P.S.: The UX Book: Process and Guidelines for Ensuring a Quality User Experience, p. 144. Elsevier, Waltham (2012)Google Scholar
- 20.Aquino, Jr., P.T., Filgueiras, L.V.L.: User modeling with personas. Paper presented at the Proceedings of the 2005 Latin American Conference on Human-Computer Interaction (2005)Google Scholar
- 21.Liu, Q., Vorvoreanu, M., Madhavan, K.P.C., McKenna, A.F.: Designing discovery experience for big data interaction: a case of Web-based knowledge mining and interactive visualization platform. In: Marcus, A. (ed.) DUXU 2013, Part IV. LNCS, vol. 8015, pp. 543–552. Springer, Heidelberg (2013). http://doi.org/10.1007/978-3-642-39253-5_60 Google Scholar
- 23.Matthews, T., Judge, T., Whittaker, S.: How do designers and user experience professionals actually perceive and use personas? Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2012)Google Scholar
- 25.McKenna, A., London, J., Johri, A., Vorvoreanu, M., Madhavan, K.: Developing and advancing a cyberinfrastructure to gain insights into research investments: an organizing research framework. In: Proceedings of the 122nd ASEE Annual Conference and Exposition (2015)Google Scholar
- 26.National Science Foundation, Funding Rate by State and Organization from FY 2014 to 2015 for NSF (2015a). http://dellweb.bfa.nsf.gov/awdfr3/default.asp. Retrieved
- 27.National Science Foundation, NSF Budget Requests to Congress and Annual Appropriations (2015b). http://www.nsf.gov/about/budget/. Retrieved
- 28.Nielsen, J.: 10 Usability Heuristics for User Interface Design, 1 January 1995. https://www.nngroup.com/articles/ten-usability-heuristics/. Retrieved
- 29.Patton, M.Q.: Qualitative Research and Evaluation Methods. Sage, Thousand Oaks (2015)Google Scholar
- 30.Pruitt, J., Grudin, J.: Personas: practice and theory. In: Proceedings of the 2003 Conference on Designing for User Experiences, pp. 1–15. ACM, June 2003Google Scholar
- 31.Putnam, C., Kolko, B., Wood, S.: Communicating about users in ICTD: leveraging HCI personas. In: Proceedings of the Fifth International Conference on Information and Communication Technologies and Development, pp. 338–349. ACM, March 2012Google Scholar
- 32.Rönkkö, K., Hellman, M., Kilander, B., Dittrich, Y.: Personas is not applicable: local remedies interpreted in a wider context. In: Proceedings of the Eighth Conference on Participatory Design: Artful Integration: Interweaving Media, Materials and Practices, vol. 1, pp. 112–120. ACM, July 2004Google Scholar
- 33.Spencer, R.: The streamlined cognitive walkthrough method, working around social constraints encountered in a software development company. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013), pp. 353–359 (2000). http://doi.org/10.1145/332040.332456
- 34.Vorvoreanu, M., Dong, Z., McKenna, A., Madhavan, K.: Dealing with data deluge at national funding agencies: an investigation of user needs for understanding and managing research investment. In: Proceedings of CHI 2013 Conference on Human Factors in Computing Systems (2013)Google Scholar
- 35.Vorvoreanu, M., McKenna, A., Dong, Z., Madhavan, K.: Dealing with data deluge at national funding agencies: an investigation of user needs for understanding and managing research investments. In: Yamamoto, S., Oliveira, N.P. (eds.) HIMI 2015. LNCS, vol. 9173, pp. 140–151. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-20618-9_14 CrossRefGoogle Scholar