Numerous prominent national reports, monographs, and popular books suggest that we are on the verge of a crisis in education (Friedman, 2005; Partnership for 21st Century skills, 2004; Shaffer and Gee, 2005). Inter-related technological, social, and cultural changes are changing the demands on our educational system. Technological advances—particularly ubiquitous broadband Internet access, low cost desktop computers, and portable computing technologies (iPods, cellphones, PDAs)—make information readily accessible, so that with the press of a button anyone with an Internet-enabled cellphone can access libraries of written information, video archives, and personal contacts (Klopfer and Squire, in press; Soloway et al., 2001). While policymakers debate whether students should be allowed to use cellphones, instant messenger, and calculators in school, it is impossible to imagine today’s graduates entering the knowledge economy without understanding how to think with digital tools (spreadsheets, calculators, visualization tools), to access and analyze online databases or to leverage social networks (Gee et al., 1996; Reich, 1991; Solomon, 1993).

Traditional schools and classrooms may be the only place where such information resources are not fully accessible, in part because schools are based on the historic literacies of print and have not adapted to the literacies of multi-modality and interactive technology (Kress, 2003; Lankshear and Knobel, 2001; New London Group, 2000). As Leander and Lovvorn (in press) argue, even “high tech” schools operate accordingly to an underlying cultural logic of “print-based” literacies and pedagogies in which teachers determine learning goals, sanction resources, and dictate what steps students ought to take to complete assignments (cf. Lemke, 1990). Leander and Lovvorn contrast this constellation of school-based literacy practices with kids’ practices on the Internet outside of school, where kids interact with multi-modal texts, produce as well as consume texts and resources, and, perhaps most critically, embrace epistemologies radically different from those underlying schooling (what is “true” is what works in experience or is the consensus of a community, as opposed to appeals to authority—what the teacher dictates as correct). Ironically perhaps, the modes of learning occurring in popular culture, particularly in video games, align with the culture of the new capitalism more closely than school-based literacies do, creating an implicit critique of schooling (Gee, 2003; 2004). Can we create pedagogies that capitalize on these new literacies and prepare students for life in the 21st century?

The goal of this design-based research project is to investigate the potential of place-based augmented reality gamingin environmental science with middle school students as a model of instruction suited to the literacy demands of the 21st century (see Rosenbaum, Klopfer, and Perry, this issue). Augmented reality games are games played in the real world with the support of digital devices (PDAs, cellphones) that create a fictional layer on top of the real world context. There are many emerging approaches to augmented reality, which range from (presently) relatively obtrusive eye displays that the user wears as they move through the world to less obtrusive handheld technologies such as employed in this study. The approach described in this paper uses virtual media (text, documents, multimedia), triggered by location (determined via Global Satellite Positioning, GPS), to create a fictional set of events occurring in the real world space. Place-based augmented reality games are played in specific real-world locations (historical, geographical sites) and use handheld computers with global positioning systems to augment users’ experience of space with additional data (text, numerical data, audio, video). Augmented reality games position players as participants in a complex system while drawing on players’ emotional and cognitive relations with the environment to create designed experiences for solving complicated problems exhibiting robust phenomena. (How these augmented reality experiences are games is explained further in the next section.)

The goal of this design-based research project is to help students think like scientists, specifically to see interactions in their environment as interconnecting geochemical processes and to use scientific understandings and scientific argumentation to understand key contemporary issues facing their local environment. This particular paper traces three case studies centered on the use of Mad City Mystery, a murder mystery game set in around Lake Mendota Madison, Wisconsin. Students investigate an untimely death caused by a murder, suicide, or the combination of several interacting toxic chemicals that are commonly found in the region. The research seeks to build on earlier work in augmented reality gaming (Klopfer and Squire, in press) demonstrating the technological potentials of the platform, but extends this prior work by using specific game features to scaffold students’ thinking, more directly tie scientific controversies to their local environments, and tie more closely to curricular goals.

Literature review: game-based learning in science education

Within science education, the necessity of building a literate populace is vitally important, as we prepare students to think critically in an ever changing world of multiple media (Kuhn, 1999; Lemke, 1998). Recent debates around the teaching of evolution, cloning, genetic engineering, stem cell research, and global warming suggest the importance of not just “teaching kids the facts,” but also developing scientific literacy skills so that they may navigate the scientific issues that face society. Indeed, scientific breakthroughs in fields such nanotechnology that did not exist 50 years ago underscore the importance of preparing both for today’s world and for tomorrow’s. Specifically, they need literacy skills to make meaning with multiple representational forms (including printed word, charts, graphs, visualizations, and simulations), robust conceptual understandings with which to think, and, perhaps most crucially, the ability to critique scientific arguments (Kuhn, 1999; Lemke, 1990).

Digital games could be one productive method for developing scientific argumentation skills in a manner aligned with the needs of a 21st century educational system. Analyzing argumentation surrounding the popular massively multiplayer game World of Warcraft and finding that it hit benchmarks for science literacy more closely than has been reported in many science classrooms, Steinkuehler and Chmiel (2006) hypothesize that multiplayer games may be a productive route for engaging students in scientific literacy practices. Analyzing World of Warcraft forums, Steinkuehler and Chmiel show how that a large number of the practices that constitute higher-end game play (e.g., collecting data in the world, creating instruments to collect data, creating spreadsheets to analyze data and create models, arguing over data collection methods, developing assertions based on data) share many of the productive literacy practices associated with the new literacy studies (New London Group, 2000; Spencer Foundation Games, Learning, & Society Group, in press). Participants use multiple modalities to make sense of experience and to produce new meanings of with various media. This paper hypothesizes that hypothesize that game-based learning approaches could help students develop and extend new literacy skills, particularly, forms of scientific argumentation.

Science Education for the 21st Century

As many researchers have lamented (Chinn and Malhotra, 2002; Gee, 2004; Lemke, 1990), a disappointing feature of today’s educational system is that students are not only graduating from school unprepared for this world, but graduating with erroneous beliefs about the discourse of science and identities constructed in opposition to those required by and available in science. In Talking Science, Jay Lemke investigates patterns of social interaction in science classrooms, using discourse analysis techniques to show how the underlying cultural models and norms of science classrooms ultimately serve to mystify science, rather than initiate students in a discourse where they are “doing” and talking science. Within most classrooms, rather than inquiry, the dominating activity system of schooling, with its emphasis on order, control, grades, and hierarchy, characterizes the activities of students and teachers (cf. Squire et al., 2003b).

Even those classrooms that do engage in inquiry typically provide in “simple inquiry tasks” rather than inquiry activities where the outcome is in genuine doubt, a hallmark of authentic inquiry (Chinn and Malhotra, 2002). Chinn and Malhotra examined inquiry-based learning practices in contemporary classrooms, observing that not only do most inquiry-based activities fail to produce authentic scientific inquiry, but they engender inaccurate epistemological beliefs about science. Despite many teachers’ desire to use inquiry based approaches (mostly to make science education activity-driven and “hands on”), the underlying epistemology remains one of “science is seen as the accumulation of facts” and laboratory experiments provide occasions to arrive (however inefficiently) at pre-determined facts. Chinn and Malhotra propose that that “simple inquiry tasks (SIT) assume an epistemology that is opposed to the epistemology of authentic science. As a result, students who learn about scientific reasoning through SIT may actually learn a nonscientific epistemology” (pp. 187). This critique is not to admonish teachers or science educators for using such activities per se (particularly within the contemporary configurations of schooling), but to remind us that at least one major goal of education is to graduate students with an awareness of how science operates, preparing them better to engage in democratic decision making around issues such as global warming, evolution, genetic engineering, and so on).

A challenge for science educators is to develop pedagogical models that engage students in authentic, deep forms of inquiry where they develop scientific thinking skills, without simply reifying power dynamics whereby the teacher is the arbiter of knowledge and science is mystified as “work done by others,” as described by Lemke (1990). In an effort to ground the relatively abstract science education standards (e.g., AAAS, 1993; National Research Council, 1996) in a more coherent framework, Chinn and Malhotra posed a framework of authentic inquiry based on studies of scientists that includes: (1) generating a research question; (2) designing a study to address the research question; (3) making observations to answer those questions (including guarding against perceptual bias); (4) developing methods, tools, and rationale to explain results; (5) developing theories based on and developing evidence; and (6) studying others’ research. Chinn and Malhotra recognize that “actual” inquiry is difficult to carry out in most school settings, given the reality of most schools, which includes the structure of the school day, constrained resources, and of course, students’ limited experience in science. Chinn and Malhotra argue that educators might develop relatively simple inquiry tasks that capture at least one core component of scientific reasoning.

Game-Based Learning in Science: A Post-progressive Approach

Game-based learning approaches have been suggested as one “post progressive” pedagogy that might situate learners in complex thinking tasks that are driven by authentic questions, incorporate multiple tools and resources, rely on learning by doing, guide learners through a path of events and into a way of thinking, and require complex performances to demonstrate mastery (see Barab, Thomas, Dodge, Carteaux, and Tuzum, 2005; Gee, 2004; Shaffer et al., 2005; Squire, 2005a). Although theories of game-based learning are still in their infancy, advocates of game-based approaches have sought to combine socio-cultural approaches to learning with the affordances of contemporary computer and video games, which are comprised of mixtures of open-ended and closed ended problems (cf. Gee, 2003; Lave and Wenger, 1991; Shaffer, 2005; Steinkuehler, 2006b; Squire, 2002, Squire, in press).

Applications of these models suggest that a core feature of educational game play may include: cycles of making choices, experiencing consequences, interpreting the game system, building casual narratives of experience, having multiple experiences within the system, and then building a cognitive model of the game system as a result (Squire, 2005a; Squire, in press). In studies of strategy computer gaming players, for example, Squire and colleagues (2006) found that players make choices about their civilizations in game play (should I build a military or an economy?), construct narrative interpretations of events in game play, negotiate these interpretations within player communities both in game and out of game, and then use these narrative accounts as the basis of future decisions. In analyzing students’ cognition while playing new games, as well as descriptions of historical events, Squire and colleagues describe students as having achieved a “systemic” level understanding—comprehending how multiple variables interact toward producing outcomes.

Shaffer (2005) argues that, while educators have created a number of projects designed to help kids “think like scientists,” as educators have spent comparably less time helping students learn to think like other professions who work in similar domains, including doctors, engineers, and science journalists. If the goal of schooling is not to prepare students for life, but engagement with it (cf. Dewey, 1938), perhaps professional role playing games (frequently called epistemic games) can expand the number of roles available to students in school. Shaffer (2004) suggests that the professions might enculturate students into a way of thinking based on those professions (Shaffer, 2004). Such an approach could avoid trappings of inquiry-based learning approaches built on the traditional school disciplines; whereas basic fields are driven by theoretical questions, applied fields (such as medicine, environmental engineering, or public health) are often driven by practical ones, opening new opportunities for problem solving (e.g., could a swimmer die from ingesting too much mercury?). Such an approach could enable students to engage in productive forms of thinking and problem solving that are frequently engaging to students and socially valued.

Principles of Games Applied to Science Education

Examining existing work on games in science education from a socio-cultural perspective, we see at least five core features pertinent to designing games for learning. First, games ask students to inhabit roles that encourage them to create what James Paul Gee calls projective identities, identities that are a melding between the game player and the role (or scientific profession). All information, goals, experiences and rewards occur within the context of this role, which in science education might be environmental engineers, marine biologists, science journalists, medical doctors, or government officials. Game designers use differentiated roles to encourage (and indeed require) collaboration across groups (similar to the way educators use jigsaw pedagogies) (Brown and Campione, 1996; Games-to-Teach Team, 2003). Character creation, selection, and development serves to reinforce these choices, giving players new and different capacities in the environment. As players ponder which skills (and hence capacities in the world) to improve, they further specialize, developing unique identities tied to particular actions usually mediated by digital tools (Steinkuehler and Chmiel, 2006; Squire and Steinkuehler, in press). In the case of science, this could include more powerful (digital) lab equipment, a larger virtual team, new contacts, or a larger budget. Building on conventions of role playing games (cf. Au, 2001), the idea is that each game mechanic is designed to deepen players’ commitment to their role.

Second, activity is organized around challenges (Malone, 1981). However, games have evolved extensively since early arcade games and employ multiple challenge/reward structures designed to support engagement, collaboration, and learning. These structures include multiple overlapping reward ladders and embedded, evocative, enacted, and emergent narrative devices that challenge players and communicate motivations and goals to them (cf. Bartle, 1996; Church, 2001; Jenkins and Squire, in press; Jenkins and Squire, 2002; Rouse, 2001; Salen and Zimmerman, 2003; Squire, 2006). As Jenkins (2002) argues, video games are the popularized version of interactive narrative and have developed multiple methods for encouraging players to take on (and create) new—even unanticipated goals that form the basis of play (cf. Robison, 2006). Games’ ability to elicit goals from players and their capability to create potential win conditions from which players then choose to take on are core features that might be leveraged for educational benefit (Squire, 2005a, b).

Third, games offer opportunities to tie goals to particular places, particularly, sites of contested spaces. As Jenkins (2002) argues, game designers face the difficult challenge (as do educators) of designing for second-order behaviors; they cannot specifically design players’ experiences, so they spatially arrange game features (game maps, levels, characters, and triggered events)—to create emotional affect. As an art form, they encourage us to experience spaces in new ways (much as romantic painters encouraged us to see landscapes in new ways). Within education, we identify opportunities for locating contested spaces (debates over land use, flows of toxins in the environment, interactions among people and populations through space) that could be the basis of games (Squire et al., 2003a).

Fourth, digital games allow for embedding authentic resources and tools that are used within the context of game play. Literacy scholars have noted that children routinely read and write texts substantially over their grade level in the context of game play (Buckingham, 2003; Gee, 2003; Steinkuehler, 2005). Role playing games such as Deus Ex frequently embed newspaper articles, videos, multi-media documents, encyclopedias, and texts (even books!) in the environment for players to read in order to gain more background information, clues, and richer context for play.Footnote 1 Websites, manuals, and FAQs provide players extra information that is mobilized in game play. Digital tools (spreadsheets, calculators, research labs, gravity guns) also mediate play, allowing players to process information and interact with the environment (including affecting it) in new ways. Situated within game play, such resources might be mobilized as tools to think with in solving future problems.

Finally, recent work on gaming has illuminated the fundamentally social nature of game play, suggesting that frequently the game community, not the game, is a productive unit of analysis for educators (Squire, in press; Steinkuehler, 2006a, b). Games are frequently designed to be a part of a social system that produces certain kinds of social interactions, rather than as media to be consumed by someone in isolation (cf. Crawford, 1982). In the context of multiplayer games, game structures such as roles are created to promote collaboration, competition, and community among players (Bartle, 1996, 2003; Steinkuehler, 2006b). Even single player games such as Civilization get taken up in different ways by different communities, with some using the game as a context for head-to-head competition, and others using the game as a tool for modeling history (Squire, in press). The kinds of practices, thinking, and learning that occurs through such games differs based on social context, and as such it is imperative for educational game designers to consider the design of media as one part of creating an overall learning context. In this study, we leverage these five components to develop a game space with the goal of supporting participants in engaging in scientific inquiry.

Research questions

This design-based research study explores the hypothesis that games designed according to the principles of contemporary computer and video games might be a productive vehicle for supporting students in developing scientific argumentation skills in a manner productive for the 21st century. This study investigates the learning that occurs within game play designed around such a game, and in particular, whether a game designed around such principles can engage students in scientific thinking, specifically hypothesis formation and reasoning from evidence. First, we investigate how game structures scaffold students’ thinking.

  1. (1)

    How do students react to the roles presented, and do they serve as scaffolds for students’ scientific thinking?

  2. (2)

    Do the inquiry-driven activities resemble simple inquiry tasks or authentic inquiry tasks, and what design features seem to promote authentic inquiry?

  3. (3)

    Does participation in a game-based activity system elicit authentic, complex scientific thinking, and how does this differ across groups?

In the context of this study, we pay particular attention to students’ argumentation, asking:

  1. (1)

    What types of discourse emerge from game play, and how do these compare with school and science-based discourses?

  2. (2)

    What forms does their argumentation take, and does students’ argumentation meet those criteria for quality described in educational standards?

  3. (3)

    Do groups with more robust scientific understandings (as suggested by Kuhn) engage in more complex forms of argumentation, lending evidence for validity of the authenticity of this task?

Finally, we investigate how situating a game in their physical environment affects students’ learning:

  1. (1)

    Does playing the game in the physical environment trigger prior knowledge?

  2. (2)

    Does playing in the world (outside the classroom) elicit non-school based aspects of students’ identities?

Because place-based augmented reality games are still in their infancy, this research in many respects serves as a pilot study designed to advance previous work investigating the potential of the technology (cf. Klopfer and Squire, in press), wedding it more closely to game-based pedagogical theory in a domain-specific manner (cf. Cobb et al., 2001; Squire, in press).

Methodology

To investigate these questions, we designed and enacted Mad City Mystery, a place-based augmented reality game with three groups of students in the spring and fall of 2005 (see Rosenbaum, Klopfer, and Perry, this issue for more discussion of augmented reality). These three classes were purposively chosen for convenience and to elicit a wide range of experiences (See Table I). Using design-based research techniques, this study pursues the design of an education curriculum built according to contemporary game-based learning theory, in an effort to both improve the quality of the design while also building more robust theory (Barab and Squire, 2004; Brown, 1992; Collins, 1992). The logic of the inquiry involves one of iterative cycles of design, theory generation, redesign, and theory refinement, with evidence for the validity of assertions residing in the quality of the educational outcomes in situ. The game was designed in accordance with contemporary game theory to encourage players (1) to inhabit professional roles with specific professional identities and perspectives, (2) to challenge players through multiple layers of narratives and tasks, (3) to situate the contested game space in a local/physical place where the meaning of a place is perceived from professional perspectives, (4) to scaffold learning through multimodal representations, and (5) to create social interactions that promote collaboration, competition, and reflection-in-action.

Table I. Participant Summary
Full size table

Augmented Reality Game Design Intervention: Mad City Mystery

Ivan Illyich is dead.

Police claimed that he drowned while fishing by the south shore of Lake Mendota.

Between January and the time of his death, Ivan put on 25 pounds and started drinking heavily. His health condition had deteriorated considerably.

As one of his friends, your task is to investigate the case with two of your best friends. It is your duty to present a clear picture about the causes and effects of these to the public.

Mad city Mystery is an augmented mystery game that takes place on the University of Wisconsin-Madison campus near Lake Mendota. Playing the game takes from 90 to 120 min, depending on group dynamics. Each group studied here played Mad City Mystery for 2–3 h, including (1) briefing, (2) game play, and (3) debriefing.Footnote 2 The game itself begins with the revelation of Ivan’s mysterious death. From there, players must interview virtual characters, gather quantitative data samples, and examine government documents to piece together an explanation. Players work in teams that may or may not compete with other teams, depending on the teacher’s preferences.

The educational goals of the game are to help students develop investigation and inquiry skills, specifically scientific argumentation skills through the virtual investigation. Game play requires them to: (1) observe phenomena in their environment and tie them to underlying scientific processes and phenomena; (2) ask questions about the human and environmental effects of human processes in the environment; (3) engage in scientific argumentation forming hypotheses, refining them based on evidence and discussing and arguing rationale in order to develop theory; and (4) develop conceptual understandings of geochemical water cycles, specifically, how chemicals move through the water system. From a content perspective, the big “idea” is that human activity has an impact on the environment in anticipated and unanticipated ways, which can be understood through a conceptual understanding of relationships among river run-off, chemical cycles, and impacts on human health. We expect that players having finished the game would have: (1) a better holistic view of the steps in conducting an authentic inquiry project, (2) a better understanding of the trade-offs involved in conducting a virtual investigation, (3) deeper (yet admittedly still developing) concepts of environmental systems, and (4) better scientific argumentation skills specifically the development of specialized academic language. (For a list of state environmental standards, to which these goals are mapped, see Appendix A.)

Overview of Problem

The problem—determining the cause of Ivan’s death—is open-ended and involves multiple causal factors. The problem is designed to encourage students to think about interconnecting causal factors, as opposed to single cause problems; in fact, no one of the possible contaminants presented in the game would be likely to kill Ivan. The most probable solution is that Ivan was experiencing deteriorating health from a combination of alcoholism, depression, and exposure to TCE at the workplace. (TCE is a common degreasing agent that we have used in past AR games because of its ubiquity.) Ivan’s exposure to excessive PCBs, mercury, and farm pesticides via the consumption of fish may have led to his general deterioration as well. A key feature of the problem is that no one of these causes would have caused Ivan to suddenly fall over and die while fishing. In combination, however, they may cause Ivan to be significantly weakened so that if he fell in he could drown. As such, the pedagogical goal of the problem is to immerse students in cycles of hypothesis formation, theory generation, evidence gathering and thinking, rather than necessarily happening upon the “correct” answer.

The problem was constructed in this open-ended fashion for several reasons. First, we wanted a problem sufficiently complex to support complex argumentation. One might imagine a different game that was all about finding one cause (indeed we have experimented with such designs around E. Coli in Lake Michigan. Such an approach, while potentially educative, might not generate rich scientific argumentation around multiple hypothesis formation, theory generation, and evidence gathering. Second, we wanted to advance a particular theory of game design that eschews single answer solutions (puzzles) in favor of complex, multiple solution problems (cf. Church, 2001). Although we believe that there will be multiple educational game design structures in the future, this open-ended problem type may be the most robust format for supporting scientific argumentation. Third, this format allowed us to explore interactions among social factors (lifestyle choices, social class) and environmental issues. In Wisconsin, heavy alcohol consumption is a known public concern that can lead to several secondary health issues, cutting across population demographics. Fishing, a popular not just as a sport but as a primary source of food in many poorer Wisconsin communities open questions about how environmental issues interact with social class (e.g. which communities are most affected by pollutants).

Finally, this format allowed us to tie together multiple symptoms in multiple groups, both raising awareness about important local health concerns and deepening the problem. Bartleby, Ivan’s co-worker also was exposed to TCE (but did not die or drown due to a lack of complicating factors). Ivan’s wife and son (Eve and Adam) also consumed large amounts of fish, and Adam in particular exhibited many signs (low birth weight, potential neurological disorders) associated with both PCBs and mercury. This component of the problem (hopefully) heightened emotional engagement while also raising important questions about the safety of feeding one’s family fish from Madison’s lakes (which is a very common practice).

The goal of the problem—to create a complete picture of the causes of Ivan’s death—requires players to weigh the various symptoms, toxins, pollution sources (fish, water, work environment), providing a coherent argument for what could have happened to Ivan. Students were instructed to inform officials of their degree of confidence in their evidence, rationale, and findings. Further, they were instructed to alert officials as to any other important discoveries they make along the way. In this way, each student might not only succeed at the main narrative, but also uncover other important health concerns (much like side quests in video games).

Roles

Players take on one of three roles (medical doctor, environmental specialist, and government official), each of which has different abilities and varied access to information. These roles are each capable of decoding and retrieving information in different ways, as mediated by tools; for example, the Medical Doctor may diagnose Non-player characters (NPCs) and retrieve their medical history during the game. Players must work together, however, as the medical history is of little use without an understanding of local toxins (provided in documents to the government official), and vice versa. These roles were constructed to map to play styles identified within popular games and past research, namely the government official (appealing to those affiliating with power, i.e. the warrior), the environmental scientists (appealing to those affiliating with nature, i.e. the hunter), and the medical doctor (appealing to those who desire to help people) (cf. Squire, 2004). Importantly, these are all productive roles that require scientific training, and a secondary goal of the program is to expose students to a range of roles that they may some day take on in science. Students were free to choose the roles most interesting to them.

Challenges

Players’ challenges (including sub-challenges that arise in the game) are presented through virtual interviews and the artifacts. These provide clues about Ivan’s lifestyle, friends, family, job, watershed, weather, pollutants and the complex interactive systems interlaced through them. Players need to decode the function of these virtual interviews and artifacts to develop either hypotheses or counterhypotheses. The emergence of a new piece of evidence, such as a medical record from Ivan’s coworker, usually verifies or disapproves the hypotheses players have developed. Each piece of information is designed with different functions in mind, and players are rewarded not by having them score, but by having the mystery unveiled piece by piece. They also suggest “red herrings,” tangential questions that might demand further investigation. These are designed to be potentially interesting lines for future research.

Place-based Learning

The site, Lake Mendota, was chosen for its cultural and emotional significance, as well as its potential for supporting scientific understandings. Central to both the city of Madison, Wisconsin and the University campus, the site is situated on an isthmus between Lake Monona and Lake Mendota, which are the subject of great local political, scientific, and cultural attention. As an urban watershed, these lakes gather runoff from over-fertilization and pesticide misuse in lawns and gardens. They are heavily fished, particularly by lower income groups as a major food source, which raises issues about the health effects of consuming fish from Lake Mendota. As with most Midwestern lakes, high levels of mercury are occasionally recorded in fish as a result of point-source mercury pollution. Finally, local industrial sites introduce further complexity, as they add the potential for chemical spills (such as TCE) and industrial waste (such as PCBs).

Resources

In the context of play, players encounter up to 13 non-player characters (NPCs) who propel the narrative, engage the player, and introduce documents. Consistent with the game-based project orientation, the NPCs were written to be as engaging as possible. In this interaction, Bartleby, a friend of Ivan’s, talks to the doctor and environmental scientist. He tells both about their friendship and his fishing habits.

“Fishing really isn’t my thing, but it turned out to be fun, mainly because I got to hang out with Ivan. You see, I don’t really like fish, so I always gave mine to Ivan. Man did he like fish! I bet that you could find fish in his refrigerator at anytime. His wife Even really loved eating fish, especially catfish because they were so much juicier.”

This passage (part of a longer passage) tells the player that Bartleby did not eat fish, but that Ivan and his wife ate a lot of fish, particularly catfish (a bottom feeding fish that frequently has mercury). Bartleby then goes on to tell the doctor,

Honestly, the past few weeks I have been feeling kind of dizzy and dull. I don’t know what’s up though. I have to admit that doctors kind of freak me out, so I haven’t been to one. No offense Doc. I worked out everyday and am feeling much better now. Working out is great. Don’t you think? I don’t touch the booze, though. You might work out sometimes, too, I think.

In contrast, the Environmental Scientist reads,

Like Ivan, I worked at Eraser for a few months as a temp. eRaser is a typewriter correction fluid producer in the northwest side of Sun Prairie, not far from Token Creek... because of budget cuts, they are hiring more temporary workers which has, or had us both a little stressed.

Here, the doctor learns that Bartleby showed symptoms (dizziness, dullness) similar to Ivan, but does not drink alcohol, suggesting that perhaps there was a chemical at eRaser (which is TCE) that has serious side effects, causing players to (potentially) ask what the interactions with alcohol might be. The environmental scientist learns about the location of the plant, which happens to be upstream from Lake Mendota, placing them as a possible contaminator of the water source via TCE. The government official received similar information, but in addition received an ATSDR (Agency for Toxic Substances and Disease Registery) document describing the health effects of PCBs (See Appendix B). Figure 1 shows the placement and functional roles of the various NPCs in communicating the story.

Fig. 1.
figure 1

Screenshot of the gameboard, potential interactions, and their functional/cognitive roles in the game.

Full size image

Collaboration and Competition

In addition to receiving differentiated information and having differentiated tools, the game includes triggered events designed to support collaboration and reflection-in-action. The first and simplest form of these events is that players must decide as a group with whom they should speak, requiring them to anticipate, estimate, and debate the relative quality of information that might come from NPCs (done face to face). Earlier studies of augmented reality environments (cf. Klopfer and Squire, in press) suggested that having triggered actions was critical in promoting the activity as one of inquiry, where participants think through what information to pursue and see such interactions as trade-offs, as opposed to seeing the activity as a “treasure hunt” whereby the goal was to accumulate as many facts as possible. Thus, as players talk to NPCs, new NPCs become available, potentially giving students reasons to evaluate what they know and do not know. Further, NPCs were strategically placed in the game to introduce counter-theories or, at times, to induce reflection. Late in the game, Willy Lowman, an insurance investigator appears in the game, providing a well supported and argued counter-theory that Ivan’s death was suicide. Lowman says,

Let me tell you the truth. Ivan’s death was an insurance fraud. This man could not live without a full-time job, and he had problems finding one. His addiction to alcohol made him sick, and he simply lost the will to live. He was a good husband, but he could not afford to raise his family. What would you do if you were Ivan? He set everything up to make it look like an accident so that his wife could get insurance compensation from his death. I know that it is hard to swallow, but what evidence suggests otherwise?

The hope was that, confronted with a strong counter theory, students would draw on existing evidence and link together rationales to provide a counter example, launching them into a productive debriefing session. In this way, NPCs were designed and placed to propel action, build engagement, promote interaction, and scaffold thinking.

After the game, students were asked to present their findings as a team to a police investigator, played by a facilitator. The investigator wanted to know their argument behind the cause of death, as well as if there were any other environmental or health concerns of which they should be aware. Groups were given 20–30 min to synthesize their findings. Every interaction that they experienced was recorded on the device, and it was expected that students would read and reread texts several times in devising an argument. In these instantiations of Mad City Mystery, there was no one “correct” answer; however, there are theories that are more plausible than others. There is good evidence of a TCE spill at eRaser that caused Bartleby and Ivan to be sick. This fact, combined with Ivan’s heavy drinking, could have caused him to be light-headed and weakened to the point where he might have stumbled and fallen into the lake where he was fishing. His wife, Eve, probably was consuming an unhealthy amount of fish, which could lead to long-term health effects in her and immediate health effects in their baby. Thus, the story was constructed in a way where mercury, one of two of the most critical health concerns in the area, does play a role in the story, whereas urban run-off does not. The plausible but fictional factor of a TCE spill working interaction with other effects is the most likely cause. In this way, the game also sets up students and teachers to think about multiple interacting variables, as opposed to single-cause solutions.

Participants

This study contains participants from three groups of people playing Mad City Mystery for approximately 3 h each. Participants were chosen for convenience, as well as their ability to illuminate key research questions. The diverse range of ages were selected in order to provide comparisons in how learners of different ages would play the game, allowing us to examine how differences in ages, reading level, maturity, and developmental stages affect the game play. Investigating how different groups played the game allowed us to examine the game’s potential to support different forms of thinking. In other words, just as studies of players’ cognition while playing chess or solving the tower of Hanoi puzzle has led to them being used as tools for studying cognition, perhaps studies of AR games may allow them to be used as assessment instruments for students’ cognition.

4th Grade Class

The first case occurred in a 4th grade classroom in a public school located in a middle class neighborhood of Madison, Wisconsin. The school draws from a diverse socio-economic base and containing multiple ethnic and immigrant groups with diverse backgrounds and abilities. The teacher had over 25 years of experience teaching inquiry-based curriculum and essentially situates the year’s entire curriculum within the context of year-long inquiry projects in the community. Eighteen 4th graders played the game as an initial step in a class inquiry-based project on local history and the environment. To ease administrative and logistical issues and to allow parents to participate, the game was played on multiple days after school. This arrangement also allowed the teachers and researchers to follow each group of kids, answer questions, and build closer ties to parents and the community via game play. Children played all three of the roles in most groups, and adults were instructed to play as advisors (allowing the children to make most decisions). One adult worked with each group, in part to supervise while crossing busy streets, and also in part to facilitate discussion. This approach, while not feasible in all schools is common within the Madison district, where teachers use such field trip activities as opportunities to build better relationships with parents. Student and parent teams completed the game in about 2 h, including a short briefing and debriefing.

Middle School Students

The second case includes three male middle school students from a local middle school. They were recruited by the researchers because of one student’s association with the university, as well as their interest in playing video games and innovative games that were designed to be fun and educational. All of them were video gamers, and one student’s mother suggested that her son learned lots of vocabularies from playing video games. They played as a team in a sunny Sunday afternoon, stayed together throughout the game, and complete the game in about 2 h, including a short briefing and debriefing.

Alternative High School

The third case includes seven male students from a local alternative high school located in a middle class neighborhood of a small size city. Students from this high school were struggling in conventional academic programs seeking individualized, project-based learning experiences. These students were familiar with inquiry-based learning and had experienced pedagogical models where they interacted in the community, so playing the game during the school day was not a problem. The seven students were divided into two groups, and both groups completed the game in about 2 and 1½ hours, including extended briefing and debriefing.

Data Collection

In order to capture the practices, thinking, learning, and argumentation emerging from participation in this place-based augmented reality game, researchers collected both qualitative and quantitative data within a case study framework (Stake, 1995). Consistent with this framework, we structured data collection around emerging phenomena that proved to be important for describing the case in terms of the research questions.

Observations

Between two and five researchers attended each session, with each researcher following a specific group, taking field notes while also videotaping events. Researchers focused data collection on participants’ talk in an effort to determine (1) how were they orienting to the task (e.g., were they emotionally and cognitively engaged?), (2) were they adopting the perspective of their roles, (3) in what forms of argumentation they engaged; (4) how and when physical place entered their discussion; (5) what prior knowledge was triggered in discussions, and (6) how they used text, audio, and video resources in the context of game play (including debriefing). Researchers kept brief field notes during each session and performed extensive debriefs afterwards, which were written up as field notes. In addition, researchers recorded roughly 25 h of game play cutting across these groups. Each tape was reviewed, with at least one group from each class transcribed in its entirety.

Interviews

Students and teachers were interviewed after playing the game. First, we conducted a 15–20 min debrief as a group to gain players’ general impressions of the game experience. Following recommended game debriefing procedures (cf. Heinich et al., 1996), this asked participants to reflect on (1) how they felt during the exercise; (2) what happened during the game; (3) how this relates to what actually happens in the Lake Mendota watershed, including what they felt they learned about the watershed from the game; and (4) how this relates to future investigations they might conduct. Selected students were interviewed separately. When possible, clinical interviews were also conducted to probe students’ conceptions of watersheds and inquiry more deeply. Teachers were debriefed separately after each experience.

Assessment Instruments

Questionnaires probing students’ attitudes toward science, attitudes toward gaming, and experiences in this game were administered before and after the game, as were questions probing their factual understanding of mercury, TCEs, and environmental issues around Lake Mendota. Students were given concept maps to critique that included pictures of lakes, anglers, and factories. Finally, they were given short text descriptions of similar problems and asked to outline how they would research the phenomena.

Data Analysis

The two researchers met and debriefed after every game session to identify watershed events in the unfolding of each case, as well as key themes relating to student engagement, thinking, and learning. Based on these interactions, researchers targeted student groups to study in greater depth and selected videotapes to transcribe and analyze in greater depth. These groups were targeted according to specific criteria, including reading level, attitudes toward science, gender composition, and success in the game. The researchers then examined the transcripts to identify examples of students’ interaction relating to the research questions. The cases were transcribed into a database so that researchers could analyze students’ talk around specific NPCs and examples. This enabled us to examine, for example, how each group reacted to the Willy Lowman NPC. Each transcript numbered 12–20 single spaced pages. Researchers transcribed speech and major gestures, but did not attempt to capture nuances in voice, full gestures, or voice inflection.

Upon completing each transcription, the researchers completed short case summaries for each group. Either in written form or orally, these summaries were shared with participating teachers in each case. Based on these debriefings, participating teachers also suggested changes to the game and game play. Careful attention was paid to both aspects in which teachers and researchers felt that the game succeeded in meeting teachers’ goals, and aspects in which it failed. After transcribing each case, reading the summaries, identifying themes, and debriefing with teachers, we constructed case studies of each group, presented here. The results are presented in the form of three case studies, each following one group that was chosen for its ability to illuminate key research themes. Each case study includes data to suggest the unique conditions of the case, as well as data by which the reader might determine its naturalistic generalizability, the extent to which the findings might be applicable to one’s own situation.

Results

The following section provides the case study results from three implementations, and is designed to give the reader a sense for both the data and the broader instructional context. Each focuses on one particular group, but also provides examples from other groups to illustrate where that particular group was typical or unique. Cross case comparisons are provided in the conclusions.

4th Grade Class: The Eager Researchers

On a series of early fall evenings in 2005, Mark Wagler’s 4th grade class stopped being students and became medical doctors (MDs), environmental scientists (SCI), and government officials (GOV). This particular group, comprised of three girls, eagerly stepped into their roles, and maintained strong communication throughout the game. They were among the highest performing groups, and we have chosen to focus on their performance to illustrate (one model of) effective communication in the game context. Other groups, particularly those with weaker readers, featured less communication. Consistent with the design-based approach, our goal is to create effective learning environments, document how they operate, and then unpack the processes by which they work.

After reading the debriefing, the girls quickly walk to unlock their first NPC, Ivan’s coworker Bartleby, and another NPC, Ivan’s doctor. Each girl took turns reading her text aloud, occasionally skipping text to summarize the findings for the group. On occasion, each girl would ask to look at the text on another girl’s machine, leaning over her shoulder to read. Here, the student playing as the doctor reads aloud, interrupting to summarize from her interview with Ivan’s doctor:

  • Student Doctor (MD): ...6–8 glasses per evening...That could have contributed to his deteriorating health and increased the risk of developing diabetes and stroke...

  • Student Scientist (SCI): True. Maybe he fell over (interrupted by MD)

  • Student Gov. Official (GOV) : Maybe has been eating ice cream and fishing, and he had diabetes...(interrupted).

At this point, the girls stop discussing and move on to the next NPC. Like all of the groups in Mark’s class, these girls frequently rushed from site to site, attempting to trigger the next point as quickly as possible. They were unsure as to why some NPCs (the doctor, the mother and son) were standing around on campus, while they took joy in finding other avatars in unusual places (such as walking out on the pier to interview anglers on Lake Mendota). This finding was consistent across all groups and came up repeatedly in the post interviews as well.

This group, like all of the younger students enjoyed interjecting their own theories, such as the theory that “perhaps he had been eating ice cream.” Ice cream had never been mentioned in the game scripts. Perhaps she was connecting diabetes to something she knows about sugar intake and diabetes? As they gathered more data, their hypotheses are more rooted in data presented in game.

Fifteen minutes later, the group meets another NPC, Mary Shirley, an Environmental Studies doctoral student. After the interview with Mary Shirley, the government official receives a document from her that she had picked up from Ivan’s wife, explaining the health effects of PCBs. (The team received the PCBs document from Mary Shirley, and the document reminded the team (especially the MD) of their earlier interview with Eve and Adam, Ivan’s wife and baby, from which the MD received a medical record that indicates Ivan’s baby was slightly underweight and showed abnormal responses in tests of infant behavior.)

  • GOV: PCBs...used as coolants...may remain dissolved, but most stick to organic particles and bottom sediments...reaching levels that may be many thousands of times higher than in water...PCBs had babies that weighed slightly less...showed abnormal responses in tests of infant behavior.

  • MD: Guys...since he likes to eat fish, maybe he ate PCBs...and the babies who are exposed to PCB....they weigh slightly less and they have abnormal tests of something...and the baby weighs less.

The passage begins with the government official summarizing her document for the others who did not receive it. This design feature of providing unique information for each character produced “jigsaw” discussions across most groups, although in a few cases the groups failed to notice that a team member had received unique information. The medical doctor makes a link among PCBs, the fish, breast milk (which is implied), and the baby, thinking that perhaps PCBs were present in the fish.

  • Teacher: (While walking) You girls have any theories about Ivan?

  • GOV: I think that he caught all these fishes and then he got these diseases and then he was fishing and then he got the diseases and then he died.

  • MD: Oh, NO. I think he...

  • SCI: I think we need to talk to more people...I think Ivan died...(interrupted).

  • GOV: Maybe he was drinking breast milk.

  • SCI: Yeah, really!!!

  • MD: No. Maybe Ivan ate the PCB and got the disease...

The teacher, who has been observing, asks the group if they had thought any more about what may have caused Ivan’s death. The government official begins with a crude, but possible theory that Ivan died of diseases contracted from fish. The medical doctor starts to present a counter theory, before the environmental scientist interjects that they need more data. The government official notes a missing link (how did Ivan get the diseases), perhaps joking that it was from breast milk. The medical doctor hones in on PCBs, before the group takes off to the next location.

The emergent hypothesis fails to explain why Ivan died at that particular time, nor does it specify which disease; but their conjecture reflects an early picture of the case that, while incomplete, begins to incorporate major variables. Students did not look for disconfirming evidence, nor did they seek out alternative hypotheses. They also gravitated toward simple causal explanations, rather than taking multiple factors into consideration. For each of the 4th grade groups we studied, the process was one of “putting together the right facts to stumble on the right answer” rather than an iterative process of gathering evidence, developing hypotheses, and testing theories. Upon receiving a new piece of information, these groups would develop a new hypothesis “whole-cloth,” without attempting to integrate it with what they already know.

However, a few times this group (unlike the others) did stop to reflect on what they know. By this point, they were exposed to nearly all of the salient facts. They still were jumping from hypothesis to hypothesis, based on their latest interviews. At this point the government official read her interview aloud while her teammates read from their PDAs partially tuning in to what she read.

  • GOV: I think there are two things we need to figure out. One thing is how it affects grown-ups and the other thing is...if could just be a coincidence that he died fishing and it wasn’t really the reason that he died...could be like he was really sick and jump off the cliff....

  • MD: (Trying to propose an alternative theory) Maybe he was depressed....because he lost his job and gained lot of weight. He was drinking too much because he was depressed.

  • GOV: What I am saying is that it might be that he was really sick, it might be something else... (she attempted to call up her PCB document but was confused by the interface and unable to).

As the group headed back to debrief, they realized they had run out of time, and the parents quickly led them through a guided debrief. However, even with guidance, the girls could not comprehend each piece of text and differentiate the relative importance of difference pieces of data. They discarded relevant variables when the game provided evidence supporting an alternative hypothesis. One group, for example, met Willy Lowman and immediately decided that “the case must be a suicide, since it’s near the end.” In more than one group, Willy’s counter-theory was strong enough that, when the students confronted it, they shrugged their shoulders, thinking it was over (understandably, they actually looked a bit depressed). However, they continued to look for a simple cause (such as a poison that would kill you immediately) and were unable to put together the key variables (TCEs, heavy drinking, mercury and PCBs for the baby) in order to generate a good theory.

This pattern of trying to find a single “right answer” was representative across all groups, although those groups with weaker readers and less active parent participation struggled more. A few groups devised strategies to make up for poor readers, including shoulder surfing to read their PDAs. However, even this strategy was flawed when one group member would fail to notice a secret document or critical piece of information. Those groups with active adult facilitators were able to stay focused on relevant theories more often. The adults engaged in fairly straightforward facilitating, asking them to reflect on what they know, telling students what they need to know, and questioning them on specific pieces of evidence that they failed to consider. Indeed, an interesting route for future research could be attempting to embed more similar scaffolds in the game.

Middle School Students

On a similarly beautiful fall weekend, three middle school students volunteered to play Mad City Mystery to further their interest in game design. Like the 4th graders, this group eagerly jumped into the game. As they read the interview with their first character, Bartleby, the doctor queried the group:

  • MD: Did he (Bartleby) say he ate any of the fish? Because if he did...

  • SCI: Nope. He gave them all...

  • MD: Because if he did, it could mean that he got dizzy and fell out of his boat and drowned. It might have been something like the fish or something, but since he did not eat the fish, it might be something in the air. We just have to go to the next person.

The medical doctor hypothesized that perhaps eating fish (presumably on the spot, in the water) could cause someone to drown. Observing that Bartleby did not eat the fish, she raises a counter hypothesis: perhaps there was an airborne pollutant (an original hypothesis; no evidence of an airborne pollutant has been introduced). They decide to gather more evidence from Ivan’s doctor.

  • MD: Yeah. This guy is his doctor. He said Ivan was also suffering from dizziness like the other guy (Bartleby) we interviewed.

  • SCI: Ivan was feeling dizziness, too.

  • MD: Yeah, they both felt dizziness...I don’t think it was fish. I think it was something in the air.

Noting that both people felt dizzy, yet only Ivan is sick, they reject the hypothesis that eating fish was responsible for the illness. They tentatively concluded that it was an airborne illness. The group decided to review their interview with Bartleby.

  • GOV: What did the other guy (Bartleby) say?

  • SCI: He said that he had been working out because he had been feeling dizzy. Bartleby said that they worked at a place called eRaser and they were good friends. But they don’t like hiring full-time people.... That’s how they met.

  • MD: Since he is an alcoholic, he is more sus- (susceptible—he is not sure how to pronounce that word).

After rereading the interview, they observe that both worked at eRaser, but did not spend significant time reading the “secret documents” that introduced specific information on TCE. The students continued to discuss potential causes, now confining their discussion to evidence raised in these two interviews.

Unlike the primary school students who read aloud to share information, this group of middle school students read the interviews to themselves and summarized the findings for group members. These summaries highlighted key information gained, addressed holes in their understandings, and frequently concluded with new “best guess” hypotheses that sought to account for as much of the data as possible. This format differed from the primary school students in that they focused more on evidence contained in the case and synthesized information into a narrative, rather than treating each piece of evidence and logical hypotheses that extend from these as exclusive. The older students also acknowledged that they needed more information and treated forthcoming interviews as new data points contributing to their emergent understandings.

The next interview was with Captain Ahab, who introduced interconnectedness and complex causality.

  • SCI: It said that there could have been pollution running into the river.

  • MD: Is that what he told you?

  • SCI: Yeah...like we learned at science that poison on the grass and grasshopper at it...and then fish ate it and got the poison, and then people ate the fish and got the poison.

  • MD: Environmental magnification, I think. (What they have learned in class should be “biological magnification”)

  • GOV: Yeah, I notice that’s magnification.

  • MD: We just learned about that a while ago...biology.

Interestingly, this group read Captain Ahab’s quotation and brought the concept of environmental magnification to bear on the problem. While walking along the pier, they looked over the edge to investigate the water depth. They were uncertain as to if the water was even deep enough for someone to drown in that portion of the lake.

At the end of the game, they have a short discussion about what happened in the game, coming to the conclusions that Ivan was poisoned by an airborne toxin (because he fell into the water while fishing), or that he committed suicide to obtain insurance money. The facilitator asks them to explain their case.

  • MD: I actually think it might be an insurance fraud because he knew how to swim. He was gaining weight, but he knew how to swim and everything.

  • GOV: Yeah, what if the toxic made him unable to swim? If he was dizzy and unconscious, it would not matter if he knew how to swim.

  • MD: Oh, yeah!

  • SCI: Maybe some people think it was an insurance fraud, but it might be an accident.

  • MD: Well, because he doesn’t have a job or anything. And he has a new-born child, he wants to keep them happy, healthy and everything. He has been drinking a lot so he gets depressed. He can’t think of any other ways to get money so he tries to make the insurance fraud look like an accident and that gives the family the money they need...like CSI.

  • GOV: Yeah!

  • GOV: It’s possible there were chemicals in the water...well, there were not chemicals in the water, but there were definitely chemicals in the fish. So it’s possible that it’s just accumulated and it happened when he was in the water. It’s a little bit less likely that way, but...

  • MD: Well, I think that might work also, but I can go either way.

After much debate, the group settled on these two probable explanations: either Ivan breathed in an airborne toxin or he committed suicide. To their credit, this group deduced that mercury, PCBs, or any of the other toxins mentioned were not going to create an instantaneous death. However, they failed to investigate the health effects of TCE or to consider the interaction of chemicals. The debates about suicide were much more zealous than expected; whereas most groups oriented to the activity as “friends” of Ivan, this team oriented toward it more as investigators, with one student even commenting that “it was just like CSI (Crime Scene Investigation, one of many crime television programs currently popular in the United States).” They did not specify where the airborne toxin came from, and they essentially adopted the insurance investigator’s argument about suicide.

Although this group developed reasonable hypotheses and sought evidence to confirm or disconfirm their hypotheses with new evidence, they were less successful in putting the case together as a whole. Unlike the elementary students, they did seek to develop a logically coherent narrative that wove together evidence and multiple narrative threads (e.g., they wanted to know why Ivan has similar symptoms with Bartleby, but Bartleby was OK). When confronted with logical conflicts, they sought to reconstruct their story or try to find new facts which might support their narrative. Also, they drew on prior knowledge—including theories from school—in developing a logically coherent story. While both processes occurred at times in the primary school groups (i.e. the SCI from the primary school mentioned the relationship between Salmon and mercury and asked parents if Salmon could be found in Lake Mendota), the narratives constructed by primary school students in this study tend to be more fragmented and incoherent.

Alternative High School Students

On a sunny spring day, seven male students from an alternative high school took off from school to play Mad City Mystery. The class divided into two groups and began investigating the death of Ivan. The group occasionally read their interviews aloud, but most often synthesized key points. The other high school groups, which are not included here due to space limitations we examined used similar reading strategies.

Upon reaching the first interview, the medical doctor proposes their first hypothesis.

  • MD: (reading the interview) Hey since you are a doctor, mind if I ask you something? For the past weeks I have been feeling dizzy and dull (now not reading). OH! HE (Bartleby) HAS BEEN POISONED BY THE FISH...(Reading continues). I don’t know what’s up though. I have to admit that doctors kind of freak me out, so I haven’t been to one. No offense, Doc. I worked out everyday and I am feeling much better now. Working out is great, don’t you think? I don’t touch the booze, though. You should work out sometimes, too, I think. (No longer reading). He has been poisoned....All right, I bet that’s why the other guy (Ivan) died.

  • GOV: In the concept map, they talked about mercury. (In post interviews, the participant also mentioned that he had heard of mercury advisories for fish).

  • MD: Yeah...I know, that’s why I am saying, and he gave those fish to that one guy so I guess...let’s go to the next point...(they saw Dr. Zhivago on map) we will go to a doctor or whatever.

Upon completing the first interview with Bartleby, the MD proposed that “he (Bartleby) has been poisoned by the fish,” which is particularly interesting because the student developed this theory based on events (prior experience, perhaps the concept map) conducted before the game. Mercury is not mentioned in the game yet.

They proceeded to read the next two interviews (which provide the story of Ivan’s life) without discussion. While he read, the doctor offered another proposition.

  • MD: This kid has been poisoned...I think...from the water. I bet that Ivan died of poisoning. The baby was feeling bad too, and so is the mother. Neither of them is overweight. (reading) “Diagnosis (Eve and Adam): Eve: Appears slightly overweight. Adam: Weighed 13.9 pounds (Average weight is 14.7 pounds for 5 months old baby). Showed abnormal responses in tests of infant behavior.” Remember that he was showing the same symptom as his wife and son, so they are...both of them are overweight but he was the only drinker, so it could have been alcohol.”

Here, the doctor sought to adjust his story by weaving together new evidence about the baby’s illness, while accounting for their different symptoms and life history. He proposed an interaction between poisons and alcohol. The government official picks up on this line of thinking.

  • GOV: Ok, let’s see poisoning. But is it like food (unclear) poisoning or is it poison (interrupted by the MD).

  • MD: I think it’s fish poison...cause there are tons of mercury in lake. (Note that thus far mercury has not been mentioned in game.

  • GOV: I know.

  • MD: I don’t think it would have been intentional poisoning, Andrew (Andrew is the Government Official).

  • GOV: Well, we don’t actually know any motivations yet.

  • MD: I mean because...remember the wife and the child were showing sings of being overweight (Adam is slightly underweight) and so was the dad...wife and child...not drinker so.. it could have been alcohol.

  • GOV: Yeah.

  • MD: And if they were all eating fish so that’s the only thing we know they are consuming.

  • GOV: Right...(unclear).

  • MD: I remember Bartlely or whatever his name was...he wasn’t eating the fish. He was giving them to...

  • GOV: He did not fish with Ivan?

  • MD: But he gives the fish to Ivan.

In this exchange, the group digs into some of the key pieces of evidence (that Bartleby, Ivan, Ivan’s wife and their baby all exhibited negative symptoms that could be associated with contaminants, but not everyone had eaten fish nor drunk alcohol). Notably, their understanding of mercury is still based as much on the concept map (and perhaps prior knowledge) as anything presented in the game.

The group proceeded through the next interviews in similar fashion, dialectically devising theories and checking hypotheses. Finally, the group decided to check a sample from a fish that has washed ashore to check for signs of mercury.

  • SCI: (paraphrasing) Big walleye...Toxic test result..allowable limit...doesn’t have any amazing mercury...

  • MD: (interrupting) All right. Let’s go to Mary Shirley!

  • GOV: (talking to MD) mercury poisoning...I know it speaks right to you.

  • MD: I know it is positive...positive answer.

Although other evidence is available, such as high PCB levels in catfish (consumed heavily by Ivan’s family) and documents linking PCB exposure to low weights in infants, the team held on to their hypothesis that mercury is the primary toxin behind these illnesses. For them, that these mercury levels were not far beyond EPA guidelines was not central; it was more important that traces of mercury were found, which confirmed “well enough” that mercury was found in fish. Given this, they interpreted the data to be evidence supporting their theory.Footnote 3

Next, the group encounters Willy Lowman, which elicits an emotional response from the MD.

  • MD: ...he (Willy Lowman) is wrong. I think obviously the runoff from something...put mercury in the lake. The catfish ate...the catfish consumed the plankton and absorbed the mercury, and then he ate the catfish and brought some home for his wife. That’s why his wife and kid are sick. And he is sick. And the wife transferred it to the baby through breast milk, but not substantially. And the kid suffering from nervous disability so honestly he (Ivan) had died of mercury or some...

With this group, we see the Willy Lowman interaction functioning as designed. Confronted by Lowman’s counter-theory, they articulate their emerging theory of how mercury, present in catfish (right at the EPA suggested limits) could be causing these health effects. The MD’s speech reflects that of a student who is emotionally and intellectually engaged with the problem. He was visibly excited, perhaps almost even offended with Lowman for this assertion.

After meeting Lowman, the group returns to a room for debriefing. As they review evidence (stored on their PDAs), they note some data that they had missed, eventually revising their hypothesis and generating a new narrative explanation in face of evidence.

  • GOV: (Reviewing secret document he received) For TCE...symptoms of headache, dizziness, nausea, and unconsciousness...Bartleby said he was...(interrupted).

  • MD: So TCE. We never found anything about TCE though.

  • GOV: I think we did.

  • SCI: We did in the fishery talks.

  • MD: So it may not have been mercury. Could have been TCE!

After roughly 20 min of discussion, the group offered their theory to the investigator. They were confident that exposure to TCE was a key variable leading to Ivan’s death. They deduced that the baby was exposed to PCBs via breast milk. They were concerned about the existence of mercury in the fish, but did not believe that it led to Ivan’s death. They suggested that local authorities investigate each of these issues. They stressed the importance of considering multiple interacting variables in the case.

In post interviews, the teacher communicated surprise with the extent to which students were engaged by this activity. In e-mail he wrote,

The students that you worked with all have a history of poor school performance and have difficulty learning in a traditional school environment/classroom. As you probably gleaned, some of them also have issues with communication! The fact that they were engaged and excited for an extended period of time is a great sign for the power of your design and the associated technology and delivery system. You are definitely on to something! The students I talked with on Monday are very interested in trying to create a game that they can share with other students at our school. Is there any way that we can get rights to the engine that you used to create Mad City Mystery?

In post interviews, students made similar comments. An overarching comment from students was “Now I look at the lake differently.” One commented, “We are using technology, thinking with complicated science content, what more could you want?” Another reported that he had heightened interest in the subject matter, “Before I never would have picked up a book on TCE, but now, I definitely would.” Another said, “I would pay for something like this outside of school.” Of course, the self-reported nature of this data makes these statements somewhat suspect, but the utterances do speak to their enthusiasm for the learning experience. A year after the implementation in this classroom, students made similar comments, asking when we would return to play another game with them.

We followed with questions investigating what they enjoyed. The students mentioned (roughly equally) the opportunities to use technology, the creative problem solving, and chances to “get a semi-view of the occupation.” When asked if the activity was authentic, perhaps tellingly they responded, “yes.” Several felt that “it gave them a preview of what’s new in game technology,” which interestingly was perceived as more authentic than normal school activities. Working with technology for these students was thought of as an element of “authenticity.” Others mentioned that they “actually got to do something.” Interestingly, for these students, working with technology, particularly games, was perceived as more authentic than typical “school-based” activities.

Cross case discussion

This study focuses on three different groups of students playing Mad City Mystery, in order to investigate the pedagogical potential of this augmented reality simulation game in science education. This convenience sample of three groups was selected to contrast the different ways in which the game can be played, to elucidate the range of thinking and problem solving that the game can produce, and to suggest the range of different educational applications. Across groups, we saw game play as a process of reacting to the emotionally laden cover stories presented raising new questions based on information given; seeking out information from NPCs, documents, and the physical environment; critically interpreting qualitative and quantitative evidence; and forming hypotheses and theories based on this evidence. Building on the work of Steinkuehler and Chmiel (2006), we present a case here for scientific argumentation as a central component of game play and illustrate reasons for designing games as mechanisms for students to engage in complex scientific argumentation.

Reading, Comprehension, and Communication

Although most students in the study read at about the 8th grade level, older students engaged in more sophisticated reading practices which increased their success in building coherent arguments to explain Ivan’s death. Although the participants in this study ranged from ages 9 to 15, most read at about the 8th grade level, providing interesting opportunities to compare how students of different ages approached the game task. The youngest kids read aloud, treating the evidence as possible “explanations” to be evaluated for the truthfulness. They readily adopted and disregarded explanations as new evidence arose. In contrast, the high school students scanned the text, reading for critical points. The primary students read almost every piece of information, but they were less strong at comparing evidence and integrating them into an overarching narrative. Both the middle school and high school group regularly returned to prior pieces of evidence, reading each interview and document 3–4 times by the end of the game in an effort to piece together the most coherent narrative. In contrast, the younger students read each passage, evaluated it on the spot for its ability to explain the case, and then largely disregarded it.

Perhaps, because reading aloud is not commonly practiced beyond elementary school, none of the middle school and high school students read aloud. Instead, they skimmed the text, reporting key information. The groups regularly quizzed each other on the content of their passages, asking if they found particular pieces of information (e.g., Did the doctor mention TCE?) or asking one another to review the general “gist” of the documents. Comparing the talk across groups, we see that the older students engaged in more rapid turns of talk querying one another for evidence, interpreting results, and building an argument for the cause of Ivan’s death.

Scientific Argumentation and Literacy

The game play engaged students’ in the practice of scientific argumentation. Simply participating in the game required students to weigh evidence, develop hypotheses, test them against evidence, and generate theories based on this evidence. All groups observed here engaged in argumentation cycles similar to those advocated by science educators and thought to be difficult to produce in classrooms (cf. Kuhn, 1999). The younger students more readily developed and abandoned hypotheses based on new forms of evidence, showing a preference toward relatively simple causal models. In contrast, the high school students entertained fewer hypotheses, holding to their hypotheses until contrary evidence was found. Because the game play itself largely became an exercise in scientific argumentation, educators might pursue scientific argumentation as an important direction for educational gaming, developing extra curricular resources around the experience designed to explicitly teach argumentation and facilitate transfer.

These activities suggest that such games could have promise for tools that develop scientific literacy (of a particular sort, in this case environmental science). Effective problem solving required students to read text, generating meanings, debate those meanings, and generate ideas based on what they read. Students must make conjectures based on what they read, seeking out new information to develop their understandings. Across groups we see students reading with a goal of understanding, in an effort to understand phenomena in the world (as opposed to memorize for a test). Further, dialogue in groups suggests that such games could be useful tools in developing students’ oral language skills, a critical variable in later academic success (Gee, 2004).

Although previous research has stressed the difficulties that students have in scientific argumentation (Kuhn, 1999), these findings suggest that some secondary students are capable of one particular sort of scientific argumentation, within at least this context. Consider how evidence, rationale, hypotheses, and theory are developed in the exchange reported in the high school case (Table II).

Table II. Players’ Dialogue and Argumentation
Full size table

The high school group in particular engaged in series of rounds of this kind of conversation, raising and rejecting hypotheses, providing evidence and counter evidence, slowly and iteratively building a theory of events. They ask themselves what they know and do not know, acknowledging areas where they need more evidence.

Not all participants engaged in this rich dialogue with reflection and argumentation. The elementary school students—although being similarly skilled readers—struggled much more with the thinking tasks in the game. They tended to raise and reject hypotheses with every piece of new evidence. This finding suggests that, as Kirschner et al. (2006) argue, a challenge for educators is creating developmentally appropriate tasks that engage prior knowledge, encourage students to enact their multiple identities (particularly out of school) and introduce and manage complexity for students to help them think scientifically.

Roles as Mechanisms for Learning

The game roles encouraged collaboration and served as a scaffolding for reading. Specifically, they encouraged students to share information, synthesize what they read, communicate orally with their group, ask questions, and debate meanings. Because no one player had enough information to develop a coherent narrative of events, the group was forced to read, synthesize, and discuss findings. The older students in particular chose not to read aloud, but to synthesize findings for other teammates. The kind of questions that students’ asked (e.g., “What did the doctor tell us?”) and observations students made during discussions (for example, “That means it could be TCE,” or “we need more evidence”) suggest that effective reading strategies (e.g., asking what do I know, what do I need to know?) would aid their learning. Social interactions became distributed in roles and in the environment, in a manner similar to jigsawing (see Brown and Campione, 1996). Consistent with literature on transfer of skills in reading strategies, educators might benefit from incorporating reflective activities that call students’ attention directly to these mechanisms, encouraging them to adopt these practices in their own reading.

Students willingly assumed roles, valued the experience of being in them, and showed partial evidence of orienting to the game from within these roles. This phenomenon was most pronounced with the middle school girls, who created “hybrid” roles that were partially the roles designed for them (government officials, environmental scientists, and medical doctors) and partially roles that they saw in science investigation shows on television. These findings are especially promising, given research suggesting girls’ interest in forensic science (cf. Laurel, 2001). The high school boys also listed “the chance to get a semi-realistic view of the profession” as their favorite part of the game, leading us to speculate that future iterations could benefit by deeper and more purposeful induction into the roles. Much like Shaffer’s (2005) findings that epistemic games motivate kids by providing ways of thinking based on the professions, we argue that games give kids opportunities to solve problems in a manner similar to the professions. What is gained and lost by a rigorous adherence to the ways of thinking of particular professions is an intriguing area for future research.

Place

Combining game structures with physical space created a hybrid “third space” that was neither completely fantastic nor completely real, enabling students to engage in plausible scientific investigations that (1) have significance (or authenticity) to them while (2) serving as cognitive scaffolding for activity. Participants developed a logical coherence for the game, willingly devising plausible theories (mercury, diabetes, environmental (biological) magnification), yet holding one another’s arguments to a standard of plausibility. The most obvious example was mercury, a toxin commonly known to be in Madison lakes; but other groups frequently mentioned algae blooms, agricultural run-off, and urban run-off as well. We argue that playing a game in places familiar to students encouraged them to apply what they know (or are familiar with) about local environmental issues to the problem, as well as challenges them to consider how abstract scientific concepts (such as environmental (biological) magnification) play out in their communities. We find this desirable not only for supporting transfer, but from a democratic perspective of framing science as a tool for understanding phenomena in local communities.

Participants expressed a desire for an even tighter coupling between characters, narrative events, and local space. Both during game play and in post interviews, participants (including teachers) commented that the game was most successful when it tied to place in specific ways, and least successful when it felt artificially layered upon the landscape. Although each group stayed engaged throughout the game, some groups felt their immersion being broken when NPCs were placed in implausible locations or were reacting to investigators in implausible ways.

Limitations of this study

We believe that this study demonstrates the potential of place-based augmented reality games on handheld computers to engage participants in scientific argumentation, but we also need to acknowledge several limitations of this research. First, the activity itself is relatively short duration, much shorter than an actual investigation and too short to develop the most meaningful scientific investigation. Each game itself took 90–120 min to play (depending on play style), plus 30–60 min of set up and debriefing. If split across a regular school week, this activity might be comparable to 1 week’s work. One might (justifiably) challenge this study based on what meaningful learning can be expected to happen in 1 week or even 1 month of activity (cf. Lemke, 2000). Particularly from a socio-cultural perspective, one might prefer to see a study conducted over not just several class periods, but instead multiple games and investigations, looking for students’ movement along trajectories from thinking like a pupil more toward thinking like a scientist. As is, we can present evidence of students engaging in meaningful scientific argumentation through participation in this program, but cannot show movement along this trajectory.

A second limitation is the constrained nature of the inquiry itself. Consistent with Chinn and Malhotra (2002), we have attempted to design an activity that represents one component of the inquiry process (argumentation and justification), while “black boxing” others such as data collection (interviewing techniques, field techniques). Students are not developing their own questions nor identifying the standards by which their inquiry will have succeeded. Their inquiry activities do not result in any “real” difference in the world (e.g., they are not writing letters to the newspaper about mercury or toxins in the environment. We have tried to create an “authentic” inquiry experience, using this intervention as an opportunity to investigate scientific argumentation more specifically. However, from a critical or democratic education perspective one might criticize this curriculum for not having students more deeply and systematically engaged in inquiry. The high school and elementary teachers both used this game as a part of a broader inquiry-based curriculum where students designed games themselves. Indeed, we believe that an affordance of this game could be using it as a springboard preparation for a longer (say 1 or 2 month long) inquiry experience. Participants in this study did ask a number of questions (e.g. is it safe to eat fish in the lake?) that might make excellent inquiry projects.

Another limitation of this study is the lack of systematic pre-post data on student performance which might be useful for making broader assertions about learning as a result of participation in this game experience. We developed pre and post-test instruments to students, but had difficulty administering them for logistical reasons (students arriving late, inadequate space outdoors to complete a test). In post-tests, we found that most students offered incomplete responses, due perhaps to being tired (it was after a 3 h intervention). Due to the implementation problems, that data set was not included here.

One of the difficulties in effective research is generating assessment exercises that yield evidence to make valid interpretations on student learning in such a rich domain. In future studies, we are exploring having students play a second or third game to identify if they engage in new forms of argumentation that draw on their experiences as students in the first game. We believe that the strongest evidence may be in having students engage in an inquiry project afterwards, examining if playing the game prepares them for future learning in an inquiry-based environment (Bransford and Schwartz, 1999). The kinds of discussion created via game play suggests that place-based augmented reality games on handheld computers may be an effective tool for gauging students’ scientific thinking.

A final (potential) limitation is the active role that investigators and facilitators played in supervising the game. In the middle school and high school cases, the group numbers were low, although in theory, there is no reason that the teachers could not play it in the school day. With the elementary school students, we must acknowledge the important role that their supervision played, whether it be simply keeping students on task, or intervening with an occasional question or prompt. These kinds of prompts can be useful in supporting learning, possibly indicating that those schools looking to implement such a program with younger students may need to leverage parents, community volunteers, or other research partners to fully implement such a program.

Implications

This study argues that augmented reality games on handheld computers are an exciting new pedagogical model for developing students’ scientific literacy, particularly their argumentation skills. Playing augmented reality games immersed learners in a kind of scientific argumentation that is purportedly difficult to achieve and yet desired by science educators as a primary goal of science education. We argue that this success is achieved by using game mechanics (challenges, roles, resources, place, and collaboration/competition) to serve as a scaffold for student thinking. We have sought to show that creating differentiated roles, NPCs designed to elicit reflection, and multi-layered challenges encourages students to engage in a deeper kind of thinking than they otherwise might. As such, we are exploring game-based approaches as a type of post-progressive pedagogy that guides and supports students’ thinking while enabling them to engage in emotionally meaningful, cognitively complex tasks. Advocates of game-based pedagogies who have expressed concern over the lack of sound pedagogical goals and methods arising from games (e.g. Gredler, 1996) might look to role playing simulation games as one family of games useful in building students’ scientific argumentation.

We argue that educators exploring such approaches might benefit by generating “mid level” instructional theory that seeks to identify links between specific game structures within specific game genres toward specific learning goals. So, whereas previous research has tended to ask questions such as “Are competitive structures useful for learning?” we argue that more holistic approaches seeking to identify how these structures interact toward creating a gaming experience (cf. Squire, in press). In this design based research study, we argue that we were able to produce rich scientific argumentation through the interaction of at least five factors (which roughly map to the design features of the game): the task, social configurations (roles), embedded resources, context (or place), and encompassing activity system.

The task is designed to be emotionally engaging, drawing on themes that elicit engagement from students. Similarly the roles invite students into new and interesting ways (for them) of participating in the world that trigger particular ways of thinking. Unlike school-based assignments, where students are expected to be students, these participants perceive themselves as role playing as investigators. These roles are also designed to scaffold thinking. They encourage argumentation as students are required to raise evidence and counter-evidence. They make students responsible for different portions of the problem, adding to engagement and scaffolding scientific argumentation. The embedded resources support argumentation by (1) communicating a sense of authenticity to students while also (2) managing complexity for students, and (3) introducing new evidence and hypotheses at particular points in the game. For example, the insurance investigator interaction was designed to (and did) elicit moments of reflection in students. Taken as a whole, these features interact creating a situation where students are able to argue through scientific problems in a more sophisticated manner than otherwise predicted.

While the previous three factors focused on particular game design features, we believe that two more general factors also contributed to educational effectiveness: the place-based nature of the game and the encompassing activity system. We argue that situating the problem in space, scaffolded students’ thinking about mapping interactions onto specific people and locations, with the map functioning as an externalized cognitive structure to guide thinking. Situating in a specific place allowed students to draw on pre-existing knowledge, such as the physical lay-out of space and known potential toxins in the environment (e.g., mercury). Students readily developed ownership in the problem and exhibited comfort moving in the space. Further study is necessary to develop more definitive evidence speaking to this issue.

Finally, we argue that the importance of the overall context in producing learning. Each of these studies occurred outside the confines of the normal classroom, with the emergent culture being a combination of the game culture and the classroom culture. In each, the pressure of grades and standardized tests were removed. This made some exercises, such as collecting pre and post data difficult, while perhaps making others (the open exploration of science problems) easier.

Similar to research on constructivist learning, we find that moving to game-based approaches involves a different orientation toward learning on part of students, teachers, and perhaps even researchers. This shift means embracing fantasy, failure, and divergent learning goals. Teachers reported students’ enthusiasm for science, inquiry, and their local communities as a major, worthwhile outcome of this study, which stands in stark contrast to the current rhetoric of accountability.