Earth science learning in SMALLab: A design experiment for mixed reality

Article

Abstract

Conversational technologies such as email, chat rooms, and blogs have made the transition from novel communication technologies to powerful tools for learning. Currently virtual worlds are undergoing the same transition. We argue that the next wave of innovation is at the level of the computer interface, and that mixed-reality environments offer important advantages over prior technologies. Thus, mixed reality is positioned to have a broad impact on the future of K-12 collaborative learning. We propose three design imperatives that arise from our ongoing work in this area grounded in research from the learning sciences and human-computer interaction. By way of example, we present one such platform, the Situated Multimedia Arts Learning Lab [SMALLab]. SMALLab is a mixed-reality environment that affords face-to-face interaction by colocated participants within a mediated space. We present a recent design experiment that involved the development of a new SMALLab learning scenario and a collaborative student participation framework for a 3-day intervention for 72 high school earth science students. We analyzed student and teacher exchanges from classroom sessions both during the intervention and during regular classroom instruction and found significant increases in the number of student-driven exchanges within SMALLab. We also found that students made significant achievement gains. We conclude that mixed reality can have a positive impact on collaborative learning and that it is poised for broad dissemination into mainstream K-12 contexts.

Keywords

K-12 learning Mixed reality Collaboration Teaching experiment Social computing Human-computer interaction Science learning 

Introduction

Digital technologies are rapidly changing as new modes of production, communication, and interaction continuously redefine our computing experiences. Users, designers, and developers from diverse communities including computer science research, industry, education, and entertainment are driving this evolution. Each technological innovation seemingly offers fresh opportunities for educators and students to teach and learn in new ways, but also poses new challenges to ensure that best practices are identified and implemented. One of the most exciting areas of recent innovation is at the level of the interface, as is most apparent in the commercial uses of the Nintendo DS and Wii. As these technologies mature to the point that they are ready to be widely disseminated into mainstream K-12 learning environments, we must develop appropriate design frameworks and an empirical base that is grounded in contemporary research from the learning sciences.

To address this need, our research team takes a holistic approach to realizing new frameworks for collaborative learning with interactive digital media. Specifically, we have implemented a new platform for embodied and mediated learning called the Situated Multimedia Arts Learning Lab [SMALLab]. We have worked in close collaboration with K-12 teachers and students to create interactive learning scenarios and associated curricula for content learning across the arts, humanities, and sciences. We have undertaken studies in informal and formal contexts including conventional school classrooms. Over the past 18 months we have been conducting a series of design experiments (A. L. Brown 1992; Cobb et al. 2003; Collins et al. 2004) to study the efficacy of SMALLab in a large urban high school in our region. Three design imperatives have emerged to guide our ongoing work which have implications for the design of future computer-supported collaborative learning environments and the field of digital learning in general. Specifically:
  • Design Imperative 1: Direct face-to-face interaction among colocated participants within the computationally mediated space should be cultivated

  • Design Imperative 2: Thought and action should be distributed across multiple participants through an active, generative process that unfolds in real time

  • Design Imperative 3: Immediate (spatial and temporal) consolidation of emergent conceptual models should follow the active learning process

The first imperative springs from our observations that students respond positively to direct face-to-face communication and the flexibility of digital tools. Nonetheless, many current platforms fail to integrate these two elements. For example, collaborative virtual worlds provide access to many distributed users, but they are often not designed to support direct interactions by colocated participants. Similarly, the traditional desktop computing platform itself is designed around a single-user interaction paradigm, and it often serves to isolate students from one another. We posit that despite the technological hurdles, hybrid physical-digital learning experiences should, when possible, be designed to integrate physical, social, and digital components into one coherent experience.

The second imperative flows from the first. Multiple participants should be empowered to directly participate in a shared learning experience. Individual actions should have immediately apparent consequences for the entire group to the extent that collaborating students can translate their collective thought into collective action in real time. This imperative requires computational platforms with multiple parallel interfaces, each with clearly defined, yet interdependent roles for participants.

The third imperative speaks to the ordering of learning activities. In SMALLab, students are continuously creating and testing new knowledge through hands-on activities. There is a need to regularly consolidate this new knowledge into robust conceptual models. This consolidation is best supported by class discussions mediated by a teacher or peer mentor. We posit that this consolidation is best served by reflection that immediately follows action. This should be immediate in time, while the experience is most fresh in memory, and in space (i.e., in the same physical space as the learning) so students can directly refer to the artifacts or outcomes of their shared experience.

These imperatives are grounded in our own research, and are further supported by the theoretical literature from the learning sciences regarding best practices in collaborative learning. In addition, they align with recent trends in Human-Computer Interaction (HCI) research that emphasize the importance of leveraging the natural social and physical capacities of users. In this article, we first present the theoretical basis for our work that draws from the learning sciences and HCI research. We next present SMALLab, a mixed-reality learning environment. We describe the context of a permanent SMALLab installation in a classroom at a large public high school in the southwest United States. We present a recent design experiment for earth science learning in SMALLab, detailing how our design imperatives ground the study. Our primary goal for the study is to better understand the impact of mixed-reality technology and our three design imperatives on collaborative learning in a classroom context. We present data that characterize the nature of student/teacher interactions while working in SMALLab. We summarize results from an invariant pre- and post-concept test that suggests that students made significant conceptual gains as a result of the experience. This study is not intended as a comprehensive validation of the impact of mixed-reality technology in the classroom. More extensive research is required to better understand the specific conditions and mechanisms that can lead to powerful mixed-reality learning experiences. Rather our aim is to demonstrate the potential for this technology and design imperatives to frame effective collaborative learning and to identify next steps in our longitudinal study.

Theoretical foundation

There has been extensive research regarding the efficacy of collaborative and cooperative learning (A. Brown and Palinscar 1989; Mesch et al. 1988; Slavin 1995, 1996). In comparison to individualistic and competitive approaches, there is overwhelming evidence that collaborative learning is superior in many respects. Collaborative learning generates significantly higher achievement outcomes, higher level reasoning, better retention, improved motivation, and better social skills (D. W. Johnson and Johnson 1984, 1989, 1991) than traditional didactics. Nonetheless, improved learning does not simply emerge from placing students in groups or providing tools that accommodate multiple users. Well-designed tools, activities, and mentoring must structure any collaborative environment to ensure collective thought and learning. Johnson and Johnson (D. W. Johnson and Johnson 1991) have identified five essential elements for successful cooperative and collaborative learning. Specifically, learning hinges on (1) positive interdependence, (2) promotive interaction, (3) individual accountability, (4) interpersonal and small-group skills, and (5) group processing. Our own design imperatives emphasize the need to bring together collaborative activity and reflective processes within one cohesive learning environment. Johnson and Johnson’s model offers a compelling framework for the design of collaborative learning environments and activities, and four of these tenets in particular have informed our design imperatives.

Our design imperatives are grounded in the concepts of positive interdependence, (2) promotive interaction, (3) individual accountability. Positive Interdependence: Johnson and Johnson describe multiple facets to positive interdependence, and most influential for our work is the notion of positive goal interdependence. When there is positive goal interdependence, students understand that they can only achieve their goals if all group members contribute toward a shared goal. Promotive Interaction is defined as “individuals encouraging and facilitating each other’s efforts to achieve, complete tasks, and produce in order to reach the group’s goals” (R. T. Johnson and Johnson 1994). Such interaction is accomplished through frequent face-to-face interchanges that foster social competencies that engage students’ full expressive capabilities including language, gesture, and oral communications. Individual Accountability means that each member of a collaborative team has a clearly defined responsibility and contribution to the common goal. This accountability can be structured into the design of collaborative tasks through formal and informal assessment tasks such as tests and oral questioning. Individual accountability is a necessary complement to group interdependence.

Our work is also grounded in group processing, which occurs when small groups and whole classes of students reflect upon the nature of their collaboration, discussing what worked and what needed improvement. This process encourages metacognitive thinking, promoting group trust, and providing positive reinforcement for successful work.

Interactive technologies for collaborative learning

With the advent of interactive technologies, educators and researchers have embraced each new wave of innovation, continuously revamping the nature of collaborative learning along the way. Nonetheless, each technology varies in its ability to support collaboration and collaborative learning.

For example, “conversational technologies” such as email, electronic bulletin boards, and chat rooms enabled new modes of collaboration that were not previously possible. They transformed the nature of distance learning (Harasim et al. 1995; Leh 2001). Similarly, Learning Management Systems (Blackboard Inc. 2008; Moodle 2008) can reach a broad community that is distributed across multiple sites, providing an important shared forum for knowledge exchange. More recently, Wiki technologies (Leuf and Cunningham 2001) have been shown to play an important role in supporting collaborative learning in online environments (Raitman et al. 2005; Rick et al. 2002). However, it is also evident that these mediated platforms are most effective when combined with face-to-face interaction (Asllani et al. 2008). These conversational technologies do not inherently support the face-to-face promotive interactions as described by Johnson and Johnson.

In recent years, massively multiplayer online games [MMOG’s] and multi-user virtual environments [MUVE’s] have been adapted to support collaborative learning. For example, CyberOne: Law in the Court of Public Opinion (Nesson et al. 2007), is a course recently offered by the Harvard Law School and the Harvard Extension School. The River City project (Dede and Ketelhut 2003; Dede et al. 2002; Ketelhut 2007; Nelson et al. 2007) is a MUVE that enables middle school children to learn about disease transmission and has been demonstrated to be an effective tool for learning (Dede and Ketelhut 2003; Dede et al. 2002). These technologies can support real-time interactions among large communities of learners, but unless purposefully designed, these open-ended virtual worlds do not necessarily facilitate structured mentoring or conceptual consolidation during the experience. While virtual worlds hold promise for effective communication and for collaboration that reaches beyond previous conversational technologies, there is still the risk of a gap between virtual learning experiences and those that are situated in the physical world.

Mixed-reality environments are interactive spaces that integrate computer-generated data with real-world components. They typically rely upon alternative display devices and tangible interaction devices. For example, flight simulators such as the Boeing 777 Cockpit Simulator, developed by NASA Ames’ Flight Deck Display Research Laboratory (W. Johnson and Battiste 2006), have been used for decades to safely train pilots in a manner that approximates real-world flight conditions. The MEDIATE environment was designed to foster a sense of agency and a capacity for creative expression in people on the autistic spectrum (EU Community Report 2004; Pares et al. 2004; Pares et al. 2005).

There are also semi-immersive configurations for mixed-reality that are particularly well suited to support so-called social computing experiences (Dourish 2001). For example, tabletop computing systems use video projectors with camera-based tracking of physical objects in a tabletop configuration (Ishii and Ullmer 1997). When coupled with 3D motion-capture technology and large-scale projections, this tabletop framework can scale to architectural spaces to form 3D multi-user environments that remain situated in everyday contexts. These are often referred to as perceptive spaces (Wren et al. 1999) or ambient display environments (Sukthankar 2005). Working in this context, we have developed a new mixed-reality environment called the Situated Multimedia Arts Learning Lab [SMALLab] (Birchfield et al. 2006).

Compared against previous technologies, mixed-reality environments offer several potential advantages for collaborative learning. First, mixed-reality environments support direct interactions by groups of colocated participants. They afford direct face-to-face social exchange between students. Second, mixed-reality environments provide a multitude of input devices that, unlike a traditional desktop computing platform, can be simultaneously manipulated by multiple users. The integration of these features is key. Mixed-reality environments afford face-to-face interactions between learners within a hands-on computational environment.

What is SMALLab?

SMALLab is semi-immersive mixed-reality learning environment developed by a collaborative team of media researchers—including the authors—from education, psychology, interactive media, computer science, and the arts. As shown in Fig. 1, SMALLab is physically open on all sides to the larger environment. Participants can freely enter and exit the space without the need for wearing specialized display or sensing devices such as head-mounted displays (HMD) or motion capture suits. Participants seated or standing around SMALLab can see and hear the dynamic media, and they can directly communicate with their peers who are within the active space. As such, SMALLab establishes a porous relationship between the mediated space and the larger physical learning environment.
Fig. 1

Students collaborating to construct a layer cake structure in SMALLab

SMALLab supports situated and embodied learning by empowering the physical body to function as an expressive interface (Birchfield et al. 2006). Within SMALLab, students use a set of “glowballs” and peripherals to interact in real time with each other and with dynamic visual, textual, physical, and sonic media through full body 3D movements and gestures. They can hear the sounds of an ocean or desert environment, see a structure take shape on the floor, and shake a physical device in their own hands to generate earthquakes.

SMALLab consists of the following sensing and feedback equipment: a six-element camera array for object tracking, a top-mounted video projector providing real-time visual feedback, four audio speakers for surround sound feedback, and an array of tracked physical objects (glowballs). A networked computing cluster with custom software drives the interactive system. In past work, our team has deployed SMALLab in a series of pilot programs that have reached over 25,000 learners through regional school and museum programs (Birchfield et al. 2008; Cuthbertson et al. 2007; Hatton et al. 2008).

Research context

In Summer 2007, we began a long-term partnership with a large urban high school in the greater Phoenix, Arizona metropolitan area. We have permanently installed SMALLab in a classroom and are working closely with teachers and students across the campus to design and deploy new learning scenarios. This site is typical of public schools in our region. The student demographic is ethnically and socioeconomically diverse: 50% Caucasian, 38% Hispanic, 6% Native American, 4% African American, 2% other. In this study, we are working with ninth-grade students and teachers from the school’s C.O.R.E. program for at-risk students. The C.O.R.E. program is a specialized “school within a school” with a dedicated faculty and administration. We are conducting a long-term design experiment at this site to support a K-12/university Professional Learning Community [PLC] (DuFour et al. 2006; Hord 1997; Wenger 1998). Four K-12 teachers and three university researchers meet once a week for two-hour sessions after school to devise new approaches to mixed-reality learning including curricula and assessment metrics. One teacher, referred to as our teacher-partner, implemented the intervention described below in his classroom.

Earth science learning

Geologic evolution is an important area of study for high school students because it provides a context for the exploration of systems thinking (Chen and Stroup 1993) that touches upon a wide array of earth science topics. Despite the nature of this complex, dynamic process, geologic evolution is usually studied in a very static manner. In a typical learning activity, students are provided with an image of the cross-section of the earth’s crust. Due to the layered structure of the rock formations, this is sometimes called a geologic layer cake. Students are asked to annotate the image by labeling the rock layer names, order the layers according to which was deposited first, and identify evidence of uplift and erosion (Lutgens et al. 2004). Students generally work individually, using reference materials to complete worksheet assignments. The classroom may come together to discuss the answers, but it is rare that students are offered a hands-on learning experience that captures the dynamic nature of the process of geologic evolution.

Our partner teacher, an experienced earth science teacher, identified a deficiency of the traditional instructional approach: When students do not actively engage geologic evolution as a time-based, generative process, they often fail to conceptualize the artifacts (i.e., cross-sections of the earth’s surface) as interconnected products of a complex, dynamic system. As a consequence, they struggle to develop a robust conceptual model during the learning process. For 6 weeks, we collaborated with members of the PLC, using the SMALLab authoring tools, to realize a new mixed-reality learning scenario to aid learning about geologic evolution in a new way. Our three design imperatives grounded the development process. Once complete, our teacher-partner led a 3-day teaching experiment with 72 of his ninth-grade earth science students from the C.O.R.E. program.

Research methodology: A design experiment

We are currently engaged in longitudinal study of the impact and efficacy of the SMALLab learning environment. This study is unfolding over the course of several years, and numerous interventions. We take a design experiment approach as proposed by Brown (1992), Collins et al. (2004) and Cobb et al. (2003). To that end, our process is iterative. Successive instances of teaching intervention are formulated, implemented, and studied. The design imperatives presented in this current study are grounded in our prior work and the theoretical literature. As proposed by diSessa (1991), these imperatives are of intermediate theoretical scope in that they draw from prior theory, and also from a set of experiences and studies within this particular school setting using the SMALLab learning environment. Nonetheless, they pose practical guidelines for the implementation of similar efforts in other contexts, yet we acknowledge that more work is necessary in order to extend these imperatives to a full-fledged learning theory.

In this particular study, we focus on the question of how the mixed-reality environment and our design imperatives impact student-driven collaborative learning in the classroom. More specifically, we expect to see that the nature of interaction and discourse in the SMALLab is increasingly student-to-student versus student-to-teacher when compared against regular instructional methods. As an ancillary question, we also want to better understand if and how our methodology is advancing student content knowledge, but we acknowledge that a more focused design is required to examine this question in detail.

Our collaborative design process yielded three parts: (1) a new mixed-reality learning scenario, (2) a student participation framework, and (3) a curriculum. We now describe each of these parts, discussing how each component reflects our design imperatives.

SMALLab learning scenario: Layer cake builders

As pictured in Fig. 1 above, during the learning activities, all students are co-present in the space. Figure 2 shows the visual scene that is projected onto the floor of SMALLab. Within the visual display, the center portion is the layer cake construction area where students deposit sediment layers and fossils. Along the edges, students see three sets of images. Along one edge they see depictions of depositional environments. Along another edge are images that represent sedimentary layers. Along a third edge they see an array of plant and animal images that represent the fossil record. Each image is an interactive element that can be selected by students and inserted into the layer cake structure. The images are iconic forms that students encounter in their earth science studies outside of SMALLab. A standard wireless gamepad controller is used to select a depositional environment from among the five options.
Fig. 2

Screenshot of projected layer cake builder scene with developing structure in the middle

When a student makes a selection, they will see the image of the environment and hear a corresponding ambient soundfile. For example, if a student selects the fast moving stream environment, students hear the sound of rushing water. One SMALLab glowball is used to grab a sediment layer from among five options and drop it onto the layer cake structure in the center of the space. This action will insert the layer into the layer cake structure at the level that corresponds with the current time period. A second glowball is used to grab a fossil from among ten options and drop it onto the structure. This action embeds the fossil in the current sediment layer. On the east side of the display, students see an interactive clock with geologic time advancing to increment each new period. Three buttons on a wireless pointer device are used to pause, play, and reset geologic time. A bar graph displays the current fault tension value in real time. Students use a Wii Remote game controller with embedded accelerometers, to generate fault events. The more vigorously that a user shakes the device, the more the fault tension will increase. Holding the device still will decrease the fault tension. When a tension threshold is exceeded, a fault event (i.e., earthquake) will occur, resulting in uplift in the layer cake structure. Fault events can be generated at any time during the building process. Subsequently erosion occurs to the uplifted portion of the structure.

Student participation framework

The process of constructing a layer cake involves four lead roles for students: (1) the depositional environment selector, (2) the sediment layer selector, (3) the fossil record selector, and (4) the fault event generator. The teacher typically assumes the role of geologic time controller. Participants interact simultaneously, each using a separate SMALLab interface (e.g., glowball Wii Remote, wireless gamepad) to accomplish their task. In this way, positive interdependence, promotive interaction, and individual accountability are encoded in the interactive technology itself. The computer interfaces define clear roles for multiple participants, and their success depends upon careful timing and execution of a collaborative choreography of action.

In the classroom, approximately fifteen to twenty students are divided into four teams of five or six students each. Three teams are in active rotation during the build process, such that they take turns serving as the action lead with each cycle of the geologic clock. These teams are the (1) depositional environment team and fault event team, (2) the sediment layer team, and (3) the fossil team. The remaining students constitute the evaluation team. These “evaluator” students are tasked to monitor the build process, record the activities of action leads, and to steer the discussion during the reflection process. Students from all teams are encouraged to verbally coach their teammates during the process.

There are two ways in which the build process is structured. In an exploratory build process, the interaction is largely open-ended. The teacher or depositional environment student leads the process, experimenting with the outcomes, but without a set of specific constraints. In the source matching process, the students can reference an existing layer cake structure as a script. Here, students must first analyze the structure to determine the sequence of sediment layers and uplift/erosion evidence to properly initiate the environments and fault events that could generate the structure. Only the few students on the “depositional environment” team had access to this source image. Thus all others’ actions were dependent on their decision making. At the end of this process, led by the “evaluation team,” all students reflect on the results of their reconstruction attempt by comparing the new structure to the source.

Pedagogy

We collaborated with our teacher-partner to design a curriculum that he implemented in a total of three, 45 minute class periods across three consecutive days. The curriculum is informed by our design imperatives, and is designed to foster student-centered, collaborative learning. Student activity is structured around a repeating cycle of activity→reflection. Simply put, students spend a period working in SMALLab to collaboratively build a layer cake structure. Then they spend a period of time reflecting on that process and evaluating the results. This cycle was repeated during each day of the curriculum, with new features and challenges added along the way.

From a modeling instruction perspective (Hestenes 1992, 1996), this activity cycle maps to students’ cognitive process of knowledge construction→consolidation. During the knowledge construction phase, students make the necessary observations to help them build a conceptual model about the elements, operations, relations, and rules that govern the underlying systems. They continuously form, test, and revise their conceptual models. Then there is knowledge consolidation process as students discuss their activities, analyze any faults in decision making, make sense of the various aspects of the layer cake structure, and challenge one another to justify their actions in the space. With each iteration of this cycle, new concepts are introduced and new knowledge is tested and consolidated, ultimately leading to a robust conceptual model of geologic evolution.

Table 1 outlines the curriculum and goals. SMALLab activities are matched with four primary learning goals that are central to high school earth science learning and are components of the State of Arizona Earth and Space Science Standards (Arizona Department of Education 2005). The principle of superposition dictates that older structures typically exist beneath younger structures on the surface of the Earth. The fossil record can provide clues regarding the age of these structures. Geologic evolution must be understood as a complex process that unfolds over time and is driven by interdependent relationships between surface conditions, fault events, and erosion forces.
Table 1

Layer Cake Builder curriculum and goals

 

Day 1—Introduction and exploration

Day 2—Exploratory construction

Day 3—Source-matching construction

SMALLab collaborative learning activity

•Introduction to SMALLab and the Layer Cake Builder scenario

•Generate fault events and discuss the effects including uplift and erosion

•Analyze source geologic structures and recreate them in SMALLab by enacting the process of evolution

•Select depositional environments

•Transition from an exploratory build process toward the source-structure matching process

•Apply sediment layers to the structure

•Add fossils to the sediment layers

Earth science concepts

•Principle of superposition

•Geologic evolution is a time-based process

•Geologic structures emerge from a complex process

•Significance of the fossil record

Data sources

Audio/visual documentation

Each SMALLab session was documented through audio and video recording. In addition, we made audio/video recordings of two 35 minute sessions of regular classroom instruction led by our teacher-partner. To assess the types of collaborative discourse in the classroom, we designed a coding rubric to classify the types of student and teacher utterances and code for their presence in the audio/video data. Our rubric allows for four types of utterances. Teacher-to-student utterances are either questions or statements from the teacher and directed to an individual student, a group of students, or the entire classroom. Student-to-teacher utterances are questions or responses that are directed to the teacher. Student-to-student utterances are comments, questions, or responses between two individual students. Student discussions occur between two students or among a group of students and contain a minimum of three exchanges. In Table 2, we omitted any utterances of a social or clearly off-topic nature. We selectively transcribed short episodes from the sessions in order to demonstrate the application of our coding framework and illustrate the nature of the discourse.

We coded two classes of students, each for the three consecutive days of the SMALLab treatment. For comparison, we coded two classes during regular Earth Science instruction. To triangulate the data analysis, two researchers with expertise in classroom instruction and qualitative research methods analyzed the video data. Inter-rater reliability was addressed as the researchers coded the video simultaneously, resolving any conflicts through face-to-face discussion until consensus was reached.

Geologic evolution concept test

To assess content learning gains, we collaborated with our teacher-partner to create a 10-item pencil and paper test to assess students’ knowledge of earth science topics relating to geologic evolution. To assess both the descriptive and explanatory aspects of students’ conceptual models, each item included a multiple-choice question followed by an open-format question asking students to articulate an explanation for their answer. The content for this test was drawn from topics covered during a typical geologic evolution curriculum and aligned with state and national science standards. All concepts were covered through traditional instructional methods in the weeks leading up to the experiment. At the time of the pre-test, students had studied the material to the full extent that would be typically expected in a ninth-grade earth science class. The SMALLab curriculum did not introduce any new concepts, but rather reinforced and reviewed previously studied topics. The test was administered 1 day before and several days after the SMALLab treatment. Every earth science class taught by our teacher-partner participated in the study.

Results

To address the question of how collaboration is impacted by SMALLab, we first present data pertaining to student and teacher interactions in SMALLab as compared against interactions during regular classroom instruction. Tables 2 and 3 contain transcriptions of episodes taken from day 1 and day 2 of the SMALLab treatment. In the right column, we annotate the types of discourse that we coded into our analysis. In the Day 1 transcription, the majority of exchanges here are between a single student and the teacher with little direct interaction among students. By Day 2 there is a clear shift. The interactions are more balanced with numerous student-led interactions and occasional interjections by the teacher. In Table 2, T denotes the teacher, and S denotes a student.
Table 2

Day 1 student and teacher interactions in SMALLab. T denotes the teacher, and S denotes a student

T: Alright, let’s go one more time.

Teacher-to-student

(Sound of rushing water. The students with the glowballs pick a sediment layer (sandstone) and a fossil (fish) and lay them into the scenario. This takes less than 10 s. When they are done the teacher pauses the geologic clock to engage them in reflection).

T: Alright, depositional environment—what are we looking at?

Teacher-to-student

Ss: A river.

Student-to-teacher

T: A river. Sandstone. Is that a reasonable choice for a type of rock that forms in a river? (Students shrug) Could be…is there any other types of rock over there that form in a river. Chuck. What’s another rock over there that might form in a river?

Teacher-to-student

C: In a river? I can’t find one…

Student-to-teacher

T: In a river. (there is a pause of several seconds)

Teacher-to-student

S: Conglomerate.

Student-to-teacher

T: Alright. Conglomerate is also an acceptable answer. Sandstone’s not a bad answer. Conglomerate is pretty good…big chunks of rock that wash down in the river. So, what kind of fossil did you put in?

Teacher-to-student

S: A fish.

Student-to-teacher

T: A fish, okay. A fish in a stream makes good sense. Let’s think about the fossils that we have in here. First we have a trilobite and then we had a jellyfish, then we had a fern and then we had a fish, alright? Is there anything wrong with the order of these animals so far?

Teacher-to-student

S: They’re aging.

Student-to-teacher

T: What do you mean, ‘they’re aging’?

Teacher-to-student

S: Evolution?

Student-to-teacher

T: It’s evolution so which ones should be the older fossils? (pause of several seconds)

Teacher-to-student

S: ...Trilobite?

Student-to-teacher

T: Trilobite in this case…why the trilobite in this case? How do we know the trilobite’s the oldest?

Teacher-to-student

S: Because it’s dead.

Student-to-teacher

T: Just look at the picture. How do we know that the trilobite is oldest?

Teacher-to-student

S: Because it’s on the bottom?

Student-to-teacher

T: We know that the oldest rocks are found…

Teacher-to-student

S: On the bottom.

Student-to-teacher

Table 3

Day 2 student and teacher interactions in SMALLab

SS2: Hurry!

Student-to-student

SS1: I picked one already.

Student-to-student

DES2: No, take your time...

Discussion

(The fossil student selects a fossil and lays it onto the center of the floor. The sediment student is still deliberating with her team on what layer to choose.)

T: (to the fossil student) Harry, you gotta wait until the rock goes in there.

Teacher-to-student

FS1: I gotta wait?

Student-to-teacher

T: You gotta have rock before you can have fossil, right?

Teacher-to-student

(The sediment student lays a limestone layer down. Students look on for a few seconds at the first layer they have built. The teacher tries to keep them mindful of the limited time.) You’ve got to pass it on. (Referring to the glowballs that fossil and sediment selectors must each pass to a teammate. They pass the balls and the new selectors move to make their choices. ) No. You’ve got to wait till the next cycle. (The clock chimes as the next cycle begins and the sound of falling water can be heard.) Alright. Now you can put in your choices.

(The sediment selector picks up coal and lays it into the layer cake and the fossil selector reaches for a trilobite until her teammate tries to stop her.)

Student-to-student

FS: No, no! (She selects a fern instead and places it into the coal layer.)

T: Okay. Pass the ball.

Teacher-to-student

SS: Shall I go now? (The clock chimes another cycle and a wind can be heard.) Oh, no.

Student-to-teacher

T: Not till it changes. Alright, this is better guys. You’re going a lot faster.

Teacher-to-student

(The sediment selector places a sandstone layer and the fossil student inspects her choices. Her teammate points to one that he thinks she should choose.)

Discussion

FS: That one right there. (She leans over and picks it up, and then places it carefully into the sandstone.)

T: Alright. Good so far. (He turns to the evaluation team.) Are you recording all the stuff they put down?

Teacher-to-student

(The clock chimes but no new environment is selected immediately.)

Student-to-student

FS: Come on dude.

SS: Which one do you choose? (They deliberate and finally select a picture of a warm shallow ocean, and the sediment and fossil selectors quickly place their choices into the layer cake and then hand off the glowball to a teammate. The clock chimes again and this time the depositional environment team promptly selects a river delta environment. Lapping water can be heard.)

Discussion

SS: Yeah. (He quickly picks up shale and places it into the layer cake, and the fossil selector places a fossil into it.)

Student-to-student

S: I want an earthquake. (The clock chimes but the depositional environment does not change).

Student-to-student

FS: Did you pick the same one?

Student-to-student

DES: Yeah.

Student-to-student

EvalS1: wait what was the second one?

Student-to-student

EvalS2: What did you just put?

Student-to-student

FS: A whale. (The clock chimes the next cycle and the sound of a whistling wind can be heard. Students place fossils without comment. The clock chimes and the sound of rain can be heard. The sediment and fossil selectors lay in their choices.)

Student-to-student

EvalS1: Alan, what did you pick?

Student-to-student

FS: I put a...um...(the clock ticks over again and the sound of a rushing stream can be heard. Students point to features on the layer cake as they lay in the next layer and fossil.)

Student-to-student

EvalS: What are you doing?

Student-to-student

FS: I put a fishy.

Student-to-student

(The clock chimes the end of the build cycle.)

During the next episode taken from Day 2, the students are engaged in a layer cake build cycle, working together with their teams to create two new layers on their structure. The teacher interjects occasionally to help the students when technical issues arise. In Table 3, SS denotes a sediment layer student. FS denotes a fossil layer student. DES denotes a depositional environment student. EvalS denotes a student on the evaluation team.

Table 4 summarizes the percentage breakdown and actual number of utterances between participants. Disc. Indicates a student discussion. SS is a student-to-student exchange while ST and TS are exchanges between students and our partner teacher.
Table 4

Summary of proportion and number of utterances between students and teachers

 

Regular Instruct.

SMALLab Total

SMALLab Hands-on Activity

SMALLab Reflection Activity

SMALLab Day 1

SMALLab Day 2

SMALLab Day 3

Disc.

>1% (2)

10% (332)

19% (210)

6% (119)

1% (3)

12% (97)

16% (108)

S→S

7% (47)

31% (1041)

62% (694)

16% (303)

14% (44)

38% (309)

25% (172)

S→T

46% (333)

27% (892)

4% (49)

38% (719)

40% (124)

23% (189)

29% (199)

T→S

47% (340)

32% (1059)

15% (165)

40% (755)

45% (139)

27% (220)

30% (202)

Figure 3 illustrates the proportion of types of utterances during a total of five SMALLab sessions and two regular instruction sessions, both conducted by our teacher-partner. We note the increased presence of student-to-student and discussion type interactions during SMALLab. Figure 4 documents the types of interactions over the course of the 3-day treatment. We observe that there is a dramatic increase in the number of student-to-student and discussion interactions from Day 1 to Day 2. Finally, Fig. 5 compares the types of interactions during activity and reflection episodes on Day 2 and Day 3 of the SMALLab treatment. Given the introductory nature of Day 1 of the curriculum, clearly defined activity and reflection cycles were not present, and thus, we have not included Day 1 in this analysis. We note the increase in student-to-student and discussion interactions while students are engaged in the activity process.
Fig. 3

Proportion of utterances during SMALLab learning and regular instruction

Fig. 4

Proportion of utterances for each of the 3 days during the SMALLab intervention

Fig. 5

Proportion of utterances comparing interaction during a build activity vs. a reflection activity in SMALLab

To collect preliminary evidence of student gains, we collected pre- and post-treatment scores using an invariant paper and pencil test with items derived from our teacher-partner’s existing curriculum materials. Mean test scores were analyzed for the 72 participating students and summarized in Table 5. The question items were broken into two categories for the multiple-choice items and the free-response explanation items. We computed a percentage increase and the Hake gain for each category—the actual percent gain divided by the maximum possible gain (Hake 1998).
Table 5

Summary of mean scores and gains on pre- and post-treatment concept test

 

Mean scores

Multiple-choice test items (n = 72)

Pre-treatment multiple choice average score

6.82

Post-treatment multiple choice average score

8.36

% increase

23%

Hake gain

48%

Free-response justifications (n = 72)

Pre-treatment explanation average score

3.68

Post-treatment explanation average score

5.17

% increase

40%

Hake gain

24%

Participating students achieved a 22.6% overall percent increase in their multiple-choice question scores, a 48% Hake gain (p < 0.00002, r = 0.20, n = 72, std = 1.9). They achieved a 40.4% overall percent increase in their explanation scores, a 23.5% Hake gain (p < 0.000003, r = 0.60, n = 72, std = 2.8). Again, the assessment of student learning gains is not the focus of this study, and extensive further study is required before any claims can be made regarding the potential impact of SMALLab in this regard. Nonetheless, these results are reported to provide a more complete picture of the learning experience and outcomes.

Discussion and limitations

Figures 3 to 5 illustrate the proportion of student and teacher exchanges during the experiment. Student-to-student and student discussion exchanges are increased in SMALLab by 33% when compared against regular instruction in a conventional classroom. We attribute this to several factors. First, as shown in Fig. 2, when working in SMALLab, students are physically arranged so that the entire class can see and engage one another. This is a manifestation of Design Imperative #1 such that students and teachers have equal access to one another, learning tools, artifacts, and representations while the teacher is removed from a position of central focus. This is in contrast to a typical classroom configuration where students are seated at individual desks, facing forward toward with the teacher in front. Second, in accordance with Design Imperative #2, the technology, curriculum, and participation framework are aligned to provide clearly defined roles for students that are mutually dependent on one another. Students must work well with peers in their teams and in other teams in order to successfully complete the layer cake build task, all the while under a constant time pressure that the game-like scenario provides. We even observed that on Day 3 in particular, many utterances by the teacher-partner were left unanswered by students who were fully engaged in the simulation and peer collaboration. Finally, in accordance with our Design Imperative #3, the learning activities focus on the alternation of hands-on layer cake build activities, followed immediately by a process of whole-class discussion and reflection on the process. Our teacher-partner observed his students readily engaged in the process of describing and defending their decision making to an extent that he had rarely seen in other classroom activities. We conclude that this is a result of the structuring and immediate relevance of the ensuing discussions.

Despite this encouraging trend, Fig. 5 reveals a reduced level of student-to-student exchanges and discussion during the reflection activity when compared to the build activity. This outcome is not unexpected given that the reflection activity is more akin to a conventional instructional paradigm than the build activity. Our teacher-partner noted that he felt his students still required substantial guidance to structure their reflection discussions during the reflection activity as unfortunately this is not a skill that students in the C.O.R.E. program often practice outside of SMALLab. Nonetheless, the design of effective strategies to retain a student-centered focus during reflection activities has emerged as a point of emphasis within the context of our professional learning community.

In Fig. 4 we see a double-digit rise in student-led exchanges from the beginning of the intervention to the end. This is an expected outcome in the sense that the first day was introductory in nature, and students required a certain amount of direct instruction to simply operate the input devices and understand the learning environment and scenario. By Day 2, students and teacher were all more comfortable with their roles. We account for the marked change from Day 1 to Day 2 in that students were able to transition away from the procedural aspects of constructing a layer cake in SMALLab toward a higher order conceptual understanding of the interdependent processes that are at work in geologic evolution. In addition, our teacher-partner consciously structured his participation in a manner that he hoped to gradually withdraw into the background, while allowing the students themselves to take ownership of the process. The data reflects the success of this design.

One limitation of the current study is that we were not able to compare the nature of collaboration in SMALLab against other types of pedagogy. We would like to better understand the nature of collaborative learning in SMALLab as compared to learning in small groups around similar problem-based tasks. We intend to explore these questions as a next step in our research.

Pre- and post-test data regarding student achievement suggests that perhaps the SMALLab experience led to improved understanding by participating students. Given the size of these gains, especially in light of the fact that these students had already studied these earth science topics in a conventional instructional paradigm prior to the intervention, we speculate that SMALLab learning might offer an advantage over other methods. However, extensive research is required to better understand the potential causes and implications of these gains. A particular limitation of this study is that we were not able to identify a suitable control group for comparison. In our future work, we intend to use a wait list control group design to further explore questions regarding the impact of our design imperatives and the SMALLab environment on student achievement.

Conclusions

Mixed-reality technologies can offer a number of advantages over other tools for collaborative learning. However, it must be used in the context of well-designed learning scenarios and teaching practices. From a theoretical perspective, there is extensive evidence that collaborative learning can be a powerful approach to learning, but it must be properly structured in order to achieve the greatest benefit. We are working toward an integration of these technological and theoretical perspectives to produce new frameworks for computer-supported collaborative learning. Based on our observations and empirical evidence collected in a series of pilot programs, we propose three design imperatives to structure computer-supported collaborative learning in mixed reality: (1) direct face-to-face interaction among colocated participants within the computationally mediated space should be cultivated, (2) thought and action should be distributed across multiple participants through an active, generative process that unfolds in real time, and (3) immediate (spatial and temporal) consolidation of conceptual models should follow the active learning process.

Through the realization of a longitudinal design experiment, we are exploring the feasibility of the use of mixed-reality technologies in a mainstream school-based context. We are expanding the program to include additional content areas and participants. We are focusing our evaluation methodologies to more specifically address questions regarding collaborative learning and student achievement. Taken in whole, this early data demonstrates that despite the additional logistical and financial hurdles, when properly designed and implemented, mixed-reality technology can be an effective and viable platform for collaborative learning in a mainstream school context.

Notes

Acknowledgements

We gratefully acknowledge that these materials document work supported by the MacArthur Foundation under the grant titled Gaming SMALLab: A game-like approach to embodied learning and by the National Science Foundation under CISE Infrastructure Grant No. 0403428 and IGERT Grant No. 0504647. We extend our gratitude to the students, teachers, and staff of Coronado High School.

References

  1. Arizona Department of Education. (2005). Arizona Academic Standards: Science Strand 6—Earth and Space Science, from http://www.ade.az.gov/standards/science/downloads/strand6.pdf.
  2. Asllani, A., Ettkin, L., & Somasundar, A. (2008). Sharing knowledge with conversational technologies: Web logs versus discussion boards. International Journal of Information Technology and Management, 7(2), 217–230.CrossRefGoogle Scholar
  3. Birchfield, D., Ciufo, T., Minyard, G., Qian, G., Savenye, W., Sundaram, H., et al. (2006). SMALLab: A mediated platform for education. Paper presented at the ACM SIGGRAPH, Boston, MA.Google Scholar
  4. Birchfield, D., Mechtley, B., Hatton, S., & Thornburg, H. (2008). Mixed-reality learning in the art museum context. Paper presented at the ACM SIG Multimedia, Vancouver, BC.Google Scholar
  5. Blackboard Inc. (2008). Blackboard: Educate, innovate, everywhere. Retrieved 07/25/2008, from http://www.blackboard.com/us/index.bbb.
  6. Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141–178.CrossRefGoogle Scholar
  7. Brown, A., & Palinscar, A. (1989). Guided, cooperative learning and individual knowledge acquisition. In L. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 393–452). Hillsdale, NJ: Erlbaum.Google Scholar
  8. Chen, D., & Stroup, W. (1993). General systems theory: Toward a conceptual framework for science and technology education for all. Journal for Science Education and Technology, 2(3), 447–459.CrossRefGoogle Scholar
  9. Cobb, P., diSessa, A., Lehrer, R., & Shauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13.CrossRefGoogle Scholar
  10. Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15–42.CrossRefGoogle Scholar
  11. Cuthbertson, A., Hatton, S., Minyard, G., Piver, H., Todd, C., & Birchfield, D. (2007). Mediated education in a creative arts context: Research and practice at Whittier Elementary School. Paper presented at the 6th International Conference on Interaction Design and Children.Google Scholar
  12. Dede, C., & Ketelhut, D. J. (2003). Designing for motivation and usability in a museum-based multi-user virtual environment. Paper presented at the American Educational Research Association Conference, Chicago, IL.Google Scholar
  13. Dede, C., Ketelhut, D. J., & Ruess, K. (2002). Motivation, usability and learning outcomes in a prototype museum-based multi-user virtual environment. Paper presented at the 5th International Conference of the Learning Sciences, Seattle, WA.Google Scholar
  14. diSessa, A. (1991). Local sciences: Viewing the design of human-computer systems as cognitive science. In J. M. Carroll (Ed.), Designing interaction: Psychology at the human-computer interface (pp. 162–202). Cambridge University Press.Google Scholar
  15. Dourish, P. (2001). Where the action is: The foundations of embodied interaction. Cambridge, MA; London: MIT Press.Google Scholar
  16. DuFour, R., DuFour, R., Eaker, R., & Many, T. (2006). Learning by doing: A handbook for professional learning commuities at work. Bloomington, IN: Solution Tree.Google Scholar
  17. EU Community Report, I. S. T. I. P. (2004). A multisensory environment design for an interface between autistic and typical expressiveness.Google Scholar
  18. Hake, R. R. (1998). Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66, 64–74.CrossRefGoogle Scholar
  19. Harasim, L., Hiltz, S. R., Teles, L., & Turoff, M. (1995). Learning networks: A field guide to teaching and learning on-line. Cambridge, MA: MIT Press.Google Scholar
  20. Hatton, S., Birchfield, D., & Megowan-Romanowicz, M. C. (2008). Learning metaphor through mixed-reality game design and game play. Paper presented at the AMC Sandbox.Google Scholar
  21. Hestenes, D. (1992). Modeling games in the Newtonian world. American Journal of Physics, 60, 732–748.CrossRefGoogle Scholar
  22. Hestenes, D. (1996). Modeling methodology for physics teachers. Paper presented at the International Conference on Undergraduate Physics, College Park, MD.Google Scholar
  23. Hord, S. (1997). Professional learning communities: Communities of continuous inquiry and improvement. Southwest Educational Development Laboratory.Google Scholar
  24. Ishii, H., & Ullmer, B. (1997). Tangible bits: Towards seamless interfaces between people, bits and atoms. Paper presented at the SIGCHI Conference on Human Factors in Computing Systems, Atlanta, Georgia.Google Scholar
  25. Johnson, D. W., & Johnson, R. T. (1984). Cooperative learning. New Brighton, MN: Interaction Book Co.Google Scholar
  26. Johnson, D. W., & Johnson, R. T. (1989). Cooperation and competition: Theory and research. Edina, MN: Interaction Book Company.Google Scholar
  27. Johnson, D. W., & Johnson, H. (1991). Learning together and alone: Cooperation, competition, and individualization. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
  28. Johnson, R. T., & Johnson, D. W. (1994). An overview of cooperative learning. In J. Thousand, A. Villa & A. Nevin (Eds.), Creativity and collaborative learning. Baltimore: Brookes Press.Google Scholar
  29. Johnson, W., & Battiste, V. (2006). NASA Ames Flight Deck Display Research Laboratory: Boeing 777 Cockpit Simulator, 2008, from http://human-factors.arc.nasa.gov/ihh/cdti/777FlightSim.html.
  30. Ketelhut, D. J. (2007). The impact of student self-efficacy on scientific inquiry skills: An exploratory investigation in River City, a multi-user virtual environment. The Journal of Science Education and Technology, 16(1), 99–111.CrossRefGoogle Scholar
  31. Leh, A. (2001). Computer-mediated communication and social presence in a distance learning environment. International Journal of Educational Telecommunications, 7(2), 109–128.Google Scholar
  32. Leuf, B., & Cunningham, W. (2001). The wiki way: Quick collaboration on the Web. Boston: Addison-Wesley.Google Scholar
  33. Lutgens, F., Tarbuck, E., & Tasa, D. (2004). Foundations of earth science (4th ed.). Upper Saddle River, NJ: Prentice Hall.Google Scholar
  34. Mesch, U., Johnson, D. W., & Johnson, R. T. (1988). Impact of positive interdependence and academic group contingencies on achievement. Journal of Social Psychology, 28, 845–852.Google Scholar
  35. Moodle. (2008). Moodle—A free, open source course management system for online learning. Retrieved 07/25/2008, from http://moodle.org.
  36. Nelson, B., Ketelhut, D. J., Clarke, J., Dieterle, E., Dede, C., & Erlandson, B. (2007). Robust design strategies for scaling educational innovations: The River City MUVE case study. The Educational Design and Use of Computer Simulation.Google Scholar
  37. Nesson, C., Nesson, R., & Koo, G. (Producer). (2007). CyberOne: Law in the Court of Public Opinion. Podcast retrieved from http://blogs.law.harvard.edu/cyberone/.
  38. Pares, N., Carreras, A., Durany, J., Ferrer, J., Freixa, P., Gomez, D., et al. (2004). MEDIATE: An interactive multisensory environment for children with severe autism and no verbal communication. 3rd International Workshop on Virtual Rehabilitation (IWVR’04).Google Scholar
  39. Pares, N., Masri, P., vanWolferen, G., & Creed, C. (2005). Achieving dialogue with children with severe autism in an adaptive multisensory interaction: The MEDIATE project. IEEE Transactions on Visualization and Computer Graphics, 11(6), 734–743.CrossRefGoogle Scholar
  40. Raitman, R., Augar, N., & Zhou, W. (2005). Employing wikis for online collaboration in the e-learning environment: Case study. Paper presented at the Third International Conference on Information Technology and Applications, Sydney, Australia.Google Scholar
  41. Rick, J., Guzdial, M., Carroll, K., Holloway-Attaway, L., & Walker, B. (2002). Collaborative learning at low cost: CoWeb use in English composition. Paper presented at the Computer Supported Collaborative Learning Conference, Hillsdale, NJ.Google Scholar
  42. Slavin, R. (1995). Cooperative learning: Theory research, and practice. Boston: Allyn & Bacon.Google Scholar
  43. Slavin, R. (1996). Research on cooperative learning and achievement: What we know, what we need to know. Contemporary Educational Psychology, 21, 43–69.CrossRefGoogle Scholar
  44. Sukthankar, R. (2005). Towards ambient projection for intelligent environments. Paper presented at the Computer Vision for Interactive and Intelligent Environment, Madonna di Campiglio, Italy.Google Scholar
  45. Wenger, E. (1998). Communities of practice: Learning, meaning and identity. Cambridge: Cambridge University Press.Google Scholar
  46. Wren, C. R., Basu, S., Sparacino, F., & Pentland, A. (1999). Combining audio and video in perceptive spaces. Paper presented at the Managing Interactions in Smart Environments, Dublin, Ireland.Google Scholar

Copyright information

© International Society of the Learning Sciences, Inc.; Springer Science + Business Media, LLC 2009

Authors and Affiliations

  1. 1.School of Arts, Media and EngineeringArizona State UniversityTempeUSA
  2. 2.School of Educational Innovation and Teacher PreparationArizona State UniversityTempeUSA

Personalised recommendations