CSCL artifacts

Artifacts are central to CSCL. In a typical CSCL setting, artifacts can play multiple pivotal roles:

• Technological artifacts like web apps can provide communication media that support collaboration.

• Further, they may structure the representations that groups of students use in building their intersubjective knowledge, making it visible, shared and persistent.

• Instructional artifacts presenting domain topics, lessons, guidance, scaffolding or scripting may supply motivation and direction to the collaborative efforts.

• In addition, the group efforts may be oriented toward co-construction of an artifact: a work of art, a design, a plan, a report or a summary text.

• Finally, the CSCL process may result in such a knowledge artifact, as the group product.

CSCL researchers can study the use of artifacts by student groups to see how collaborative learning takes place and to judge its success. They can observe how artifacts mediate the communication—whether synchronous, asynchronous, face-to-face, textual and so on. They can see how the groups enact the representational guidance and use it to structure their shared understanding of their goals and the co-construction of their knowledge. They can observe the evolution of collaboratively generated artifacts to track processes of productive group interaction.

Moreover, CSCL researchers themselves make use of artifacts and produce their own knowledge artifacts. Like most scientific fields (Latour and Woolgar 1979), CSCL research can be conceived as an international effort to generate a growing corpus of academic textual inscriptions, specifically discussing the use of CSCL artifacts. While CSCL research involves many activities (designing innovative technologies, intervening in classrooms, analyzing data, teaching theories, etc.), the lasting product of this work is primarily publications in conference proceedings, specialized journals and edited-book chapters.

In this issue of ijCSCL, we present two papers that look in detail at some of the roles of artifacts in CSCL settings. We also present two papers that mine the corpus of CSCL publication artifacts for indications of the nature of our research field.

Artifacts in the CSCL paradigm

In the introduction to his classic volume of CSCL studies, Koschmann (1996) proclaimed that CSCL provided a new paradigm of research on instructional technology. A few years later, he clarified what he thought distinguished CSCL from earlier approaches:

Traditional theories of learning treat learning as a concealed and inferred process, something that “takes place inside the learner and only inside the learner” (Simon 2001, p. 210). CSCL research has the advantage of studying learning in settings in which learning is observably and accountably embedded in collaborative activity. Our concern, therefore, is with the unfolding process of meaning making within these settings, not so-called “learning outcomes.” It is in this way that CSCL research represents a distinctive paradigm within IT. By this standard, a study that attempted to explicate how learners jointly accomplished some form of new learning would be a case of CSCL research, even if they were working in a setting that did not involve technological augmentation. On the other hand, a study that measured the effects of introducing some sort of CSCL application on learning (defined in traditional ways) would not. (Koschmann 2001, p. 19)

Then, in his keynote talk at CSCL 2002, Koschmann proposed a definition of a paradigm of CSCL. He said, “CSCL is a field of study centrally concerned with meaning and the practices of meaning making in the context of joint activity, and the ways in which these practices are mediated through designed artifacts” (Koschmann 2002, p.17).

Anticipating the findings of the papers in this issue, we might modify Koschmann’s definition of CSCL as follows:

CSCL is a research community that produces papers centrally concerned with intersubjective meaning and the practices of meaning making as joint activity, and how best to design CSCL artifacts to mediate these practices.

This definition incorporates a number of points:

• CSCL research is not defined by a set of fixed attributes, but by the work of an international community, whose focus shifts over time as its established understanding evolves.

• The nature of the community is externalized in the corpus of its publications.

• An “artifact” is defined as a physical object created by people and embodying human meaning—thereby overcoming the old distinction between what is in the mind vs. in the world.

• The meaning may be projected by the original designers of the artifact into the form of the artifact, but it must also be enacted by the users of the artifact.

• Creation of meaning is a social process, in which the meaning is necessarily intersubjectively defined.

• Meaning is not created in the mental processes of an individual, but in joint activity, typically in accordance with established social practices. (This is why learning in CSCL settings can be observable and understandable to researchers—without looking inside the learners.)

• CSCL research is both a theoretical enterprise, concerned with how groups make meaning, and a design endeavor, concerned with how to design artifacts of collaboration media, representational guidance, group interaction and pedagogical approaches to promote collaborative learning.

In an excerpt quoted in last issue’s editorial, Dillenbourg, Baker, Blaye and O’Malley (1996) concluded “that the group itself has become the unit of analysis” and that CSCL requires a “more process-oriented account” of how interaction is mediated—in contrast to the traditional orientation to learning outcomes of individual students. In Koschmann’s example of a seminal analysis in this paradigm, Teasley and Roschelle (1993), pursued a specific version of this question by asking how dyads of students created a joint problem space around a computer representation of physics concepts. In their often cited introduction to CSCL, Stahl, Koschmann and Suthers (2006) characterize CSCL as involving: (i) meaning making (social constructivism as opposed to positivist realism), (ii) examination at the group unit of analysis (rather than exclusive focus on or reduction to the individual mind) and (iii) investigation of group processes (instead of just measuring pre/post learning outcomes). In other words, a paradigm-shaping research question for CSCL would treat learning as essentially an intersubjective, interactional process, and would study it by investigating the dynamic developmental processes through which individual, small-group and community cognitive practices emerge. We can summarize this by saying that a post-cognitive approach to CSCL should be: dialectical (the problem and technological artifacts that mediate the group interaction must be seen to be enacted by that interaction), interactional (group-cognitive phenomena should not be reduced to individual mental structures) and dynamic (there should be a concern with developmental processes at multiple levels).

Recent editorial introductions to ijCSCL (Stahl 2012; 2013a; 2013b; Stahl, Cress, Law and Ludvigsen, 2014), have suggested that artifacts can serve to connect different levels of analysis in CSCL settings, providing a synthesizing role within CSCL theory. Interactional artifacts—like the representation of acceleration in the Envisioning Machine (Teasley and Roschelle 1993) or a scientific question in Knowledge Forum (Scardamalia and Bereiter 1996)—can mediate between the perceptions of an individual student, the work of a small group and the accepted discourse of a scientific community.

Having noted some ways in which artifacts are central to CSCL, let us now see how the papers in this issue advance our understanding of CSCL artifacts.

Artifacts and other layers in CSCL

In her analysis of effective collaboration, Crina I. Damşa distinguishes three inter-connected layers of small-group learning: productive interaction, knowledge-object development and shared epistemic agency. Through her focus on how a group is oriented to co-constructing its final artifact, she sheds new light on these three distinct phenomena as components of computer-supported collaborative learning.

The group interaction is analyzed in terms of its potential productivity. This provides a framework for analyzing and even judging the interaction: to what extent is it aimed at effectively producing the group’s knowledge object? One can then analyze how the interaction works to contribute to evolving the discussion and the successive drafts of the report that the team must present. In the past, analysis of interaction tried to discover generic discourse forms or to code the discourse for various categories of utterances. Here, the objective of producing a report provides a specific, grounded basis for the analytic approach. Accordingly, productive interaction is interaction that (potentially at least) contributes to the group production of its target object.

The objective of the interaction thereby also defines the second level, the knowledge object that provides the group’s goal. This objective is largely proposed from outside the group interaction. In the particular case analyzed, it is provided by a university course and the companies associated with the course. However, it is up to the student groups to interpret, refine or enact precisely what their problem or goal is. As they work on their knowledge object, they learn more about it and clarify just what problem they are pursuing. Thereby, the knowledge artifact mediates the interaction even as it emerges as a product of that interaction. Notably, the knowledge is not a mental phenomenon, but a material artifact, physically and jointly observable as persistent text, which, however, evolves over time through drafts.

Setting a goal, deciding how to pursue it, keeping on track and concluding when the goal has been accomplished is part of what is known as agency. The notion of shared epistemic agency is perhaps the most interesting of the layers discussed here. As reviewed by Emirbayer and Mische (1998), agency has historically been associated with individuals. However, the definition they develop could apply equally to group agency or even community agency. Given a dynamic notion of group agency, one can analyze in excerpts of productive interaction exactly how a group negotiates, reflects upon and carries out its action objectives. This does not necessarily involve a rationalist sub-goal hierarchy, but can include group members orienting the group in various subtle ways toward projected joint goals and reminding the group of objectives articulated in the past. Like knowledge, agency is here conceived as observable in the interaction, rather than as a hypothetical psychological state. Furthermore, the concept of agency can be applied to artifacts as well. Designed artifacts exercise a referred agency, designed into the form of the artifact by the intentions of the designer—and visible in that form. A software developer embeds certain goals in the software, which determine to some extent how it can and cannot be used. Thereby, the interaction that takes place can be viewed as an inter-action among many sources of agency, coming together in a concrete and unique situation formed by that agentic confluence.

The document drafts produced by the groups are conceptualized here as knowledge objects. The study concludes that increases in the discourse about these textual objects lead to improvements in learning or better knowledge building. However, the study does not reflect upon the advantages of the affordances of literary text over verbal discourse for the development of complex ideas. The students’ texts are not simply “objects,” but are shared inscriptions. Written language has powerful knowledge-building affordances, as seen in the difference between oral and literate cultures (Ong 1998). Issues of idea organization (outline), structure (sentence and paragraph) and conceptualization (choice of words) become explicit in the translation from verbal discourse to report construction. Persistence, shared attention, longer sequences and other affordances of literary texts make huge differences as vague objectives become refined artifacts. Materialization and objectification facilitate co-authors building on each other through multiple drafts and edits. The history of CSCL began with attempts to support group literacy (Stahl et al. 2006), yet current CSCL research often ignores the design of technological media for supporting joint writing.

Bringing artifacts into use

Technological artifacts are not simply present to users with fixed attributes; they must be enacted by the users through specific ways that the users discover for making use of them. This notion is forcefully put forward by Maarten Overdijk, Wouter van Diggelen, Jerry Andriessen and Paul A. Kirschner in their analysis of a dyad of students using a form of planning software.

There are many similarities between the first two studies in this issue. They both adopt and advance the dialectical, interactional and dynamic paradigm of CSCL research. They explore the processes of enacting by observing the details of interaction and tracking processes of group practices. The second study builds on Rabardel’s notion of instrumental genesis (Lonchamp 2012b; Rabardel and Beguin 2005; Ritella and Hakkarainen 2012) by developing an analysis of the dialectic of resistance and accommodation driving the enactment of designed artifacts by users. There are also striking contrasts between the two studies, notably in the very different level of maturity of the students and the span of time given for their interactions.

Of particular methodological interest is the way the second paper analyzes enactment. It compares the practices of the group before and after the artifact is introduced. It adopts a design-based research perspective by looking at successive iterations of an instructional intervention: before a technological artifact is introduced and afterwards. In this case, the artifact is a software medium for inscribing steps in a planning process. Initially, the student dyads simply transferred steps from their instructions into a generic spreadsheet. Given the introduced software support, they then figured out a way to arrange the given steps within the new format.

One can well ask the same question here as with the first study: How is the medium for co-constructing knowledge objects designed to support the collaborative knowledge-building goals? Here, a sophisticated application is suddenly inserted into the instructional setting. We are not told where this medium came from or what its design objectives were. What were the referred intentions contributing to the resistance of its material agency? It is not presented as part of a design-based research integration of iterations of software development with classroom trials, data analysis and theory refinement. We are told that the student pair “does not engage in a real problem representation. Instead of simulating, evaluating and revising planning decisions, they stick with the specification and following the order of the assignments as they are presented in the syllabus. To them, this is an acceptable solution to the planning problem.” Presumably, the software artifact was intended to guide the student groups to learn new planning practices. The student practices changed, but not necessarily in the ways intended by the intervention facilitators or the artifact designers. What are the implications of the analysis of how the students enacted the software for redesigning that medium of inscription?

Coding CSCL journal artifacts

Given the two papers on the theory of artifacts in CSCL, we might wonder if they represent a latest stage in a trend within the CSCL corpus of publications. Are they part of a paradigm shift within the learning sciences that was anticipated in the mid-1990s, but has been slow to materialize, or are they simply examples of one approach among many unrelated methodological fashions? The very first article published in ijCSCL, (Kienle and Wessner 2006), aimed to display major trends in the field through analysis of CSCL publications. More recently, Lonchamp (2012a) tried to map changes in the field through computational analysis of ijCSCL content. Now, we have two new energetic examinations of the CSCL literature.

The first of these is a report from an on-going effort by Heisawn Jeong, Cindy E. Hmelo-Silver and Yawen Yu. It investigates CSCL empirical articles from 2005 to 2009 in seven journals that publish CSCL studies. The report clearly represents significant work gathering, filtering and analyzing publications. It applies a large number of interesting analyses, combining automated and manual examination in order to explore various relevant issues. It dissects and categorizes hundreds of empirical CSCL papers along multiple dimensions: (1) research design, (2) setting, (3) data and (4) analysis.

The discussion in the report reflects a deep understanding of relevant issues for analyzing the CSCL corpus and attempts to avoid potential biases. Yet, when one views the specific findings of the analysis or even scans the list of papers selected as representative of CSCL journal articles during the time period, one is struck by the low number of publications by well known CSCL researchers and of papers that show up in lists of most often cited or most frequently downloaded articles. Are the items that made it through the selection and filtering process the most influential CSCL publications of the period, or are they, rather, primarily archival reports of uncontroversial experimental confirmations of generally accepted findings?

For instance, given the papers in the present issue of ijCSCL, which of these papers would be included in the final list if the study were extended to 2014? Would it be clear within the selection method that the first two papers report stages of larger design-based research efforts and that they systematically focus on practices and group processes rather than on individual learning outcomes? More abstractly, they can be considered to be exemplars of research approaches in a distinctively CSCL paradigm that is post-cognitive. One would like to know if there has been a general paradigm shift in this direction from 2005 to the present. What kind of analysis of the literature would be necessary to determine this?

The findings of the reported analysis suggest that post-cognitive approaches were not prominent in the selected sample. First, the majority of data sources for the selected articles were questionnaires—sources of self-reports rather than knowledge objects or discourse interactions. (In recent years, both ijCSCL and JLS automatically reject submissions that rely solely on questionnaire data.) Second, the dominant analysis method other than statistical is code-and-count, an approach that usually systematically eliminates the sequentiality of interaction, which is necessary for analyzing group-level interaction and practices. Third, many of the remaining qualitative or mixed-method studies rely on “loosely defined” narrative, suggesting that their authors do not adopt a rigorous method for analyzing group practices or processes. Fourth, the technologies used in the CSCL settings were not analyzed, so there is no way to know if conventional commercial technologies like Blackboard or Facebook were used or if innovative technologies were being explored. Fifth, the role of theory is not well defined: are the theories just given an obligatory mention in a paper or does the study contribute to expanding the theory, for instance by fleshing out new concepts—like group agency or artifact resistance?

Of course, this examination of methodology and theoretical frameworks during 2005–2009 does not claim to answer all the possible and interesting questions. It defines its goals and reflects carefully on its limitations. However, the larger questions arise insistently for our community. Studies of the CSCL corpus like these attempt to implement an objective approach to selection and analysis. The question is then whether the self-imposed limitations of the objective approach themselves impose bias. Should one restrict oneself to journal articles as the most highly ranked formats for research? Is it possible that the more innovative and influential inscriptions are presented at conferences and workshops? Or might books—both edited volumes and monographs—still provide a medium for more significant statements than a journal article, limited to a single intervention and one focal finding? In particular, does the selection from seven highly ranked journals unavoidably entail biases based on the commitments of those particular journals within current academic practices and institutional pressures?

Co-citing CSCL journal artifacts

The final contribution in this issue takes up some of the challenges just mentioned. The presentation by Kai-Yu Tang, Chin-Chung Tsai and Tzu-Chiang Lin expands the collection to conference papers and book chapters as well as journal articles. It also brings the coverage up to the present. Furthermore, it provides a useful triangulation by approaching the analysis of CSCL literature with a quite different, but still objective, method. It applies the well-established bibliometric technique of co-citation analysis, and then clusters the results using social network analysis (SNA).

Academic papers always reference previous publications, which they build upon. Thus, it is natural to categorize articles together that reference or are referenced by each other. A more sophisticated approach is to discover which pairs of papers are cited by the same sources; the members of these co-citation pairs are likely to be especially closely associated within the field.

The co-citation analysis of CSCL literature yields a set of “seminal works” for the field according to the authors of this study. The resultant list is striking in several ways. First, it has little overlap with the results of the previous study. Although all eight of the central works are from journals included by the other study published within its time period, only three of them (37.5 %) are even included in the previous study’s list of 278 selected papers. Second, with the exception of the paper by Suthers, they are all from a couple of labs in northern Europe, who interacted with each other as part of the European Union’s Kaleidoscope network. They knew each other and worked closely together, so it is not surprising that they cited each other and were co-cited in later publications (probably especially by each other and their colleagues). It seems likely that the method over-generalized and focused on a sub-network of the CSCL community.

Some of the papers may have been excluded from the previous study because they were not primarily empirical in their focus, but more theoretical. The Suthers (2006) paper argues for a post-cognitive CSCL paradigm, Cress and Kimmerle (2008) propose a combination of systems theory and cognitive psychology and Kobbe et al. (2007) introduce the theme of scripting in ijCSCL. The others report on examples of content analysis, argumentation support or scripting that are more empirically based. These were probably all important publications within the CSCL research community, so they confirm the method to some extent despite the fact that the list itself may also be biased by certain factors such as personal associations, funding sources and timing.

Some of the papers on the list of eight seminal works reappear on other catalogs of popular CSCL publications, such as most frequent downloads of ijCSCL papers, top citations in the World of Science (WoS) or Google Scholar. In order to check the clustering based on co-citation and SNA, it would be interesting to analyze which papers cite the same sources (e.g., Piaget vs. Vygotsky, or the foundational works of different theories).

Another approach would be to cluster papers based on their latent semantic similarities, using LSA or LDA. Now that conference proceedings, journals and book chapters are readily available on the Internet, it would be possible to form a large corpus of CSCL publications and mine or cluster it automatically to discover themes, sub-areas and focal topics. Looking at different time slices by publication date could document trends and evolutionary directions for the field.

What are the core questions that we would like answered by analyses of the CSCL corpus? It seems that we want to know about both the bulk of CSCL publications—what are the various approaches taken—and about the leading-edge pioneering articles—which are they and how are they trending. In what ways is CSCL defining a distinctive research paradigm and in what ways does it remain an eclectic mix of incommensurate approaches? The first of the literature analyses here looks at the breadth of empirical work during a formative period in CSCL history and the second one shows how some seminal works stand out in this corpus. How do the findings of these analyses compare with your subjective sense of the field?