Support of grounding in the classroom

More and more educators and researchers use ICT-tools to support a great variety of collaborative learning tasks (Jonassen 2000; Norman 1993). In particular, the support of complex problem-solving has received much attention from researchers and developers in the field of ICT. Specific ICT-tools have been designed to support problem-solving activities such as group design (Buckingham Shum et al. 1997), scientific reasoning (Suthers 2001), and computer modelling (Löhner et al. 2003). As such they all address aspects of the structure of complex problems, often based on a problem’s ontology. Buckingham Shum et al., for example, set out from the notion that any design problem can be structured in terms of design questions, options for answering the questions, and criteria that have to be fulfilled in order for the options to provide an answer to the questions (i.e., the QOC approach to group design).

As complex problem-solving typically is a collaborative activity, the above methods face two problems. First, they all are predicated on the combination of knowledge sharing and knowledge integration. However, research has shown that individual learners have to engage each other’s thinking in order for collaborative learning to be more effective than individual learning (Barron 2003). This means that learners need to achieve a common cognitive frame of reference, or common ground (Bromme 2000; Clark and Brennan 1991), to reap the benefits of having multiple problem perspectives within a learning group. While the tools mentioned offer support for coping with complexity in terms of specific problem structure, they do not offer complementary support for the negotiation of common ground. In other words, none of the tools mentioned address the structure of the groups that solve these complex problems, and specifically the common ground they need for problem-solving.

Various researchers have pointed out the importance for common ground for collaborative learning. According to Baker et al. (1999), any task that demands a collaborative effort will also demand that learners negotiate a shared task representation. In the context of complex problem-solving, this would mean that they have to negotiate a shared problem definition in order to successfully solve the problem. The present article presents NTool, an ICT-tool that addresses the group aspect of multiple perspectives by facilitating the negotiation of common ground. NTool is an online communication tool (see Fig. 1 for an impression) with embedded support of grounding processes (“grounding processes” and “negotiation of common ground” are used synonymously in this article).

Fig. 1
figure 1

An impression of NTool, with pictures of the message types (lower left-hand), the discussion window (lower right-hand), and the chat window (upper right-hand), with a snapshot of discussion (in Dutch)

Full size image

Although many ICT-tools indirectly affect grounding processes, as an effect of using external representations for group support, NTool aims to facilitate the collaboration by directly influencing the grounding process. It does so by making users externalise their private understanding of others’ contributions. The word “private” is used because such individual understanding can remain un-externalised, in which case the contributor has no way to find out what the other understood. Externalising one’s private understanding is important for the negotiation process. Facilitating the grounding process will in turn increase the extent to which learning groups establish common ground.

Like other ICT-tools, NTool uses specific communication rules (a formalism) and constraints (coercion) to attain this facilitation. Communication rules are rules that prescribe a certain communication procedure. An example of a communication rule would be: “If someone introduces a new topic, all conversation partners are required to explicitly verify whether they understood what was meant.” Coercion is a means of making participants adhere to a formalism so as to increase its effectiveness. In the case of the above example rule, applying low coercion might be that discussion partners are informed about the rule, but can decide for themselves whether or not they follow it. An example of high coercion would be that a chair-person would not allow new topics to be put on the table before the rule was followed (i.e., understanding was explicitly verified).

The goal of the present study is twofold. The first goal is to replicate the results of an earlier study (see Beers et al. 2005). The main differences with the earlier study, which are discussed in detail in the next section, is that this study is performed in a more ecologically valid setting, and some social aspects of the group are measured, so as to gain an impression of how robust the positive results of the original study are when NTool is tried in a more demanding setting. The second goal is to control for possible unintended effects of NTool by measuring additional variables (also see the next section). In this study, cognitive load is used as a check for possible cognitive overloading, and the perceived social affordances of NTool and group psychological safety are used to increase explanatory power in case of unexpected results and control for confounding due to social differences between groups.

We first discuss the previous study by Beers et al. (2005), the shortcomings it had in terms of measured variables and ecological validity, and how we dealt with that in the present study. We then describe our perspective on the relation between collaborative learning and negotiation of common ground, after which we present our theoretical framework.

Earlier research with NTool

In a prior study, Beers et al. (2005) explored the influence of different levels of coercion (low, medium and high coercion conditions) on the effectiveness of NTool in terms of the negotiation of common ground and the extent to which common ground itself was established. Participants were students of business sciences, psychology, or cultural sciences university seniors who were divided into multidisciplinary groups of three. Each group was assigned a collaborative problem-solving task to be carried out using NTool. Six groups were in the low-coercion condition, five were in the medium coercion condition, and six were in the high-coercion condition. NTool’s influence on both the grounding process and the extent to which common ground was achieved was shown to increase with increasing level of coercion. There was a significant correlation between level of coercion and negotiation of common ground, r s(N = 17) = .51, P < .05; the higher the level of coercion (i.e., the fewer degrees of freedom that the participants had to operate and thus had to adhere to the formalism), the greater the effects of NTool were on the negotiation of common ground. However, the results also showed that the medium coercion groups required significantly more regulation behaviour than the other groups, U(N = 17) = 4.00, P < .01. This result was unexpected and its causes were unknown, partially due to the inclusion of only measures of expected outcomes. Furthermore, being a typical laboratory study, it had limited ecological validity.

The study presented here is a partial replication of that, but with additional measures and increased ecological validity. With regard to the additional measures, research has shown that achieving common ground is also influenced by social factors. For instance, while designers of ICT-tools typically facilitate task content communication, they may be limited in the extent to which they afford (see Norman 1993) social communication aspects (Kreijns et al. 2002). The question then arises as to whether such ICT-tools have sufficient social affordances to allow for a good team atmosphere to emerge. Social affordances are particularly important in the case in hand, as research has also shown that social beliefs such as team psychological safety can influence grounding processes (Van den Bossche et al. 2004). It is not surprising, then, to assume that the social affordances of NTool might influence grounding processes. This article, therefore, includes measures of social affordances for NTool and of psychological safety as a measure of team social beliefs.

Also, the medium coercion condition in the prior study needed disproportionately more regulation than the low and high coercion versions. For instance, participants would ask each other about why NTool would not allow them to post the message they intended to post, or they would need to explain to each other what steps they were to take to keep the discussion going. This may indicate that the participants experienced difficulties using this version of the tool that were not experienced by participants using the other versions. This could indicate some disruption of communication processes because the communication was constrained to the extent that participants had trouble carrying out their task (Dillenbourg 2002). Indeed, sometimes aspects of ICT-tools may lead to cognitive overloading, in which case they can even have negative influences on group learning processes and outcomes (Van Bruggen et al. 2002). However, no quantitative measurements were included in the earlier study to substantiate this explanation. Measures of cognitive load were, thus, included in the present study to rule out the possibility that the combination of NTool and the learning tasks cause too much cognitive load to properly carry out the task itself.

With regard to differences in ecological validity, the 2005 study was a laboratory study in which 17 groups of three students carried out a collaborative learning task. For each group, members were chosen so that they did not know each other in advance, so as minimise differences between groups with regard to social climate. The study was conducted under highly regulated circumstances with highly motivated participants, so that influences of social processes on the effectiveness of NTool could be assumed to be small. Furthermore, the collaborative task to be carried out—solving the problem of high-school drop-out—was chosen on the basis of prior experiences with the same task (Van Bruggen 2003). Though a current and relevant problem in general, it was not very realistic in the sense that the participants would not have received such a task as part of their normal education. In contrast, the task used in the present study served as both pilot for a new, and thus real, multidisciplinary learning task in a secondary vocational education institute as well as an experimental task. The study involved a problem-solving task for multidisciplinary groups of three, carried out in a computer lab in the vocational secondary school with approximately 30 participants at once. This setting was identical to a real class setting used for some of the participants education and the task was constructed in collaboration with teachers from each of the disciplines present in the student groups.

Collaborative learning and common ground

We distinguish two ways in which negotiation of learning is related to collaborative learning. First, negotiation of common ground can be seen as being part of any collaborative learning process in which groups collaboratively negotiate the meaning of some learning content. This point closely follows the argument made by Baker et al. (1999) who defined a collaboration task as an effort that results from a shared (read: grounded) problem definition, after which it follows that collaborative learning involves grounding by definition. In other words, any collaborative task requires some common ground to begin with, and continued attention to maintain common ground after that. Learning about how to negotiate common ground can help learners avoid the more common pitfalls in negotiation processes that can undermine collaboration tasks. For instance, research has shown that learning groups will generally concentrate on that knowledge the group knows it shares (i.e., knowledge that has already become part of a group’s common ground), instead of unshared knowledge (Fischer and Mandl 2005). Similarly, Bromme et al. (2001) found that people overestimate the extent to which their own disciplinary knowledge is known by others. Furthermore, their research even suggested that it is up to the collaborator with the least understanding of another’s field to monitor the common ground itself. These findings suggest that the negotiation of common ground is a difficult process with some pitfalls easily trodden into. Instructing learners how to negotiate common ground may be able to help them learn how to collaborate.

While the first way posits the negotiation process as a prerequisite for collaborative learning, the second, complementary way sees the negotiation process and the learning process as largely overlapping. In this case, it concerns specific tasks that benefit greatly when done in a co-ordinated effort instead of individually. What learning tasks are we talking about? First, they must be group tasks in the sense that they can not be carried out individually (Cohen 1994). Second, they should concern ill-structured problems, meaning that there is no clear problem-solving path, nor one right answer, typically require concerted group efforts (Cohen). Such tasks involve exchanging ideas, hypotheses and speculations. Making sure that the group as a whole constructs from this exchange a set of mutually shared ideas, hypotheses and speculations can itself be seen as a grounding process, for it means that the group needs to negotiate the meaning of a set of topics that most or all group members are only partially familiar. For instance, in cases such as problem-based learning (Barrows and Tamblyn 1980) where learners study the same topic, but may use widely different sources of information. Grounding is important here because learners need to verify and clarify those differences that arise from studying different sources on the same matter. In this case, learners arrive in groups with incomplete knowledge, and learn from each other by exchanging information and grounding it. Conversely, negotiation of common ground is of lesser importance for learning tasks where individual differences in knowledge and understanding play a small role, such as in class-based mathematics education. On the whole, one might say that grounding is important for any learning task with an important collaborative aspect.

Negotiation of common ground

Theory on negotiation of common ground originated in linguistics (Clark and Schaefer 1989). Bromme adapted the theory to the field of cognitivism in his writings on cognitive interdisciplinarity (Bromme 2000). The linguistic approach describes how negotiation of common ground appears in conversation. It emphasises the process as it can be observed from communication, and describes how people can contribute to communication, in order to add to a group’s common ground. Although Clark and Schaefer do mention some individual influences on grounding (“prior beliefs, assumptions, and other information”, p. 260), their writings focus on the conversational mechanisms. In contrast, the cognitivist approach focuses on the way new knowledge is processed, the role that prior knowledge plays in this process, and how individual perspectives affect this. Specifically, it describes how mutual perspectives (assumptions of prior knowledge of the other, and whether one’s own knowledge is shared with the other) influence what people will contribute in conversation. In other words, the linguistic approach focuses on surface characteristics of communication and the cognitivist approach addresses the underlying cognitive influences. In that sense, the two theories can be seen as largely complementary. The combination of the two links the content of a learning process to the way it is communicated between people.

The following exemplifies how both theories can be combined; Clark and Schaefer (1989) write that people will assume that the content of an utterance is part of common ground unless there is evidence to the contrary. Bromme et al. (2001) found that people overestimate the extent to which their own knowledge is known by others, and that they make their contributions partly based on their assumptions of the other’s prior knowledge. It would follow that many contributions will require subsequent clarification before their content is sufficiently understood in the same way by all conversation partners.

This article combines linguistic (Clark and Brennan 1991) and cognitive (Bromme 2000) approaches to the negotiation of common ground. In this conceptualisation, the grounding process starts when team members contribute their, as yet, unshared knowledge, so that others can try to apprehend that knowledge. At this point, a number of biases come into play that cause differences between the intended meaning of a contribution, and the contribution as it is understood. While constructing their own individual understanding, the other team members use their knowledge of aspects like the contributor’s background and views held, and the current situation, to better “understand” the contribution. Also, their own beliefs and assumptions play a role while trying to understand a contribution. A contribution is thus always understood against the presumed perspective of the other and one’s own perspective (Bromme 2000). Therefore, having shared a contribution with a team does not necessarily mean that the team members all have acquired the same understanding. Instead, it is often the case that different people will have dissimilar representations of the content and meaning of the contribution, about the objects, people, interactions or situations the contribution discusses (Boshuizen and Tabachnek-Schijf 1998). People can have different representations as a result from interpreting a contribution only in their own perspective or from minimising or rejecting its validity or plausibility due to differences in conviction or opinion. Also, presented with the same information, different people may still come to different conclusions. The negotiation of common ground then is the iterative minimisation of these representational differences, through providing feedback based on one’s own perspective by word or action (Alpay et al. 1998; Baker et al. 1999).

In the present article, negotiation of common ground is conceived of as a dual concept. The first aspect is negotiation of meaning which leads to agreement regarding meaning and understanding of a contribution. This entails making one’s as yet unshared understanding of some contribution public to others, who in turn verify whether and to what extent their own understanding of the contribution is the same or different from what others intended, receiving feedback on this (clarification), re-verifying, and so on, until “the contributor and the partners mutually believe that the partners have understood what the contributor meant to a criterion sufficient for the current purpose” (Clark and Schaefer 1989, p. 262, the grounding criterion). Negotiation of position, the second aspect, concerns people making public to others their private opinion about a contribution, checking whether one’s position is clear to others, and vice versa.

The above description of the grounding process is the basis for the design of the NTool formalism to support negotiation of common ground. The formalism consists of negotiation primitives, basic building blocks that constitute a specific dialogue model (Dillenbourg 2002), and rules that prescribe the use of these primitives. Table 1 summarises these rules. Together these rules and primitives mimic the negotiation process as explicitly as possible. Note that this formalism models an ideal negotiation process; in regular communication, the status of people’s statements in terms of negotiation primitives often remains implicit. The formalism must enable the user to more easily distinguish between original contributions, clarifications, verifications, et cetera, thus making the negotiation process more explicit. This way, individual differences in understanding and opinion between users should more easily surface.

Table 1 Rules for the NTool support principle
Full size table

Negotiation starts with a contribution (Primitive 1) such as a hypothesis or a position, which is assumed not to be part of a team’s common ground (Rule 1). To detect differences between individual representations, team members must verify (Primitive 2) their understanding of the contribution (Rule 2) because people articulate and understand a contribution against their own background knowledge (Fischer et al. 1995). Third, a contribution needs to be elucidated (clarification, Primitive 3), using the ideas upon which it was based. For example, the educational background or the political orientation of the contributor may shed light on the meaning of a contribution. A clarification need not always be made by the original contributor, but may also be performed by another team member who feels knowledgeable or that (s)he understands the contribution. Rule 3 is that all verifications require a clarification. Together, Rules 2 and 3 can be iterated until common understanding of the contribution is reached.

The fourth primitive is acceptance/rejection of a contribution, whether one can judge a contribution as true (acceptance), or untrue or unintelligible (rejection), based on the explanation given and one’s own prior knowledge. For example, the statement 1 + 1 = 10, is true only if we understand (through Rules 1 and 2) that the contributor is using the binary system. A contribution should be accepted as part of the common ground if it is true, or after it has been modified so that it has become true. Rule 4 is that every contribution needs to be accepted or rejected by the team members. Finally, Rule 5 is that people must explicitly state their own position (position, Primitive 5) on the contribution. In the case of irresolvable disagreement about previously accepted statements, Rule 5 may result in multiple scenarios, each based on another position (i.e., agree to disagree). This means that one may accept a certain contribution, but disagree all the same, for example when neither person can prove the other wrong. In such cases, people can agree to disagree, and alternate representations that are equally legitimate can ensue.

Note that the work by both Clark and Schaefer (1989) and Bromme (2000) is descriptive, describing how the actual grounding process unfolds. In contrast the above formalism has a very prescriptive nature. Also note that the formalism is particularly task-oriented, that is, it aims to facilitate grounding activities related to the task at hand and its informational content, and disregards other grounding activities. In that sense, the formalism theoretically departs from both Clark and Schaefer and Bromme.

Conceptually, there is little difference between common ground as we use it here and knowledge convergence as it has been used by Fischer and Mandl (2005). The latter concept, however, focussed not only on knowledge that is shared, but also on shared use of that knowledge, or, in other words, the same learning outcomes across collaborating learners. Conversely, our use of the concept of common ground allows different learning outcomes, it only requires the outcomes reported by one group member to be known by the other group members to be common ground. Learners in collaborating groups may well disagree on certain aspects of a collaborating task. They would still be on common ground if they at least know of their respective different opinions, but they would not portray knowledge convergence.

The negotiation tool

NTool is based on a newsgroup reader for asynchronous, distributed, text-based discussions. To optimise NTool for negotiation of multiple representations, the formalism was implemented to structure the negotiation process in two ways with different levels of coercion (cf. Dillenbourg 2002).

Coercion, a form of scripting, is defined as the degree of freedom participants have in following a formalism. Coercion and formalism together constitute a collaboration script. The higher the coerciveness of a script, the more the participants are required to adhere to the formalism. Scripting requires “subjects on most or all occasions to make a particular type of speech act in a specific context” (Baker and Lund 1997, p. 176). A script that uses very little coercion leaves participants many degrees of freedom such that usage of the formalism attains a high degree of idiosyncrasy. A script with a high level of coercion constrains the number of options participants have, thus guiding them along the lines of the formalism (Weinberger et al. 2005).

In the present study, two different ICT-implementations of the formalism were employed (see Methods). One implementation had very little coercion; although it did offer the various message types as support, participants could use them as they liked. The other implementation used scripts aimed at interaction and collaboration (high coercion) and so applied strict rules to when and in what situation a message type could be used. In each implementation, coercion was aimed at the verification and clarification primitives, that is, the extent to which people were required to verify and clarify contributions in specific circumstances.

Social influences and unintended effects

In addition to measuring the intended outcomes of NTool (increased negotiation and common ground with increased coercion), we aimed to control for the influence of social variables on grounding processes, and we wanted to rule out unexpected social and cognitive effects.

With regard to social influences on grounding, research has shown that several team beliefs may have an important influence on the grounding process (Van den Bossche et al. 2004). In this study, we focus on one such factor, psychological safety, defined by Edmondson (1999) as “a shared belief held by members of a team that the team is safe for interpersonal risk taking” (p. 350). Her studies show that psychological safety affects team performance by augmenting a number of team learning behaviours, some of which are seeking feedback and sharing information. Conversely, she found that a lack of psychological safety made team members reluctant to ask feedback or question team goals, out of fear for sanctions. Among the important mechanisms, Edmondson found that psychological safety makes it easier for people to build on each other, to voice disagreements, and to ask feedback. Van den Bossche et al. (2004) reported similar results for the influence of psychological safety. Their results showed that psychological safety facilitates the negotiation of mutually shared cognition, which in turn leads to increased perceived team effectiveness. The above results suggest that psychological safety might be able to facilitate making a verification and voicing one’s opinion, because learners have more trust that this will not lead to repercussions from their team partners. Therefore, in the present study psychological safety is treated as an important covariate for the effect of NTool on negotiation. Psychological safety is expected to have a positive influence on the negotiation of common ground.

With regard to ruling out unintended social and cognitive affects of NTool, it is important to first understand how coercion may adversely affect collaboration. Dillenbourg (2002) argues that there are trade-offs between coercion and team processes. On the one hand, increasing coercion may increase the effectiveness of the formalism in question. However, too much coercion may disturb interactions (disruption of communication, Dillenbourg 2002; also see Cohen 1994). An example is the aforementioned study by Beers et al. (2005), in which one specific version of NTool resulted in a disproportionately large number of regulative utterances. This might indicate two things: the additional need for regulation (1) shifts attention away from social processes and therefore negatively affects social aspects of NTool; and it (2) is an additional demand on working memory processes which may cause cognitive overload. Furthermore, the extent of coercion needs to be geared toward the learners’ capacities. The less coercion, the more learners need to fulfil the prerequisites needed to execute the learning processes as facilitated (Cohen 1994). In this study we measured social aspects of NTool to explore whether low and high coercion versions of NTool differed with respect to the extent in which they afforded social processes, and we measured cognitive load to be able to detect significant differences in working memory load.

With regard to social aspects of NTool, we used work by Kreijns (2004) on social presence and sociability in ICT-learning environments. Note here that social presence and sociability are aspects of the ICT-environment, whereas psychological safety is an aspect of teams. Social presence is defined as “the ability of learners to project themselves socially and emotionally” in a learning environment (Rourke et al. 1999), and is thought to support critical thinking processes in learning groups (Garrison et al. 2000). In the case of NTool, social presence is thus dependent on the extent to which NTool affords participants in conveying social and emotional messages. Social presence is closely related to sociability, defined by Kreijns as “the extent to which the CSCL environment is able to facilitate the emergence of a social space” (p. 7), that is, a “human network of social relationships between group members” (p. 7). In the present study, both sociability and social presence are measured.

With regard to cognitive load effects, we build on the work of Van Bruggen et al. (2002) on the effects of external representations on cognitive load. Van Bruggen et al. (2002) found that poorly designed instruction for knowledge externalisation may increase cognitive load, due to the effort needed to apply the instruction in action. In the case of NTool, this effect conceivably could be caused by too much coercion. Cognitive load theory (Paas et al. 2004; Sweller et al. 1998) distinguishes between germane cognitive load, which is caused by working memory processes that lead to schema construction and automation, and extraneous load, which is “imposed by the manner in which the information is presented to learners and by the learning activities required of learners” (Paas et al. 2004, p. 2). Extraneous load can thus be caused by understanding the instruction aimed at generating germane processes. Because of working memory limitations, it is important that extraneous load is minimised, and that total load does not exceed working memory limitations (Sweller et al. 1998). With regard to NTool, this means that high coercion could lead to increased extraneous cognitive load, because high coercion requires the participants to allot working memory resources to understanding the way coercion is implemented, but also to increased germane cognitive load, due to increased working memory allotted to verifying and clarifying contributions. The present study compares cognitive load measurements with differences in negotiation and common ground between Stringent and Idiosyncratic versions of NTool to study germane and extraneous cognitive load effects of coercion.

Hypotheses

NTool was expected to increase the negotiation of common ground because it forced team members to make their private understandings and opinions public, making differences in understanding and opinion visible or salient (Bromme 2000). We hypothesised that (1) the higher the level of coercion, the more negotiation would occur. Likewise, we hypothesised that (2) common ground would be highest in the Stringent version and lowest in the Idiosyncratic version. Both hypotheses presume that more coercion will make participants follow more closely an ideal model of negotiation, as laid down in the formalism. In the present study no measures for actual individual learning were made. We did not measure task performance, either on the group level, or on the individual level.

Psychological safety was expected to be positively associated with negotiation and common ground, but not to differ between low and high coercion versions of NTool. We expected cognitive load differences due to increased negotiation (germane load) with the high coercion version of NTool, but also due to extraneous effects of coercion such as increased regulation.

Method

Participants

Participants were 66 second year students (age 17.9 years, SD = 1.22) in senior secondary vocational education from three different education programmes, High-tech Metal-Electrics, Infrastructure, and Architecture. Participants were assigned to 22 three-person multidisciplinary teams.

Materials

Task

Participants were assigned the task of making a functional design of floating housing as a solution for sea-level rise; a very important issue in the Netherlands. The task was designed in collaboration with executives from the educational institution to ensure that the difficulty level was appropriate for the level of expertise of the participants, and that the task was interesting to all different educational programmes. The task was split in two parts. In the first part (“Floating Houses”) the participants were to design a floating house. In the second part (“Amersfoort-by-the-Sea”) additional information was given to keep the participants going.

Ntool

Each team was supplied with three computers running NTool, one for each participant. Two different collaboration scripts were used.

Idiosyncratic

This version used all primitives. On-screen information was presented about every contribution, and whether it needed yet to be verified or decided upon (agreeing or disagreeing; see Fig. 2 for an example of on-screen information for a contribution that needed to be decided on). Furthermore, each participant was informed when he/she had not yet verified all contributions, and when he/she had not yet decided on all contributions. Participants were free to choose to what extent they verified and rejected, agreed, or disagreed upon the contributions in the discussion. No coercive rules were used to require this. This means, that if they wanted, participants could keep posting new contributions without ever making a verification, they could completely refrain from using agreement, rejection and disagreement messages. This situation resembles giving a person a set of lines and symbols to be used in constructing a diagram, but leaving it up to her/him to decide which symbols and lines are used for what purpose, hence the name “Idiosyncratic”.

Fig. 2
figure 2

The discussion window with one contribution selected. Information about the status of the contribution is presented in the lower right-hand corner

Full size image

Stringent

This version used the same primitives as the idiosyncratic version, but used coercion to allow negotiation of only one contribution at one time. Furthermore, participants were not allowed to compose reject-, agree-, and disagree-messages before the contribution had been verified. While using the tool, options for message types that could not be used were greyed out and made unavailable to the participants. For example, if a participant wanted to send a disagreement message, but had not yet posted a verification, then the disagreement option would be greyed out in the menu, and the participant would not be able to choose it (see Fig. 3 for an example). Using prompts, participants were informed as to whether they had to verify or decide on a contribution. This version was called the “Stringent” version because it applied strict rules for the situations in which each message type could and could not be used.

Fig. 3
figure 3

The message menu, with greyed-out options for verification, agreement, disagreement, and rejection

Full size image

Eleven groups used the Stringent formalism and eleven groups used the NTool Idiosyncratically.

Questionnaires

All questionnaires used in this study were adopted from other researchers who had been able to use them reliably. The reliabilities reported here were computed using the data from the current study. Group means of psychological safety (Cochran’s α = .69) were used as a covariate, to control for differences in common ground and negotiation (Van den Bossche et al. 2004). Social aspects of NTool were measured using scales (Kreijns 2004) for sociability (α = .86) and social presence (α = .86). Cognitive load measurement was done through self-report of invested mental effort on a one-item symmetrical scale ranging from 1 (very, very low mental effort) to 9 (very, very high mental effort) (Paas et al. 2003). Mental effort refers to the cognitive capacity that is actually allocated to solve the problem and can be considered to reflect the actual cognitive load (Sweller et al. 1998).

Procedure

Practice phase

The participants first received a tutorial that addressed the basics of NTool and then proceeded to emphasise the rules of the formalism, and the way they constrained communication. To ensure that participants were proficient with the NTool a practice case (solving a road traffic safety problem) was used to enable them to gain experience with the NTool. The practice phase took 75 min.

Experimental phase

The experimental phase was divided in two identical sessions, one in the morning and one in the afternoon, with a 75-min lunch-break in between. After a 15-min coffee break, participants started with the morning session with the experimental (sea-level rise) case. To promote the construction of an individual problem representation, as well as to allow the researchers to determine what this representation was, participants first had to carry out the task individually (pre-test, 15 min). Participants could take notes while working individually on the task. Next, they solved the problem collaboratively (60 min), and after that individually again (post-test, 15 min). All resulting individual problem representations and solutions, as well as the group discussion were logged. In their post-test, participants were also asked to state the points on which they felt that they had differences in opinion with their team members, to account for agreeing to disagree. The afternoon session had the exact same structure, but used a slightly different task called “Amersfoort-by-the-Sea”. This task built on the sea-level rise-case. No differences between morning and afternoon session were expected due to time of day.

Cognitive load was measured after each pre-test, each collaboration task, and each post-test.

Analysis

Negotiation

Negotiation was operationalised in two ways, namely the number of explicit negotiation activities, and the amount of negotiation per conversation topic. A coding scheme for the coding of function and content of messages during collaboration was developed (cf. e.g., Avouris et al. 2003; Fischer et al. 2002; Mulder et al. 2002; Swaak et al. 1998; Thomas et al. 1982). All messages were coded with regard to:

  • Cognitivecontent—directly related to solving the problem.

  • Regulative content—related to monitoring the problem-solving process and regulating the collaboration process.

  • Other content—not in any other category.

  • Nonsense—uninterpretable messages.

Messages with cognitive content were specifically coded for function. The following subcategories were used to code negotiation:

  • Contribution: A new topic of conversation that has not been discussed before is introduced.

  • Verification: Information is directly or indirectly requested about the intended meaning of a contribution or elaboration.

  • Clarification: A reaction to a verification or a perceived lack of understanding, in which the intended meaning of a contribution or elaboration is elucidated.

  • Acceptance: A reaction to a contribution in which the contribution is judged intelligible and/or correct.

  • Rejection: A reaction to a contribution in which the contribution is judged unintelligible and/or incorrect.

  • Agreement: A reaction to a contribution in which the sender voices his/her agreement with the contribution.

  • Disagreement: A reaction to a contribution in which the sender voices his/her disagreement with the contribution.

During the analysis it became apparent that some messages did not fit any of the above subcategories, for example if people built on each other’s communications without explicitly negotiating meaning of, or position on, a contribution. Such messages were coded Elaboration when a contribution was elaborated upon by adding information or summarising. Verification and clarification, in contrast to elaboration, were considered indicative for explicit negotiation activities. The total number of contributions discussed was used as an indicator for the range of topics discussed. Furthermore, messages with regulative content that aimed at Monitoring the problem-solving-process were distinguished from those that were only used for Regulation of the conversation.

A research-assistant received 40 h of training on how to use the coding scheme. Comparing three randomly selected experimental sessions coded by the first author and the research-assistant resulted in a substantial (Landis and Koch 1977) inter-rater reliability (Cohen’s kappa) of .73 (SE = .024). All data were further coded by the research-assistant.

To measure the number of verifications and clarifications per conversation topic, episodes in the discussion that dealt with one conversation topic were first identified. The contributions identified with the coding scheme for negotiation were considered the starting points for a new discussion episode. An episode generally started with a contribution and ended when one of the participants made a new contribution, and all of the discussion in between these contributions dealt with one conversation topic. For each group, negotiation per conversation topic was then calculated by dividing the sum of all clarifications and verifications by the number of contributions.

Common ground

Common ground was conceptualised as the degree of overlap in individual representations after collaboration, in terms of contributions. For each of the episodes identified earlier, we counted the number of individual representations they were present in. This was used as a measure of spread for each episode. Common ground was then computed as the average spread for all episodes in one group.

Cognitive load

Pre-tests were expected to be the same for both versions of NTool. Group tests were expected to differ between conditions, with the Stringent version of NTool expected to result in more cognitive load, due to an increase in germane load as a result of increased grounding activities (germane load) or due to additional attention required by the NTool formalism (extraneous load). In other words, extraneous effects cannot be ruled out if cognitive load differences are found after the group task. Post-tests were also expected to differ between conditions, with participants in the Stringent groups expected to report an increase in cognitive load because of increased processing of knowledge acquired from the other participants due to increased grounding activities (germane load).

Statistical analyses

Negotiation and common ground were analysed using repeated measures ANCOVA. Psychological safety was included as a covariate. In the case of significant main effects from condition, but non-significant main or interaction effects of psychological safety, analysis was repeated without psychological safety. All analyses were performed with SPSS version 11. Due to participant drop-out after the first experimental phase, the number of groups used in the statistical analyses was lower than 22. Data for 9 groups in the Idiosyncratic, and 5 in the Stringent conditions were eligible for statistical analysis. Significant effects of phase I on phase II (effects of time) were not considered relevant to our hypotheses, and are not reported here.

Cognitive load measurements were analysed using nested repeated measures ANOVA. The first level consisted of cognitive load measurements after the pre-test, after collaboration, and after the post-test. As these measurements were done repeatedly, once in the morning and once in the afternoon, session was included as a second level in the analysis. Since we assumed that there would be differences after collaboration and after the post-test, but not after the pre-test, we expected to find an interaction between condition and cognitive load. We expected possible main effects of condition on cognitive load after collaboration and after the post-test.

Results

Negotiation and common ground

Repeated measures ANOVA revealed a significant interaction between session time and coercion on verification F(1, 11) = 8.12, P < .05, and also main effects of both psychological safety, F(1, 11) = 5.83, P < .05, and condition, F(1, 11) =5.61, P < .05, on verification. In both sessions, Stringent teams made more verifications than Idiosyncratic teams. Inspection of Table 2 shows that this effect is significantly stronger in the morning sessions than in the afternoon sessions. Main effects of psychological safety and coercion on clarification also were observed, F(1, 11) = 6.67, P < .05 and F(1, 11) = 6.17, P < .05, respectively. The Stringent groups had more clarifications than the Idiosyncratic groups, and psychological safety positively affected both verification and clarification. Furthermore, there was a significant positive main effect of psychological safety on agreement, F(1, 11) = 7.35, P < .05. A non-significant, but notable effect was found for coercion on regulation, F(1, 11) = 4.56, P = .06, which is mentioned here because it may assist in explaining the unexpected results. Regulation was highest in the Stringent groups. Finally, there was a significant main effect of coercion on ‘other’ communication, F(1, 11) = 5.60, P < .05; eliminating psychological safety from the model still resulted in a significant main effect of coercion F(1, 12) = 6.05, P < .05, which means that ‘other’ communication occurred most in the Stringent groups.

Table 2 Negotiation and common ground
Full size table

We found significant main effects of both psychological safety and coercion on negotiation of meaning per contribution, F(1, 11) = 9.34, P < .05 and F(1, 11) = 8.17, P < .05 respectively. Negotiation per contribution was highest in the Stringent groups, and increased with psychological safety. Finally, there was a significant main effect of coercion on common ground, F(1, 11) = 9.78, P < .01, but not in the expected direction. Eliminating psychological safety from the model still resulted in a significant main effect of coercion F(1, 12) = 8.46, P < .05. Common ground was highest in the Idiosyncratic groups.

Cognitive load

Nested repeated measures ANOVA (see Table 3) revealed a significant interaction of session time, cognitive load, and coercion, F(1, 52) = 7.67, P < .01, and of cognitive load and session, F(1, 52) = 4.54, P < .05, which shows that there were significant differences between the three cognitive load measurements (pre-test, group work, post-test), and that these differences were specific for session time (morning or afternoon) and coercion. Figure 4 seems to indicate a difference between the afternoon session in the Stringent groups and the other sessions. In the afternoon Stringent session, cognitive load on the post-test is higher than cognitive load after the pre-test and after group collaboration, whereas in the Stringent morning session and in both Idiosyncratic sessions it is lower. Furthermore, cognitive load measurements in the Stringent morning session were higher than cognitive load measurements from the other sessions. Finally, we found a marginally significant main effect of coercion on cognitive load, F(1, 52) = 3.67, P = .06, suggesting that cognitive load was highest in the Stringent groups.

Table 3 Cognitive load
Full size table
Fig. 4
figure 4

Cognitive Load Measurements

Full size image

Social aspects of NTool

No significant differences were found with respect to social aspects of NTool (see Table 4).

Table 4 Social aspects of NTool
Full size table

Discussion

This article reported on the relationship between negotiation, the negotiation formalism, and coercion, with the ultimate goal being to design an ICT environment that facilitates knowledge construction. The main goal was to design a formalism for the facilitation of common ground, which is an important aspect of collaborative learning and appears to be a prerequisite for knowledge construction.

The results showed that, as expected, the Stringent version of NTool increased negotiation activities, both with regard to the number of verifications and clarifications, and the amount of negotiation per contribution, although the effect of coercion on verification was significantly stronger in the morning sessions than in the afternoon sessions. This means that, as in our previous study with NTool, high coercion increases negotiation activities during interaction. However, contrary to our expectations, and also to previous findings, common ground was highest in the Idiosyncratic groups. It seems that high coercion did have the expected influence on interaction, but that this change in interaction did not result in associated effects on common ground.

Disruption of collaboration (Dillenbourg 2002), which can be caused by over-scripting collaboration, may explain some of the results. Two unexpected differences that did not occur in our previous study are quite telling in this respect. The first is the marginally significant difference in regulation activities and the second is the difference in ‘other’ communication. Both figures were highest in the Stringent groups. The difference in regulation might show that stringent use of NTool was quite taxing. The difference in ‘other’ communication may signal a lack of motivation for the task at hand, that may have been a side-effect of the need for regulation.

The nature of the specific participant population may explain some of these results. In a previous study of the influence of coercion on the effectiveness of NTool (Beers et al. 2005) senior college students did not show more regulation with high coercion. Secondary vocational education institutes, however, generally draw a student population that is in many respects different from senior university students. On average, the participants of the current study were about 4 years younger, and therefore may have had less knowledge of the domains they were to use in the task, and were less serious than the university students. Also, they had little prior experience working in groups. Indeed, recent research suggests that university-level students benefit more from tools that facilitate reflective thinking (which NTool does), due to prior learning experience, than high-school learners, who still need more general group support (Song et al. 2006). This may explain why the secondary vocationally education participants needed to regulate more than the university seniors studied by Beers et al. (2005). The high occurrence of ‘other’ communication with the high-coercion NTool may in fact signal that the participants were distracted or frustrated. Explanations involving cognitive overloading and a lack of social presence and sociability of the Stringent NTool can both safely be ruled out. The cognitive load measurements do not suggest increased cognitive load during group work, and no significant differences were observed with regard to sociability and social presence.

The above explanation indicates a number of individual and task prerequisites that may be needed for NTool to be effective. The target population may need some minimal level of prior experience with collaborative work and with problem-solving before a very process-specific support such as offered by NTool can be effective. In other words, the prior experience of the current population with collaboration and problem-solving may have proved too large a gap with the individual prerequisites required by successful use of NTool. This would still demand explanation of the observed increase in negotiation activities.

We did find a significant difference in cognitive load after the post-tests, which could indicate increased processing of others’ knowledge. However, common ground being highest in the Idiosyncratic groups may actually indicate that the participants from Idiosyncratic groups may have performed processing during collaboration that participants from the Stringent groups could not do before the post-tests. Overall, these results would then indicate that NTool indeed affected negotiation as expected, but that this influence was limited to surface aspects of interaction, and did not co-occur with increased processing of other’s knowledge. As it is, they may reflect the fact that NTool only influenced the communication at the surface level in the present study. That is, NTool tells the learner what a verification is, and the learner makes one, and we assume that this activity will have a cognitive substrate in the guise of actively reflecting on others’ contributions and one’s own perspective. Instead the cognitive substrate of making a verification message may have been the act of making a message comply with the given description of what such a message looks like. In other words, the participants may have been busy wording their messages as verifications and clarifications, instead of being busy actually verifying their understanding, and clarifying their intentions.

The above also has some implications for the use of coercion. If some type of learning support, a formalism, supports or facilitates task aspects that require unmet individual prerequisites on the part of the learners, then such a formalism may not have the ability to have any beneficial effects on the learning process. In that case it would not matter, from the perspective of the learning outcome, to use coercion or not, because the coercion only applies the formalism with more strength, it does not make understanding it easier or harder. From a disruption of communication perspective, however, it may actually be detrimental. In sum, when learners have too little prior experience to benefit from the use of a formalism, coercing such a formalism may be detrimental to the learning process. In terms of cognitive load, such coercion would be entirely extraneous to the task.

With regard to psychological safety, our results are similar to the results of other researchers (Chang and Lee 2001; Edmondson 1999; Van den Bossche et al. 2004), but different in that ours were obtained in an ICT-environment instead of face-to-face. Psychological safety positively affected both verification and clarification, as well as negotiation per contribution. This means that a sense of being safe within a team affects the extent to which people explicitly verify their understanding. Possibly, making a verification is a form of risk-taking because the one doing it feels s/he reveals a lack of knowledge or understanding. Making a verification thus can constitute taking a personal risk.

The results are promising with regard to the facilitation of the grounding process, but they also indicate limitations in the applicability of such facilitation. In her study, Barron (2003) showed that interaction is important for problem-solving, and that engaging in each other’s thinking was related to better solutions. The present study has shown that ICT-tools can be used to facilitate such interactions, by using a formalism for negotiation, and coercing the user into following it. Furthermore, it was shown that NTool can be adapted for different populations, ranging from students of secondary vocational education to senior level college students. However, the laboratory results obtained by Beers et al. (2005) were more promising than the current results. The ultimate implementation of a tool like NTool should therefore carefully be weighed against the expected benefits, and the capacities of the intended audience. More research is needed to obtain guidelines for ‘tweaking’ NTool so that it is better adapted to secondary vocationally education.

More research is required to test our ultimate aim of facilitating collaborative learning and complex problem-solving. The present study does argue a relation between common ground and the quality of problem solutions, but does not explicitly measure it. With regard to support of learning processes, the current results do not offer sufficient proof that NTool indeed influenced learning as intended, nor did the study include measurements of learning performance. However, this is also due to the fact that the grounding framework was mainly used as a prescriptive theory for the design of NTool, not as a theory to be tested. To indeed test the relation between grounding and learning, and the ways the two might be mutually reconceptualised, would require either extensive exploratory study of grounding and learning processes in collaborative learning groups, or conceptual comparisons of theories of collaborative learning and grounding.

It would seem that NTool, when used to teach about grounding and collaboration, ought to be used with groups that have more experience in working together and solving problems. Overall, it can be concluded that NTool and its underlying framework affect negotiation of common ground, and that adding some coercion increases this effect. However, one should be careful with the specific task and audience before implementing NTool, because otherwise the effects may be limited to surface aspects of communication.