Keywords

1 Introduction

Joint problem-solving can be a difficult matter and the mere joining of people’s forces does not suffice unless people know how to collaborate. Hence, learning to learn and to work together must become an important goal in education and professional training. To support this enterprise, the design research project ORBIT [1] has developed the interactive tabletop (ITT) activity “Orbitia” that aims at developing collaborative conduct among participants [2,3,4]. Collaboration is understood here as “a coordinated, synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem” [5, p. 70].

In this paper we provide a detailed account of how a group of three participants jointly solve a problem in the Orbitia environment. In our case study we analyze “the situated processes that take place at the group level of description” [6, p. 542]. Thereby, we intend to develop a better understanding of the situated processes of problem-solving at a group level of description at an ITT-mediated activity and to gain design knowledge about inducing such episodes.

2 Orbitia

Fig. 1.
figure 1

Annotated screenshot of ‘Orbitia’. Complementary responsibilities are distributed among the crew members: steering directions; damage control, mining & energy.

Full size image

Orbitia is designed as a serious game where adult participants are acting as a space-mining crew on an imaginary planet (Fig. 1). The crew must collect minerals with a rover and rely on a camera drone for reconnaissance. Relying on pedagogical and design principles, several devices have been developed and implemented to induce collaborative conduct [3]. The relevant rover-steering-device (RSD) here consists of a rover with complementary control buttons distributed over three control stations (Fig. 2a). Two directions are missing and can only be taken through the composition of two other directions. Figure 2b shows the two ways to move north by one square, which becomes relevant in the analyzed extract.

Fig. 2.
figure 2

RSD: (a) Distribution of directions (missing ones in red); for reasons of convenience the directions are represented by (inter)cardinal points (b) two ways of moving north by one square.

Full size image

3 Joint Problem-Solving

In general, one speaks of a problem when there is a discrepancy between a current and a goal state, there is no ready-made solution [7], and the problem is considered worth solving [8]. Problem-solving then corresponds to finding the unknown [8] and can be undertaken as a joint endeavor. In the latter case, “to make sense of a problem together”, participants must establish “common frames of reference”, resolve “discrepancies in understanding”, negotiate “issues of individual and collective action” and come to “joint understanding” [9, p. 404–5].

Unlike many other studies in this field, “we are not oriented toward theoretical hypotheses, statistical generalizations, individual mental representations, or sociocultural influences”, rather we attempt to “understand the situated processes that take place at the group level of description in actual case studies” [6, p. 542]. To do this, we rely on an ethnomethodological conversation analytic (EMCA) inspired approach [10]. It has been shown that the latter approach has proven to be particularly suitable to address issues of design and collaboration [6, 11].

4 Setting and Study Design

The here analyzed excerpt is taken from the dataset of the first phaseFootnote 1 of the project ORBIT. We considered the video recordings of the pilot study and three subsequent trials (14 group sessions lasting 30 to 40 min) and searched the data for moments where the participants explicitly discussed and reasoned among themselves on how to proceed. This frequently happened when they had identified a problem and were trying to solve it. In line with an emic approach (characteristic of EMCA), we interpret an incident as a problem when participants do so. Usually, one participant either verbally refers to an incident as a problem (for example, “we have a problem”), states the problem (for example, “so we don’t know basically what is here”), or utters a response cry (for example, “oh shit” or “oh my God it’s hard”), and at least one other participant then acknowledges the problem (for example, “ok”). So, the problem-solving episodes were not selected according to criteria “external to the interaction” but were “grounded in the discourse as structured by the participants” [6, p. 553]. We further found that 9 out of 20 episodes were related to the missing directions feature of the RSD (Fig. 2).

Since the RSD is one of our ‘flagship’ design features [3], we focused on the identified instances to investigate how it induces collaborative conduct by reconstructing how participants interactionally accomplish doing joint problem-solving. This means tracking “in detail the various conversational strategies and devices which inform and drive its production” [12, p. 114]. According to conversational analysis (CA), communication is sequentially organized. Sequences are ordered series of turns through which participants accomplish and coordinate an interactional activity. The most common type are dyadic adjacency pairs uttered by two different speakers producing one turn each; for example, a question creates a conditional relevance for an answer [13]. The “understanding of the preceding turn displayed by the current speaker” then constitutes “the basis for any other type of intersubjective understanding” [14, p. 156].

To investigate participants’ problem structuring, Stahl [6] relies on EMCA and introduces, among others, the concept of ‘discourse move’. Each move includes a base adjacency pair, which drives the interaction (several utterances of secondary structural importance may, however, introduce, interrupt, or extend the base pair) and is part of a longer chain of moves to accomplish discussing the conversational topic. In the here analyzed case, the topic corresponds to finding a way to reach a destination not directly accessible through the operation of one single steering command.

5 Analysis

The analyzed episode takes place about 3 min after the beginning. During this time, the three participants (Ava, Viv and JoeFootnote 2) have explored Orbitia. In particular, they have discovered some of their competencies regarding the different control stations [2]. They now jointly designate a destination for the rover and decide to go there. More precisely, Ada suggests a potential destination for them (“onFootnote 3”) by pointing towards the visible mineral icon. Both Joe and Viv agree. Furthermore, Viv identifies and declares herself as the competent participant (“I”, li 03), which is validated by Ada. In other words, they formulate and agree on a common project (reaching the square with the mineral). However, things will not proceed accordingly, as we are about to see.

figure a

5.1 Problem-Solving Move 1: Signaling a Problem and Accounting for It

Viv pushes her straight arrow (S) twice. The rover moves accordingly and overshoots the targeted square with the mineral (fg. 2). This event is witnessed by all parties and triggers three overlapping reactions. First, Viv produces a response cry (“oh shit”, li 07), followed by an account of her exclamation resp. Action (“I was too fast”, li 07) [15, p. 693). By uttering this imprecation, she produces a trouble alert signaling that there is some sort of problem. Through this elaboration, she links the trouble to something she previously did and went wrong (self-blame). Second, Joe acknowledges that something occurred (“o.k.”, li 09); he reports what happened on the screen (“it was two times”, li 09) and points to a consequence regarding his competency as the one overseeing energy (“it has taken way too much energy”, li 10). So, he reads Viv’s response cry as signaling trouble. However, by relying on an impersonal formulation (“that”, li 09) and by referring to the rover (“it”, li 10), he does not sanction VivFootnote 4. Rather, he produces a factual description (“that was two times”, li 09), and an interpretation of what could be problematic here: the rover’s excessive energy loss (li 10). Third, Ada starts smiling after Viv’s self-declared mishap, thus acknowledging that something noticeable occurred.

figure b

So, they all concur that there is some kind of trouble. However, there is no shared understanding of the ‘nature’ of the problem yet. Viv causally links the trouble to her ‘clumsiness’ in the past, Joe highlights a consequence, and Ada does not utter an interpretation. Viv then chooses not to expand Joe’s topic of excessive energy loss but restates her position (oversteering). So, the driving move here consists of calling out a problem and – beyond acknowledging – accounting for it. Next, we shall see how they start grasping the problem.

5.2 Problem-Solving Move 2: Formulating the Problem and Converging on a Solution

Here, Viv provides a formulation of the problem: “how do we move back” (li 12). Moreover, beyond specifying the problem (“how” to do it?), her phrasing also contains a solution (“move back”) and points to its distributive character (“we”), meaning they must all participate in solving the problem. In light of this statement, her wandering gaze (transcript 2, li 11) can be read as a lookout for the competent participant (who is the one who is able to move back). Both Ada and Joe acknowledge Viv’s problem formulation by repeating it simultaneously, and their laughter further displays a shared interpretation of the problem situation.

figure c

Viv announces the proposal of a joint solution (li 18) and lays it out by performing circling movements with her right hand, first in the vertical (li 18) and then in the horizontal plane (li 21, fg. 3a-3c). So, she accounts for her understanding of the designed constraint that there is no direct steering option to directly move towards her (i.e., “back” from her perspective both literally and in the sense of undoing her previous ‘mishap’) and gesturally suggests a ‘workaround’ solution (by roughly pointing out a route for the rover). Ada also announces a suggestion (“maybe”, li 19) and then verbalizes Viv’s embodied proposal (“we have to go around”, li 19). After displaying insight (“oh yes”, li 20) Joe also aligns himself with the proposal. Finally, Ada confirms her endorsement of the workaround solution (”we have to go around”, li 22) and verbally emphasizes the location (“there”, li 22).

figure d

So, the participants jointly announce and ratify a solution. Ada’s formulation carries the insight that the solution cannot be implemented by one participant alone and consists in a workaround. Indeed, the option of moving back directly is (by design) not available. The emerging solution of “moving back” (li 11–15) is multimodally transformed here into “going around.” Thus, the problem-solving move we witness here is composed of posing the problem (‘how to move back’) and proposing a first approach to the solution (‘going around’). Now, we will see how they begin to implement and specify the solution.

5.3 Problem-Solving Move 3: Seeking and Identifying the First Agent

While Joe starts objecting (li 23), Viv requests the information on who can move forward (li 24). While turning her head towards Ada resp. her station and pointing toward the target location for the rover (fg. 4a – fg. 4b), she refers to Ada and so provides herself with the requested information in the same turn. Ada overlappingly moves her left hand towards her steering controls and points to the NE-arrow (li 25), which moves the rover to the designated square (fg. 4b). Joe confirms (“yes”, li 26) and points with his finger to ADA (fg. 5). Simultaneously with Joe’s hand movement, Ada moves her finger between her control arrows (li 28, fg. 5).

figure e

Viv’s multimodal utterance (li 24) does the following work: the first step of the solution is formulated (“move forward” & pointing to a target location); the competent participant is sought and identified (“who” - “you” & gaze/head movement). Through Ada’s overlapping action, she self-identifies as competent and accepts the attribution of being the competent first agent regarding movement. The just described action of seeking and identifying the first agent drives the interaction here. Despite this rather straightforward discourse move, Ada is facing somewhat ambivalent displays of direction giving. Viv points to the square that can directly be reached by Ada’s diagonal arrow (NE), and Joe’s straight pointing finger matches her straight-forward steering arrow (E). Ada reacts to this display of conflicting directions by positioning her finger between the two moving options (E & NE), thus expressing some hesitancy while remaining ready for both options.

5.4 Problem-Solving Move 4: Co-instructing the First Agent

In response to Ada’s hesitant conduct, Viv prompts Ada to move forward (li 29). The latter shows no apparent reaction, so Joe multimodally instructs her to proceed (li 30), which is endorsed by Viv (li 32). Ada then asks for confirmation (li 33) and displays her understanding of ‘move forward’ by positioning her finger over the E-arrow (fg. 6). Both Joe and Viv validate (li 34 & 35), at least temporarily.

figure f

So, we can observe here that the meaning of “move forward” is not straight forward. There appears to be some ambiguity here, which slows down the process. Does “move forward” mean moving to the square Viv pointed at in the previous extract (fb. 4), entailing pressing the NE-button, or literally moving forward from Ada’s perspective, entailing pressing the E-button. Thus, the central move here is an instructed action that is not fully completed. A directive (”move forward”) is issued to Ada, and by requesting a clarification, the addressee acknowledges that she is willing to operate the appropriate direction button. However, she does not carry out her part (yet), so the instructed action remains open and pending, which will trigger an evaluation of the procedure.

5.5 Problem-Solving Move 5: Assessing the Solution(s)

figure g

The carrying out of the instruction to move forward remains pending. Ada is not moving “forward.” So, Joe announces an alternative (li 36), but then – regarding Ada’s finger position on the E-button – specifies his previous instruction by indicating the number of moves (“one time”, li 38) twice. Viv initiates a repair (li37) and challenges Joe’s directing by multimodally pointing (li 39, fg. 7 & fg. 8) to the constraint that they cannot move towards her from the position Joe is directing the rover to.

figure h

Joe reaffirms his suggestion (“yes yes”), repeats his previous instruction (“you move forward”), and adds the next prospective step (“and then it’s me”, li 40) to lay out the complete path and address Viv’s concern. Ada reacts to the raised doubt by placing an alternative first move on the table. She moves her ‘trigger’-finger to her second steering button (NE) and inquires whether they do not have to turn left (from her perspective, the NE-arrow can reasonably be described this way). Her option challenge Joe’s instructing and the carrying out of her alternative would take the rover to the square Viv was previously pointing to (fg. 8). Joe shows some puzzlement and attempts to hold the floor (“eh:”).

figure i

Viv now displays insight into Joe’s variant and multimodally takes up his second move suggestion (li 43, fg.10–11). Upon noticing Viv’s beginning hand movement (fg. 10), which parallels his NW-button, and Ada’s hesitation, Joe points out that Ada can operate any of her two steering buttons. So, he validates both options as equally valid (li 45). Nevertheless, Ada still asks for confirmation (li 46), which is granted both by Viv, who expresses a preference by pointing to the NE-arrow (li 47), and by Joe (li 48). When Ada continues to hesitate, Viv prompts her one more time (li 49), and Ada takes up Viv’s previously suggested option.

The outside observer can easily spot the two equivalent bi-step ways to “move-back” (see Fig. 2b) and that the controls to carry out both ways are equally distributed among Ada and Joe. However, the participants must first figure this out and jointly agree on how to proceed. Ada’s display of hesitation leads to an exchange where the two options are multimodally displayed and recognized as being alike. So, both alternatives are on the table, and the designed first operator (Ada) is verbally given the choice, although Viv embodiedly endorses the NE-direction.

In other words, Joe’s announcement of an alternative and Viv’s objection indicate that the first step of the projected solution is put to the test. Viv then negatively assesses a potential second step by pointing out its non-feasibility (the unavailable N-direction). Joe then disagrees with her evaluation and explains why the challenged first step (E) is a valid option. Then follows a series of questioning, disagreeing, suggesting an alternative, and finally, two trajectories (E-NW, NE-E) are validated as equivalent. Now that both solutions are positively assessed, one must be taken up.

5.6 Problem-Solving Move 6: Taking up a Solution

After Ada has moved the rover, Viv validates the option chosen by Ada and induced by herself. She then points to the result (“but that’s it”, li 52) which also does the work of closing the first step (NE) and orienting to the next action [16]. Joe then goes on to identify himself as the competent participant and simultaneously announces and carries out the second move to reach the mineral (li 53). Viv overlappingly confirms this (li 54). After the rover has moved (li 55), she introduces herself as the next competent participant and highlights the next step by pointing to the means (the crane button) to perform it (li 56, fg. 12). Joe validates both Viv’s competency and the next step (li 57). So, Viv picks up the mineral (li 58), Joe and Ada close the whole procedure (li 61 & 62).

figure j

They have now solved the problem of how to move back and concluded their initial project of collecting a previously designated mineral. Note that they do not just carry out the steps to move back in a coordinated and smooth way, but they also announce and substantiate the projected or carried out actions (instructing themselves and one another). By vocalizing their actions, they render the physical actions mutually accountable and establish a shared awareness of ‘moving back’. So, what drives the interaction here, is the actual joint uptake of a previously outlaid solution. This steering maneuver(s) now become(s) a stable resource for their ongoing task accomplishment. Furthermore, the procedure will subsequently be adapted to move back by two squares by combining the NE and the NW steering buttons, enabling faster and more efficient progress if necessary (not shown here due to space limitations).

6 Conclusion

We thoroughly analyzed the multimodally embodied conduct of the three participants engaged in the ITT-mediated game activity Orbitia. So, we reconstructed how they interactionally accomplished jointly solving a problem in situ, or, in other words, we told the ‘story’ of their collaborative endeavor to do so. One participant signaled the problem (“oh shit I was too fast”), and they acknowledged resp. Accounted for it. So, they all became aware that there is a problem, and they could start developing a shared conception of it [5]. They did this by formulating it, and this formulation (“how do we move back”) was also the first approach to an upcoming solution. The competent first agent was sought and identified, and her two co-players started instructing her. However, her hesitation in front of two ‘floating’ options lead to a positive assessment of both of them and an uptake of one.

By means of these six identified discourse moves, they coordinated themselves and constructed and maintained a shared conception [5] of their problem at hand, thus collaborated. That this particular sequential organization took place is “evidence that the machinery for its production is culturally available, involves members’ competencies, and is therefore possibly (and probably) reproducible” [13, p. 50]. The additional instances of problem-solving, we have identified in our video corpus, are currently analyzed, and will contribute to the establishment of a collection of similar patterns of joint problem solving. Since nine instances are related to the RSD, we can reasonably assume that this design feature, especially designed for Orbitia [3], has a solid potential to induce collaborative problem-solving.

Note on Transcription Conventions.

ADA/JOE/VIV: participant speaking or doing the embodied action

  • ITT: action of interface

  • ((pushing the button)) non-verbal conduct

  • ? question

  • [ overlapping talk

  • (0.6) timed pause

  • (.) micro pause

  • fg. frame grab