Employing Eye-Tracking Technology in Researching the Effectiveness of Recasts in CMC

Abstract

This chapter reports on an exploratory study, which asked whether eye-tracking technology could help determine what learners attend to in an L2 SCMC task-based learning environment. Eight learners of English completed a two-way information gap with a native speaker of English. The learners’ eye movements were captured using a Tobii 1750 eye tracker. Learner eye movements immediately following a recast by the native speaker were evaluated for duration of eye fixation on the recast. Instances of learner uptake following recasts were also calculated. Results suggest that learners notice just over 60% of intensive recasts. The results also show that lexical recasts were much easier for learners to notice, retain, and use productively in subsequent chat interaction than were grammatical recasts. Further, learners were able to use the noticed lexical recast items more accurately than the noticed grammatical items on the post-test writing measure. Successful uptake ­following recasts was found to occur rarely. Finally, the data suggest that methodologically it may be problematic to consider learner uptake a strong measure of noticing. A call is made for follow-up studies using eye-tracking technology in the exploration of noticing, especially those which seek to correlate the eye-gaze ­measure of noticing with other, more established measures of noticing.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

6.1 Introduction

Text-based synchronous computer-mediated communication (SCMC) has been referred to as a “lean” medium—one of its clearest contrasts to face-to-face communication. This leanness requires learners to signal communicative trouble more explicitly in SCMC through linguistic material and typographical signs, since prosodic and paralinguistic markers present in the face-to-face mode used to indicate communicative trouble (e.g., segmentation, intonation, and stress) are not available for use in text-based online communication (Ortega 2009). At the same time, ­computer technology affords researchers in educational linguistics and related fields avenues to capture and analyze learner interactional data in highly effective ways. Nevertheless, much SCMC research fails to capitalize on many of these affordances. What is needed is for researchers to explore more innovative methodological approaches in capturing SCMC data. It is from this perspective that the following study employs eye-tracking and screen capture technology to explore the relationship between recasts, noticing, and performance.

One aspect of learner communicative interaction which has witnessed intense interest in recent years, where we might expect some meaningful differences in online versus face-to-face interaction based on this issue of “leanness,” is in the area of negative corrective feedback, specifically recasts. To date, however, though the discussion surrounding corrective feedback in SCMC environments mirrors that in the face-to-face literature in that we have switched from addressing whether such feedback works and have shifted to examining what kind works best (Ellis 2007); there is still little data-driven consensus on the utility of recasts in the SCMC environment. It seems that much of the difficulty in arriving at a more stable view on the role of recasts may be due to the impoverished nature of the SCMC data that researchers have been satisfied with.

6.1.1 Using “Alternative” Data Sources in CMC Research

Calls for employing richer and more valid CMC data collection techniques have been offered and answered by several researchers over the past few years (Lai and Zhao 2006; O’Rourke 2008; Smith 2008; Smith and Sauro 2009; Smith and Gorsuch 2004). Together, these studies make a compelling case for the view that it is now time to move beyond the practice of relying on the “impoverished picture” that chat logs paint of the SCMC experience for users; we should instead integrate these additional data sources to a certain degree as a matter of course in CMC/SLA research.

Several years ago, Smith and Gorsuch (2004) argued that relying purely on text-based chatscripts when interpreting task-based SCMC discourse is unsatisfactory in many ways. In their exploratory investigation of task-based, meaning-focused SCMC, they found that additional information in the form of video and audio records integrated with a screen capture dramatically altered their initial interpretations of what learners do in SCMC. They suggest that claims about the occurrence of certain interactional moves and strategies should rather be based on information gathered from more than simple traditional chat text logs of the interaction. They argue that not to do so requires researchers and consumers of that research to infer too much. Smith and Gorsuch showed that this more dynamic approach provided a greater precision in understanding what participants were attending to through the record of their verbalizations, scrolling behavior, and their facial expressions. Additional interpretations regarding learner output, strategy use, negotiated interaction, and pragmatic moves were also afforded by this approach.

In a survey of the recent research on tracking student behavior in Computer Assisted Language Learning (CALL), Fischer (2007) argues that without knowing what students really do when they use a particular program, CALL researchers and developers run the risk of operating in a theoretical vacuum. How, for example, can we begin to evaluate claims of the effectiveness of certain software components unless we know whether or not students use them? Indeed, Fischer demonstrates that there is very often a poor correlation between students’ reported and actual use of specific CALL program components. That said, it is clear that although tracking techniques can tell us what students do, they cannot tell us why they do it. Thus, the right technique and measure must be matched to the appropriate questions.

O’Rourke (2008) argues strongly against the overreliance of output logs in interpreting chat interaction and suggests incorporating keystroke logs, video screen capture records, eye-tracking, and conventional video recordings of the user’s physical environment to enhance the richness of the SCMC data collected. Echoing O’Rourke, Smith (2008) examined the nature of CMC self-repair in the task-based foreign language CALL classroom. Chat data were evaluated first by using only the chat log file; and second by examining a video file of the screen capture of the entire interaction. He found that the results led to a fundamentally different interpretation of the chat interaction, which varied according to the data collection and evaluation methods employed. Employing a combination of screen capture video files, chat logs, and a customized coding scheme, Smith and Sauro (2009) examined the relationship between interruptions which occur while one is typing a chat message (referred to as incursions) and deleted text as well as the effect such incursions have on the subsequent output produced by learners. Results showed that incursions by the interlocutor during the message construction phase seem less and less likely to lead to a deletion the further into a “message in progress” this incursion appears. Further, in connection with self-repair, there was some evidence that learners created more linguistically complex output when this self-repair was self- rather than other-initiated.

In the spirit of capturing as dynamic a record as possible, the current study employs chat logs, screen capture and eye tracking technology to better account for what learners produce textually as well as what they seem to attend to in the input and their own output. The purpose of this paper is to simultaneously explore a currently debated theme in the field of SLA (the role of recasts) while employing new methodological techniques for capturing and examining these data (eye tracking and screen capture technology), all within a computer-mediated communication setting. Before launching into the details of this study, some discussion of the role of recasts is in order.

6.1.2 Definitions of Recasts

Recasts are a type of implicit negative feedback, which have been argued to facilitate SLA. Though there are several competing definitions of recasts in the SLA ­literature, they are essentially discourse moves (by an interlocutor) that rephrase a learner’s utterance to be more target-like by changing one or more sentence components while still retaining its central meaning (Trofimovich et al. 2007). Sheen (2006) defines recasts as a teacher’s reformulation of all or part of a student’s utterance that contains at least one error within the context of a communicative activity in the classroom.

Recasts have been shown to be one of the most common types of corrective feedback in the L2 classroom (Braidi 2002; Lyster and Ranta 1997; Oliver 1995; Sheen 2004) and are widely viewed to promote SLA in the context of meaningful interaction (see Mackey 2007 for recent work in this area). Recasts are believed to be powerful since they simultaneously provide both negative feedback and positive input (Leeman 2003). They may occur in reaction to a breakdown in communication—in which case they may be embedded in a negotiation routine of some sort—or they may be provided by the interlocutor (in response to a learner’s non-target-like utterance) without any true non-understanding having occurred.

Though several recent studies have reported limited or no effects of recasts on SLA (Lyster 2004; Ellis et al. 2006; Loewen and Erlam 2006; Sauro 2009), a large and growing body of evidence largely supports their potentially facilitative effect on SLA. The pedagogical context (Ellis et al. 2001; Nicholas et al. 2001; Oliver and Mackey 2003; Sheen 2004, 2007) in which recasts occur as well as the nature of the recast itself have been shown to be important intervening variables in the effectiveness of recasts (Loewen and Philp 2006; Sheen 2006; Tarone and Bigelow 2007). For example, Sheen (2006) reported that more explicit recasts, that is, those that were shorter in length, introduced fewer changes, and involved lexical or phonological errors rather than other types of errors, led to higher levels of learner uptake. Loewen and Philp (2006) found that short, interrogative recasts which targeted a single error were predictive of learning whereas Tarone and Bigelow (2007) provide evidence that multiple recasts of the same target item are noticed more readily than those provided only once.

Recasts have been shown to be beneficial for improving learner grammar. Doughty and Varela (1998) found that intensive recasts directed at simple and conditional past tenses led to interlanguage development. Examining whether computer-delivered oral recasts could facilitate the development of linguistic accuracy and increase the production of modified output, Sagarra (2007) found that such recasts improved learners’ development of grammatical accuracy in written tests and oral face-to-face interactions as well as the subsequent production of modified output. Generally speaking, however, the evidence in support of recasts suggests that they are more effective for lexical items than for grammatical items. There is evidence that learners first focus on (and arguably notice) the semantics of interactional exchanges and only later on form (Mackey et al. 2000; Tarone and Bigelow 2007).

6.1.3 Noticing

The argued positive effect of recasts is very often tied to the construct of noticing in the L2 literature, which has been argued to be a prerequisite to L2 learning (Schmidt 1993). Indeed, noticing has been shown to be a critical factor that mediates L2 input and interaction driven learning (Gass and Varonis 1994; Long 1996; Gass 1997). Current research suggests that factors affecting noticing of recasts include the nature of the recast itself (Mackey et al. 2000; Long et al. 1998; Trofimovich et al. 2007), learner proficiency level (Ammar and Spada 2006; Philp 2003; Mackey and Philp 1998), and working memory capacity (Mackey et al. 2002; Sagarra 2007; Tarone and Bigelow 2007). Generally speaking the research suggests that learners seem to be more able to notice lexical recasts than grammatical recasts. This is likely the explanation for the higher “effectiveness” of the lexical recasts over grammatical recasts discussed above. Clearly, then, in this line of research it is essential to operationalize how one measures noticing.

6.1.3.1 Measures of Noticing

Though there is no real consensus on how best to measure noticing, two methods have been employed most widely. One is the analysis of immediate uptake in the discourse (Braidi 2002; Lyster and Ranta 1997; Mackey and Philp 1998; Tarone and Bigelow 2007). Uptake was initially identified by Lyster and Ranta (1997) as a potentially good indicator of noticing. The other well established strategy for measuring noticing is the collection of introspective data via stimulated recall (Egi 2007; Gass and Mackey 2000; Mackey 2006; Mackey et al. 2000, 2002). This notwithstanding, researchers have used many other approaches to measure noticing including think aloud protocols (Sachs and Suh 2007), immediate reports and cued immediate recall (Egi 2007; Philp 2003), learners’ comments via online journals (Mackey 2006), modified output (McDonough 2005) and questionnaires (Mackey 2006). Other approaches to measuring the effectiveness (and therefore indirectly the noticing) of recasts include the use of some immediate or delayed productive measure (Doughty and Varela 1998; Leeman 2003; Long et al. 1998; Trofimovich et al. 2007).

6.1.3.2 Synchronous CMC Studies on Recasts and Noticing

When considering recasts in computer-mediated communicative (CMC) environments we are normally considering written recasts rather than oral recasts, though there is an increasing amount of research on recasts provided in a voice chat environment (see, for example Heins et al. 2007; Satar and Özdener 2008). The beneficial potential for written CMC recasts mirrors the argued benefits of chat interaction in general, which includes more processing time (Payne and Whitney 2002; Pellettieri 1999; Shehadeh 2001; Smith and Gorsuch 2004) and, by extension, increased online planning time during these “conversations in slow motion” (Beauvois 1992). SCMC also holds “particular promise” for the learning of especially complex or low-salience forms due to the argued heightened visual saliency of these forms afforded during written interaction as well as the enduring, as opposed to ephemeral, nature of the text produced. Together, these differences put SCMC in the form of text chat at an advantage for encoding recasts in ways that facilitate cognitive comparison (Sauro 2009). Just how often recasts occur in a SCMC pedagogical context and certainly their effectiveness, however, is still not at all clear.

Iwasaki and Oliver (2003) found that there was less negative feedback (including recasts) and less uptake in an online chat context than in face-to-face verbal interactions. Loewen and Erlam (2006) reported a total of eight cases of uptake following 89 recasts, or a very low rate of 9%. In perhaps the most compelling study of this sort, Lai and Zhao (2006) compared recasts provided on similar SCMC and face-to-face tasks in their study with ESL dyads. They reported low amounts in the SCMC mode overall, with a total of only 17 recasts produced by their six dyads combined. Of these, about 78% of all recasts in their online chat condition were grammatical in nature (morphosyntactic and sentence fragment reformulations), with only 17% targeting lexical items. Though they did not distinguish between noticing of each type of recast, their overall (combined) noticing rate was about 10%. One possible reason they offer for this relatively low rate was that nearly half of the recasts were “non-contingent” in nature, with an average of three to four turns in between the problematic utterance and the recast. Such a time lapse might have made it difficult, if not impossible, for the participants to notice the recast.

Other studies have reported higher amounts of noticing of recasts. Further examining questions raised in Lai and Zhao (2006), Lai, Fei, and Roots (2008) found that 132 of the 290 instances (or 46%) of recasts in the CMC sessions were noticed. Contingent recasts (those that occurred immediately after a non-­­target-like utterance by the learner) had higher degrees of noticing than did non-­contingent recasts. This is an important point for CMC researchers, given the disjointed turn adjacency and split negotiation routines characteristic of SCMC interaction (Smith 2003). Though their study explicitly contrasted contingent against non-contingent recasts, their overall data report that of all the noticed recasts, lexical (word choice) recasts were noticed about 60% of the time whereas grammatical recasts (morphosyntax and fragment reformulations) were noticed 36% of the time. Tudini (2007) notes that the 41 recasts she found in her data showed a high level of immediate uptake (24, or 59%). Sachs and Suh (2007) compared the efficacy of textually enhanced and unenhanced CMC recasts in the development of certain target language forms. They found that though textual enhancement was related to reported awareness and that higher levels of reported awareness showed stronger correlations with post-test performance, there was no direct significant relationship between enhancement and post-test performance. It seems that (unenhanced) recasts alone did not (in many cases) lead learners to report meta-awareness of specifically targeted forms.

Thus, it becomes clear that studies of online interaction have found varying amounts of negative feedback, specifically recasts, though comparison across these studies is difficult since they explore a mix of proficiency levels, chat interfaces, public versus private chat sites, pedagogically versus non-pedagogically-oriented sites/tasks, dyadic versus multi-user (or whole class) interaction, voice versus text chat, and varying degrees of “treatment” duration. In terms of methodology, perhaps the most important issue to keep in mind in interpreting these studies is whether or not the native speaker, interlocutor, researcher, etc. provided systematic and frequent recasts, or whether these recasts were more incidental in nature. Clearly, if the goal of the study is to examine the effects of recasts on learners then the former is indicated, whereas if the goal is to simply explore recasts as “naturally-occurring” interactional phenomena then we would expect the latter.

As Lai et al. (2008) mention, noticing is a crucial condition for the claimed utility of recasts. That is, unless recasts are noticed by the learner, they are of little value. Thus, simply knowing whether or not recasts are likely to occur is of limited value since their likely presence can be manipulated by the teacher or researcher. What is more compelling is determining their potential effectiveness once they do occur. The review of existing studies of recasts and noticing reveal some of the methodological limitations in establishing the efficacy of recasts. These common approaches are essentially retrospective/introspective or product-oriented in nature. A trait that the retrospective/introspective approaches share is that they rely on various sorts of self-report data. The more product-oriented measures are by definition indirect measures of noticing and are largely unable to provide a more direct link between noticing and performance (see Sachs and Suh 2007).

6.2 The Current Study

The present study is an attempt to evaluate the application of an eye tracker—a technology regularly used in educational psychology and reading research—to explore whether recasts are noticed in an SCMC environment. This exploration is in response primarily to limitations in retrospective methods. By tracking the eye movements of learners engaged in SCMC interaction, we may gain more compelling, objective, and concrete process-oriented evidence about what learners attend to in the input rather than simply relying on more indirect and product-related measures of noticing.

6.2.1 Research Questions

The motivation of this study was to apply a methodologically sound and objective measure of noticing corrective written feedback, specifically recasts. The overarching research question asked whether eye-tracking technology could help determine what learners attend to in an L2 SCMC task-based learning environment. The specific research questions are as follows:

• 1. Are intensive recasts noticed by learners?

• 2. If intensive recasts are noticed by learners, are some types of recasts noticed more than other types?

• 3. If recasts are noticed by learners, does this lead to target-like use of the recast item?

6.2.2 Methodology

This study is different from previous studies of recasts in SCMC in that it provides learners with intensive rather than naturally-occurring recasts and isolates recasts from negotiation of meaning. It also provides exclusively contingent recasts. That is, though there was some degree of meaning negotiation that the task elicited incidentally, the few recasts that occurred within negotiation episodes were not included in the data. Also, recasts were provided to the learner during the next turn in all cases. This differs slightly from Lai et al.’s (2008) definition of contingent recasts in that the appearance of any line of text sent by either participant that divides the targeted utterance from the recast is enough to classify the recast as non-contingent. The present study opts for a more lenient coding procedure since recasts provided by native speakers in their next possible turn (see Smith 2003) were considered immediate (contingent). All recasts in the present study were of this immediate sort.

6.2.2.1 Participants

Eight non-native speaker volunteers were recruited for this pilot study. Participants were a diverse group from China (2), Columbia (2), the Czech Republic (2), Japan (1), and Korea (1). As a group, participants had a range of TOEFL scores (paper and pencil) from 497 to 617. They ranged in time spent in an English speaking country from 6 months to 9 years. All were students at the same large southwestern university in the United States.

6.2.2.2 Materials

In order to capture the eye-gaze of participants, a Tobii 1750 eye-tracker was used as the participant monitor. This monitor is outfitted with two infra-red cameras that remotely track the pupil movements of the participants. The Tobii 1750 was connected to a normal Dell PC owned by the researcher. Thus, to participants, the eye-tracking monitor appeared to be part of a normal PC terminal. The accompanying ClearView 2.1.0 software was used to capture and evaluate the raw eye tracking data produced. ClearView allows one to execute a screen capture and superimposes a small blue dot onto the screen to indicate eye focal points at any given time. As a participant looks away from the computer screen, the blue dot disappears; when the participant’s gaze returns to the screen, the dot reappears. In addition to tracking the path of a participant’s eye movements across the screen, ClearView also calculates the duration of each eye-gaze fixation point. That is, it records the precise time, location, and duration of each fixation. The video prompt (described below) was played on an iPod Touch. The chat program used was PSI, an open source, jabber-based cross platform chat client. All chat logs were automatically saved in PSI. It should be noted that the font size for the chat interaction was enlarged to 36 point font. This allows one to see more precisely where the eye fixations occur once playback is initiated. MS Word was used for the post-­treatment writing task.

6.2.2.3 Procedures

The first step in using an eye-tracker is to calibrate the device to each participant. The calibration program asks each participant to follow a blue dot with their eyes as it moves around the screen. The calibration for each individual is saved, which allows individuals to resume work with the eye tracker for subsequent sessions without having to re-calibrate. This process takes about 1 min Learners then viewed a short video clip (just under 3 min), which was a clay animation with sound, but with no spoken language used. After viewing the video clip the researcher activated ClearView’s recording function and instructed learners to maximize the PSI chat program which was already running. Participants ­interacted in English in a synchronous chat environment with the researcher. The premise of the task was that there were three versions of the video and the native speaker had to choose which of the three options the learner viewed. Learners were asked to “re-tell” the story in as much detail as possible in order to allow the researcher (native speaker) to choose the correct video. There were indeed three versions of the video clip, but in actuality these three only differed in the very ending of the video. In this way we were able to ensure that learners attempted to use sufficient detail in their descriptions of the video. The researcher was permitted to ask questions, provide feedback, etc. in order to successfully complete the task. This task structure, then, creates a situation where participants have a shared goal and (the learners) are obliged to share/exchange information with their interlocutor.

Chat time was limited to about 25 min Learners were allotted 1 h to complete all aspects of the session. The researcher provided intensive recasts to learners as errors were made. After the task was brought to a successful resolution, participants were given 15 min to re-tell the story in writing from beginning to end using MS Word. Participants were not permitted to use any outside materials, review the clip a second time or examine the chat log just created.

6.2.2.4 Data Coding

The variables of interest were as follows:

Independent variables:

• 1. Eye fixation/Noticing (categorical)

  • (a) Noticing = eye fixation on the target item of 500 ms or longer

  • (b) No Noticing = less than 500 ms fixation on the target item or no evidence

• 2. Recast type (categorical)

  • (a) Lexical

  • (b) Grammatical

Recasts were defined as episodes in which the interlocutors rephrase a learner’s utterance to be more target-like by changing one or more sentence components while retaining its central meaning and without breaking the flow of the communication.

• 3. Successful uptake (categorical)

  • (a) Yes

  • (b) No

Though generally speaking we may not expect immediate uptake to occur in significant amounts in a CMC environment due to the ability of learners to review what has been written by interlocutors (Iwasaki and Oliver 2003; Loewen and Erlam 2006; Smith 2005), uptake when viewed in a less restrictive way¹ is indeed interesting since it shows attempts to incorporate recent feedback by the interlocutor in a productive way.

Dependent variables:

• 1. Noticing (categorical)

  • (a) Yes

  • (b) No

• 2. Accuracy of use of target item (on post-task writing sample) (Scale)

Percentage of target-like use (from 0 to 100) on each recast item from the chat session.

Note: targeted items that were not attempted by learners on the post-task writing sample were not included in the data. Therefore, a score of 0% (0.00) means attempted but used in a non-target like fashion in each attempt (if more than one).

One of the output text files the ClearView software produces shows the exact time and duration of each fixation. These fixations are measured in milliseconds (ms). Though there does not appear to be an “industry standard” for what constitutes a meaningful eye fixation, much of the psychologically-based reading research suggests that fixations shorter than about 250 ms are of little interest. Since the present study is largely exploratory in nature—seeking to simultaneously examine the role of recasts in SCMC interaction and advance the methodology for exploring recasts in this environment—the conservative bar of 500 ms was set as the minimum threshold for counting a fixation as such. Further, fixations of this length or longer were counted as instances of noticing as long as they occurred while participants were actively engaged in the chat interaction part of the task. To this end, a two step process was used to establish the legitimacy of each eye fixation. First, the text output was examined and fixations were sorted in descending order of length. Those which were 499 ms or shorter were not considered further. Those which were 500 ms or longer in duration were checked against the video file of the chat interaction, which shows the path of a learner’s eye gaze through trails and each fixation in terms of a blue dot superimposed on the screen. This blue dot grows larger as the duration of the fixation grows longer. That is, all legitimate eye fixations appeared on both the text output (indicating the precise length and location) and the video file. Those “fixations” that occurred before or after the actual chat task were not considered further. This two-step procedure allows one to quickly zero in on those eye fixations of interest (the longer ones) without having to replay in real time each learner’s ClearView video file from start to finish. Once the approximate location of each fixation is established, one may play the video file from just prior to this point to determine the nature of the eye fixation, for example, whether it may have occurred in reaction to an immediately preceding recast or some recast item from earlier on in the chat (both are considered here).

Table 6.1 shows a segment of the ClearView text output pre-sorted by fixation length. The first entry under fixation is number 812 (out of 1,352 total for that learner). This was also the longest fixation (2.376 s) registered by ClearView for that learner. The time stamp column shows precisely when in the video file this fixation occurred (404,447 ms or about 6 min 45 s into the interaction). Though there was no pre-test in this study, those non-target-like words and phrases that were used by learners and subsequently recast by the native speaker were counted as “unknown,” thus allowing the accuracy of these “target items” to be compared against subsequent use in the chat interaction as well as during the post-chat writing story retelling. In cases where the post task writing yielded more than one attempt at a recast item from the chat interaction, the total number of target-like uses was divided by the total number of attempts. This yielded a ratio between 0 and 1.00 for that specific item for that particular participant. Finally, instances of successful uptake were coded following Smith (2005). Successful uptake is uptake in which a student repairs a linguistic feature to be target-like. By capturing and coding the chat data in this way, it is possible to show the relationships between recasts, noticing, uptake, and accuracy of subsequent production.

Table 6.1 Sample of output data from ClearView

6.3 Results

The chat transcripts yielded 61 recasts total. Accordingly, there were 61 potential instances of eye fixations (noticing) following recasts, and also 61 potential occurrences for uptake (since uptake as it is defined here is always possible in CMC). There were only a total of 44 attempted uses of a recast item in the writing sample, however. That is, not all items which were recast resulted in use or attempted use by learners. Given the small sample size (n = 8) of this exploratory study, inferential statistics is not warranted. Further, though there were 61 instances of recasts (and potential noticing and uptake), these instances were not evenly dispersed among the participants. This, along with the fact that we cannot consider each of the 61 (or 44) observations as “independent,” also complicates the analysis. Accordingly, descriptive statistics alone are presented below.

With respect to the first and second research questions, 60.7% (n = 37) of all recasts were noticed in the SCMC environment as measured by fixations of 500 ms or longer. Just under 40% (n = 24) of recasts resulted in no noticing. Table 6.2 shows a breakdown of this same data by the independent variable of recast type (lexical or grammatical). Of the 61 recasts, over 80% (n = 49) were grammatical and just under 20% (n = 12) were lexical in nature. Of the lexical recasts 75% were noticed by learners (9/12) as opposed to only about 57% (28/49) of the grammatical recasts.

Table 6.2 Recasts noticed by type

With respect to research question 3, Table 6.3 shows the mean accuracy score (dependent variable) for each type of recast item (independent variable) across the condition of noticing (independent variable). Since the dependent variable in this comparison is accuracy score, only those recasts which were also attempted in the post-task writing sample are included in this analysis. The overall proportion of lexical to grammatical recasts is about the same as in Table 6.3, as might be expected.

Table 6.3 Accuracy of noticed/unnoticed recasts by type

At least three things are immediately evident from these data. First, we notice that all lexical recasts that were subsequently attempted in the writing sample were noticed by learners, while those that were not noticed were not later attempted. Second, among those lexical recasts that were noticed during the chat interaction and that were attempted in the post-task writing sample, the target-like use percentage was about 89%. That is, when learners attempted to use these lexical items in the story retelling task, they used them correctly about 89% of the time. This contrasts sharply against those grammatical recasts which were noticed in the chat interaction. For these recasts, learners used them correctly only about 67% of the time—not much better than the 61% success rate for those grammatical recasts that were not noticed. Finally, the similarly low post-test means for the noticed and unnoticed grammatical recasts is interesting itself, but when we consider this ­number in conjunction with the relatively high percentage of grammatical recasts noticed in this comparison (almost 50% of the total) we see that though grammatical recasts were likely to be noticed, this did not translate into successful productive use of the same. At least not to the same extent as “noticed” lexical recasts. A discussion of these points follows in the next section.

Table 6.4 shows the relationship between noticing and successful uptake. As might be expected, all cases of successful uptake of recasts in the data (n = 12) were also noticed by learners during the chat interaction. Where there was no successful uptake (this includes where there was no attempt made at subsequent use in the chat interaction), we see an evenly split degree of noticing. Again, as expected, the majority of recasts (noticed or not) resulted in no uptake. In considering the evenly split “no uptake” condition, it seems problematic to consider uptake to be a strong measure of noticing. If this were the case, we might expect a markedly higher percentage of “No fixation” relative to “Fixation” which is not the case. This suggests that noticing (as it is measured here at least) is not necessarily related to successful uptake.

Table 6.4 Uptake and noticing

The successful uptake data from Table 6.4 are further delineated in Table 6.5 according to attempted use in the post-chat writing task. Here we see that though relatively few in number, those recasts that showed successful uptake in the chat interaction had a mean score of 83% correct when they appeared in the post-task writing sample, whereas those where no uptake occurred had a mean score of 66% correct.

Table 6.5 Uptake and accuracy of recast use

Table 6.6 examines this same data across the independent variable recast type. From this table, we see that noticed lexical recasts (since there were no cases of “unnoticed” successful uptake) which witness successful uptake show a substantially higher post-task accuracy score than do noticed grammatical recasts. Of particular interest is the modest advantage for noticed lexical recasts in the successful uptake condition over the no uptake condition. Also of interest is how narrowly distributed the lexical recast (+/− successful uptake) mean scores are relative to the grammatical mean scores. Again, though the numbers are modest, the combination of the data in Tables 6.4–6.6 seem to suggest that though the occurrence of successful uptake is in the direction one might expect, systematically increasing alongside higher mean scores, it is not likely to be a causal variable in influencing mean scores, but rather it correlates nicely with these scores.

Table 6.6 Uptake and accuracy of recast use by recast type

6.4 Discussion and Conclusion

The data seem to suggest that learners fixate on lexical recasts much more than grammatical recasts. Consistent with previous research on recasts, it seems that lexical recasts are much easier to notice, retain, and use productively in subsequent chat interaction and on the post-task writing sample than are grammatical recasts. Also consistent with previous research, successful uptake (when uptake was possible) was found to occur rarely—appearing only 20% of the time. Further, uptake alone seems to be a poor measure of noticing. Overall, noticing resulted in successful uptake about one-third of the time (see Table 6.4). As one might expect, all cases of successful uptake were also noticed by learners. However, there is no clear evidence that items where no successful uptake occurred were not noticed. Indeed in instances where recasts did not result in successful uptake, there were about equal amounts of noticing versus no noticing.

The real interest in exploring the relationship between recasts and noticing is determining whether recasts, once noticed, have some beneficial effect on SLA. Tables 6.5 and 6.6 show that although (noticed) recasts that resulted in successful uptake had higher productive scores than did those recasts that did not result in successful uptake, we see that the mediocre mean scores for those that were grammatical in nature strongly weigh down this overall mean. This suggests that noticed lexical recasts, which also result in successful uptake, are easier to process and retain at least in the shorter and middle term. The large standard deviations for grammatical items reported in Table 6.6 may indicate, however, that this ability to retain such grammatical information is highly variable across individuals. Follow-up research should explicitly examine this point, perhaps beginning with the variable of working memory capacity (WMC). Those with a higher WMC may have an advantage for retaining grammatical information of this sort over those with a lower WMC. In contrast, lexical items may be less complex and, therefore, such WMC differences may not affect the accuracy of subsequent productive use of these ­lexical items. Finally, Table 6.6 shows that “No Uptake” lexical items had a higher mean score on the productive measure than did grammatical items where uptake did occur. This further calls into question the notion of successful uptake itself as playing a significant role in SLA. What we can say from these data is that where one has evidence of successful uptake, we can be reasonably sure that these same items have been noticed by learners and that these items will be subsequently used with slightly more accuracy than their no uptake counterparts.

6.4.1 The Methodology

Using eye-tracking technology provides us with a clear and precise aspect of learner interactional data that has not typically been captured and analyzed in CMC/SLA studies (for a rare exception see O’Rourke 2008). Specifically, it seems to provide a good indication of noticing (at least at some level) as reflected in the post-task written productive measure. The most useful types of output generated by the ClearView software in this study is found in the video files, which show real time screen capture along with the path (saccades) and eye gaze fixations (see Fig. 6.1), as well as the text output file that shows the time and duration and spatial location (on the screen) of each fixation. The text output file alone, though perhaps quite useful for advertising or usability testing where the screen image is much more stationary, is limited in its usefulness for examining interactional chat data without the screen capture file. It does help us quickly narrow down potential points of interest on the video file, but in chat interaction one needs not only a measure of duration but also an indication of multiple fixations on a given point of text such as a morphological ending, lexical item, etc.

Fig. 6.1

Eye tracking saccade and eye fixation

6.4.2 Limitations

This pilot study was a principled exploration of the efficacy of using eye-tracking technology to explore chat interactional data from a cognitive interactionist theoretical perspective. An obvious limitation is the very small sample size, which precluded any typical inferential statistical analysis of the data. However, even with such small numbers certain trends seem to emerge from the data, which perhaps warrant a closer look.

Since this study seems to be the first principled application of eye-tracking technology in SLA research of this sort, many decisions regarding coding needed to be made. For example, though not arbitrary, the cut off length of 500 ms for considering a fixation as evidence of noticing may be too short or too long. With little guidance in this area one way to better establish this cut off may be to carefully correlate such a measure of noticing with other sources such as think-aloud protocols or stimulated recall. It is precisely this need to correlate the eye-gaze measure of noticing with other, more established measures that is a first step in legitimizing this approach as a tool in future inquiry into noticing and SCMC.