The effect of communicative purpose and reading medium on pauses during different phases of the textualization process

Abstract

Multiple factors simultaneously interact during the writing process. The effect of these interacting factors must be considered if writing is studied as a dynamic and complex process that constantly changes. Based on that premise, the study aims to determine how the interaction between the reading medium and the communicative purpose of a writing task affects the pauses during different stages of the writing process. At the same time, it seeks to determine how this interaction is related to the writer's competence. Using a 2-by-2 experimental design, undergraduate students (n = 66) read documents (print or digital format) and completed a computer-embedded writing task with different communicative purposes (to persuade or to inform). While writing, pauses-related keystrokes were recorded. The results show that neither the reading medium nor the task purpose or the interaction of these two variables affect pauses. However, communicative purpose interacts with the stages of the writing process, so regardless of participants' writing competence, they took longer at the end of the writing process when the purpose was to persuade. Other interactions indicated that the type of pause interacts differentially with the stage and communicative purpose. These interactions and main effects were systematically related to the writer's competence; because once it was statistically controlled, these interactions and main effects were no longer significant. The results are analysed in terms of the cognitive processes underlying pauses, considering evidence from studies on hybrid reading-writing and integrated writing studies.

Introduction

It is common for students in higher education to use various information sources to learn and acquire skills (Granado-Peinado et al., 2019; Solé et al., 2013; Vega et al., 2013). Students are required to demonstrate, validate, or certify their ability to understand the texts (Bråten & Strømsø, 2011; Plakans & Gebril, 2013) and produce new knowledge, mainly through writing (Chan, 2017; Granado-Peinado et al., 2019). This source-based writing is called integration writing. In it, different factors and sub-processes of reading and writing interact, such as the presentation medium of the sources, the communicative purpose of the task, the forms of instruction, the collaborative work, the thematic convergence of the sources, or the frequency of use of the sources, among others (Granado-Peinado et al., 2019; Mateos et al., 2020; Primor & Katzir, 2018; Rouet, 2006; Segev-Miller, 2007; Vandermeulen et al., 2020). In this way, integration writing can be explained as a hybrid process that emerges from reading comprehension of multiple sources and leads to integration in a single large text (Primor & Katzir, 2018). In this area, the writer must connect the intratextual or intertextual information by highlighting similarities and differences, linking, comparing, contrasting ideas, or formulating new hypotheses (Granado-Peinado et al., 2019; Mateos et al., 2020; Primor & Katzir, 2018; Segev-Miller, 2004).

For this reason, integration writing is not an easy activity. Several studies show that students have problems generating quality integration texts (Escorcia et al., 2017; Mateos & Solé, 2009). Given this, understanding the writing process can provide evidence about where these problems occur, for example, knowing which stages of writing require more time or the periods in which spelling is checked (Alves et al., 2008; van Waes & Leijten, 2015; van Waes & Schellens, 2003). Consequently, analysing the integration writing process is relevant. Moreover, integrative writing is a complex, dynamic and highly recursive activity in which it is necessary to evaluate how different factors interrelate with the writing process (Britt & Sommer, 2004; Granado-Peinado et al., 2019; List et al., 2020; Mateos et al., 2020). Therefore, this study analyses how the interaction between the reading medium (print or digital) and the communicative purpose of the writing task (to persuade or to inform) affects the writing process.

Regarding the reading medium, previous studies show that reading printed texts have a positive effect on reading proficiency compared to the use of digital media (Ackerman & Lauterman, 2012; Clinton, 2019; Delgado & Salmerón, 2018; Delgado et al., 2018; Salmerón & Delgado, 2019). The literature has explained this difference, stating that reading from a screen affects sustained attention and concentration (Delgado et al., 2018; Liu, 2005). In addition, the literature proposes that digital media generate a deficit in metacognitive monitoring and, consequently, more superficial reading (Ackerman & Lauterman, 2012; Delgado & Salmerón, 2021). In contrast, some research suggests that reading print texts fosters a deeper understanding of what is read (Ackerman & Lauterman, 2012; Clinton, 2019; Delgado et al., 2018). Despite these significant findings, one of the limitations of these studies is that they have predominantly used multiple-choice tests to assess reading proficiency (Delgado et al., 2018). In this sense, there is less evidence on the effects of the reading medium on writing tasks and the writing process itself. This knowledge is necessary to understand a hybrid task, such as integrating reading and writing. Perhaps, for this reason, recent research evaluated the differences between media in a hybrid reading-writing task. In this study, people performed better when using printed media sources than on a screen (see Latini et al., 2020). However, although the final written product was analysed in-depth, the writing process was not.

Regarding the communicative purpose of the writing, previous studies have shown that the writing process varies depending on some characteristics such as content, linguistic factors, rhetorical aspects, or information external to the text, such as the source or context (Conijn et al., 2019; Medimorec & Risko, 2017; Ransdell et al., 2002; Vandermeulen et al., 2020). Specifically, it has been pointed out that the communicative purpose impacts the writing process (Conijn et al., 2019; Medimorec & Risko, 2017; Vandermeulen et al., 2020). For example, in several studies, writers were less fluent when writing argumentative text than narrative or informative text (Alves & Limpo, 2015; Beauvais et al., 2011; Medimorec & Risko, 2017). This is because argumentative texts are linguistically complex, with more sophisticated vocabulary and more elaborate sentences, among other aspects (Medimorec & Risko, 2016, 2017). However, it is less known how these communicative purposes interact with other factors present in source-based integration activity.

Based on the previously described, writing is a complex activity in which various factors interact. In order to manoeuvre correctly with these factors, writers must plan, review, monitor, and permanently evaluate while writing. These processes impose a significant cognitive load on attentional and working memory requirements (Alamargot et al., 2007; Kormos, 2012; McCutchen, 1996; Medimorec & Risko, 2017; Ransdell et al., 2002). Writing is not a continuous activity because writers stop writing at certain moments. These periods of inactivity, observable and measurable, are pauses (Barkaoui, 2019). Pauses occur in a specific location and time, and they have a duration that affects the continuous flow of the writing experience. Even though pauses can be considered marginal phenomena during writing, there is compelling evidence that they represent 70% of a person's time (Alamargot et al., 2007; Matsuhashi, 1981).

Pauses provide information about the underlying cognitive processes of writing (Alves & Limpo, 2015; Alves et al., 2008; Chukharev-Hudilainen, 2014; Medimorec & Risko, 2017; Wengelin, 2006). These processes refer to goal setting, concept activation, memory, lexical decision, and higher-level processes such as planning, monitoring, revision, and regulation (Matsuhashi, 1981; Schilperoord, 1996; Spelman Miller, 2006; Wengelin, 2006). Thus, as cognitive load increases, writers may experience a slowing or suspension of a cognitive process. Similarly, the simultaneous activation of specific processes, such as a lexical decision, may lead to the gradual deactivation of others (planning and revision). In both cases, suspension and deactivation may be processes underlying the observed pause (Xu & Qi, 2017). On the other hand, pauses may be intentional in strategy implementation or a deliberate attempt to reduce system overload (Alamargot et al., 2007). Thus, long pauses may be associated with global planning processes, such as paragraph organisation, while shorter pauses may be associated with more specific processes, such as grammatical and lexical decisions (Spelman-Miller, 2006).

In addition, several studies have found that pauses are not expressed regularly or consistently in the writing process. Instead, the frequency and duration of pauses vary throughout the writing process (Barkaoui, 2019; Roca de Larios et al., 2008; van Weijen et al., 2008; Xu & Qi, 2017). For example, Roca de Larios et al. (2008) reported that less proficient writers had a constant time allocation throughout the writing process, whereas more proficient writers showed a more diversified time allocation to different activities during the writing process.

In line with these findings, some studies show that pauses arise from an interaction between the writer's competence and the writing process. For example, second language (L2) studies have shown differences in proficient and less proficient writers (Sasaki, 2000; Xu & Ding, 2014; Xu & Qi, 2017). Sasaki (2000) compared the pauses of proficient and novice writers, finding that proficient writers paused more before starting to write than novices. In addition, novices tended to pause each episode more often than experts, suggesting they had to stop and plan frequently. In contrast, expert writers did not pause to think while writing as often as novices (Barkaoui, 2019). Along the same lines as these studies, Xu & Ding (2014) found that proficient writers tended to pause less frequently and significantly longer in the prewriting stage than less proficient. In another study, Xu & Qi (2017) reported that proficient and less proficient L2 writers differed significantly in their pausing patterns at different intervals in the writing process. Proficient writers tended to pause longer but less frequently than less proficient writers during the first interval of the task. During the second interval, competent writers reported shorter and more frequent pauses, whereas less competent writers showed less frequent but longer pauses, suggesting that their translation process was frequently interrupted by other processes such as lexical processing. Finally, and in line with previous studies, Zhang & Deane (2015) reported that proficient writers’ writing process tends to occur efficiently in longer bursts, and pauses are more likely to occur in places necessary for planning, such as sentence boundaries. On the other hand, writing tends to occur less efficiently in less proficient writers, and pauses appear in places that suggest difficulties within visual-motor coordination, phonological, lexical, and syntactic processing.

Other studies have focused on analysing how the goal or purpose of the task affects pause duration (Olive & Kellogg, 2002). In one of the first studies on pausing, Matsuhashi (1981) found that the average duration of pauses was longer in cognitively demanding writing tasks (to persuade or generalise) than in less cognitively demanding tasks (to report). Longer pause durations in persuasion and generalisation tasks were associated with a greater tendency to make more conceptual revisions than formal ones. Some studies indicate that task purpose differentially affects where pauses appear during writing. For example, Schilperoord (1996) found that different writing tasks had a more significant influence on the length of pauses between paragraphs and, to a lesser extent, on pauses between sentences and clauses, but not on pauses between words and phrases. The occurrence of paragraph-level pauses could indicate planning states. However, pauses at the word and sentence levels do not affect the task goal. Medimorec and Risko (2017), in their study, asked university students to compose narrative and argumentative essays in L1. They found that pause rates were higher at word and sentence boundaries in argumentative essays but not in narrative essays.

In sum, integration writing is a complex activity. People must read diverse sources in content, quality, and style. In addition, the writer must organise what he or she has read into a new piece of writing that synthesises, analyses, or describes. The reading medium (digital or print) affects the writing process, and the collected evidence indicates that print formats promote a better understanding and a better writing process than digital formats. In the same way, the purpose of writing (to argue versus inform) affects writing, causing more processes to be activated when the writer has to argue than when it has to inform.

Writing is not only about writing words, phrases, sentences or paragraphs but also about the pauses that occur when writing. Evidence suggests that the reading medium and the purpose of writing affect the length, frequency and type of pauses observed. Furthermore, the detected differences in pauses also depend on the writer's experience (novice or expert) and on the phase or stage in which the writing process is segmented (at the beginning, in the middle, or at the end). Thus, the need arises to study the effect of each of these variables and the interaction between them experimentally.

In this way, considering the dynamic and complex character of the writing, the factors that affect said writing could vary from one moment to another. Thus, studying how these factors and their interactions affect pausing while writing states could improve our understanding of the writing process and its underlying cognitive mechanisms.

Thus, this study aimed to identify how the reading medium (print or digital) interacted with the communicative purpose of a source-based integration writing task (to inform or persuade) to affect pauses (frequency, average duration, and type of pause) at different phases of writing (beginning, middle and end). In addition, the study sought to evaluate how this interaction is related to the writer’s abilities to generate a text that is understandable, coherent, and with a high academic standard (writer’s competence). Thus, we did not only study the main effects of these variables on the writing process but also how they interacted with each other. These four sources of interaction have mainly remained unexplored until now. Additionally, we analysed the location of certain types of pauses (within and between words; before and after sentences) as a function of the interaction of these four variables (medium, purpose, stage, and writer competence).

This study identified and analysed pauses while typing on a computer. For this purpose, we use the keystroke logging technique (KSL). With this technique, we automatically recorded each keystroke in real-time and in a non-intrusive way (Conijn et al., 2019; Leijten & van Waes, 2013; Zhu et al., 2019).

Method

Participants

Sixty-six first-year students from three disciplines in education,¹ who have no history of neurodevelopmental or sensorial disorders, were recruited at a Chilean university (68.18% women; M = 20.5 years, SD = 2.15). The sample size was calculated using G-Power software (Faul et al., 2007); based on a medium effect size of (ρ = 0.235), an alpha error of (α = 0.05), a power of (1-β) = 0.95, four groups (2-by-2 design) and three measurements.

All participants were native Spanish speakers. In addition, all participants stated that they were familiar with writing on a word processor and practised the tasks used in this research (reading from multiple sources and integration or synthesis tasks). Participants signed an informed consent guaranteeing that the data would be kept anonymous (#23–2018, approved by the Scientific Ethical Committee of the University of Talca, Chile).

Design

We implemented a 2-by-2 experimental design in which the communicative purpose of the task (to persuade or to inform) and the medium of presentation of the sources (digital or print) generated four experimental conditions in which participants were randomly assigned (see Table 1). We segmented the writing process into three equal time intervals for each participant: an initial, an intermediate and a final stage.

Table 1 Structure of the experimental design

The dependent variables were the average pause frequency, the average pause duration, and the percentage of the type of pause (see Table 3). The writer´s competence, an ad-hoc variable, was included as a covariate (see the procedure and Appendix 1).

Materials

Writing tasks

To carry out the research, we designed two integrative writing tasks whose main objective was to generate a new text from reading three sources of information (see Table 2). The tasks differed only in the communicative purpose they had to fulfil. On the one hand, the task was to generate a text to persuade, which means that participants had to take a position on the proposed topic and justify their position with arguments. On the other hand, the other task required writing a text to report on a specific topic; in this case, participants were asked to synthesise information from sources without taking a position on the topic.

Table 2 Description of academic writing tasks

All participants received the exact instructions in their mother tongue (Spanish), regardless of the communicative purpose of each task. The instructions included the characteristics of the task (inform or persuade), the number and type of reading sources, the potential readers, the style, the length of the text (100 and 150 words), and the time to complete the task (maximum 30 min). In addition, as a reminder, a brief explanation of what it is to write an integrative text was included in each condition.

Presentation medium of the sources

The participants used the same three source texts as stimuli for the writing tasks. These texts were adapted from three research articles whose subject matter referred to "Admission to higher education in Chile". Two source texts had a similar position on the topic, and one had a contrary position. All three texts were presented in Spanish. The first text contained 518 words; the second 146 words and 2 graphics; and the third one, 218 words and 1 table.

Regarding the presentation medium of these sources, two formats were used: print and digital. The print configuration consisted of a letter-size sheet (21.59 cm × 29.94 cm) printed only on one side. On each page, there was a source text. Each participant was given three pages in addition to another sheet with the task instructions. The content in the digital configuration was presented on a 21-inch monitor (1024 × 768 resolution). Here, the source texts were presented in Portable Document Format (PDF) with Foxit Reader software. As in the print configuration, text was presented on each page of the document, in addition to the instructions. The font type, size, and sheet margins were the same for each medium: Times New Roman, size 12, standard margins (top and bottom, 2.5 cm; left and right, 3.0 cm).

Writing compilation instrument

The writing tasks were recorded on a computer with the software Inputlog 8.0.0.6 (Leijten & van Waes, 2013), executed in the word processor Microsoft Office Word, version 2016. In addition, a standard QWERTY keyboard in Spanish, an optical mouse, and a 21-inch PC monitor was used. Spelling and grammar options were disabled in the word processor. It should be noted that the computer, the keyboard, and the mouse were located on a large desk (90 cm × 160 cm) so that those reading from print would not have to change places.

Procedure

The experiment was administered individually in a room specially equipped for the study. Before, participants received a brief instruction on using the Inputlog program and completed a default Copy Task in Inputlog. Once the students were familiar with the writing software and the identification data, they received a folder with the materials. The folder included task-specific instructions: a brief explanation of the communicative purpose of the writing task, information about the assessment indicators, length of the text, and task duration time. In addition, the folder contained a pencil and a sheet of paper to take notes and, if necessary, to use these notes while the writing task was completed.

After this introductory phase of getting used to the programme, the reading phase began. As mentioned above, participants could take notes while reading. Once the reading phase was over, the learners worked on the integrated writing task on the computer. For the participants with the digital format, the sources were open in a PDF document on the computer. While for the participants who used the printed format, the sources were on the back of the instruction folder. Thus, participants who read the sources in the digital medium continued in the same medium to complete the writing task.

In contrast, participants who read in paper format switched devices to complete the writing task. To match the contextual conditions, for the participants who changed the medium (paper reading—digital writing), the computer was located on the same desk where they had performed the reading phase. In addition, to facilitate writing, participants could use their notes. However, they could not use the source texts (independent of the source support). Finally, upon completing the writing task, participants had to write the word END (this was indicated in the task instructions) so that the researcher (or technical staff) closed the program from the control computer.

Writer competence analysis based on the quality of the text

A guideline instrument was designed to assess the quality of writing tasks (Appendix 1). This guideline incorporated two dimensions: written communication skills and degree of fulfilment of communicative purpose. Written communication skills include textual properties (organization of ideas and cohesive resources) and language conventions (register and tone required for the communicative situation). The degree of fulfilment of communicative purpose refers to the internal coherence of the writing and whether the text fulfils the communicative purpose. The two dimensions had four levels of proficiency (from highest to lowest).

Two raters or judges evaluated the writing documents generated by each participant (66 documents in total). These raters were professors in academic writing who were previously trained in the use of the guideline instrument. The raters scored all written products, marking the level of competence (1–4) on a worksheet for written communication skills and the degree of fulfilment of the communicative purpose. The inter-rater agreement was 0.85. A third judge analyzed interrater discrepancies (e.g. 1 and 3), and this evaluation was considered for the score on that dimension. Finally, with the result of both dimensions, an overall competence index was elaborated. This index corresponded to the arithmetic sum of the two dimensions. Based on this overall competence index, the writer's competence was estimated per participant.

Pause measures

The pauses analyses during the writing process were performed with Inputlog 8.0.0.6 software (Leijten & van Waes, 2013). This software was programmed to identify the frequency and average duration of pauses equal to or above 2000 ms. Additionally, the average percentage of pause types was analysed. These pauses were related to location (within and between words; before and after sentence). However, in terms of average pause type percentage, there was an important difference. All pauses during the writing process were automatically identified and selected by the Inputlog software. In this case the software selects all those pauses with a threshold greater than 100 ms. Thus, the frequency and duration of pauses used a different threshold than the used for pause type percentage.

Analysis of the temporal organization of the writing process

The writing process's temporal organisation was also analysed with Inputlog 8.0.0.6 software (Leijten & van Waes, 2013). Each participant was estimated at three intervals of the textualisation process: the initial moment, the middle moment, and the final moment. This temporal division depended on the duration of the writing task per participant. Previous studies have used a similar segmentation to describe writing behaviours performed at different times during a writing activity (Aguirre, 2015; Breetvelt et al., 1994; Escorcia et al., 2017; Valenzuela, 2020). Using this method, dividing the textualisation process into equal parts is a consistent and straightforward means of observing changes in the writing process over time (Xu and Qi, 2017).

Analysis of data

For the frequency and average duration of pauses, the first analysis strategy consisted of performing mixed ANOVAs. The between-subject factors were the communicative purpose of the task (to persuade or to inform) and the presentation medium of the sources (digital or print). The within-subject factor was the stage of the writing process (beginning, middle, and end). Subsequently, mixed ANCOVAs were carried out to control the writer's competence effects. Thus, the overall competence index of the writer's competence was incorporated as a covariate.

Additionally, the average percentage of pause types was analysed. These pauses were related to location (within and between words; before and after sentence). In this analysis, again, the communicative purpose of the task and the reading medium were between-subject effects; pauses type and the writing stage were selected as intra-subject effects. The analysis's rationale is as follows: If ANOVA and ANCOVA reproduce the same effects, then the writer's competence index (writing score) does not affect pauses. If the results change, the role of the writer's competence index is relevant and deserves to be incorporated into the model.

Although the writer's competence index can be considered as an independent variable, the strategy of using this variable as a covariate is a statistical procedure to neutralize its effect and estimate the isolated effect of the others independent variables; in the hypothetical scenario that all participants have the same level of writing competence (Keppel & Wickens, 2004). In order to implement ANCOVA analyses, we evaluated assumptions in terms of normality and homogeneity of variance. By using the Levene’s test, we found no differences of variance among the four groups, W_(3,62) = 0.32, p = 0.81. Based on the Kolmogorov–Smirnov test to evaluate normality within each group, we found no difference between expected and observed values related to normal distribution: G_{Inf_Print} |Diff|= 0.210, Z_K-S = 0.839, p = 0.483; G_{Inf_Digital} |Diff|= 0.209, Z_K-S = 0.834, p = 0.490; G_{Pers_Print} |Diff|= 0.262, Z_K-S = 1.08, p = 0.195; G_{Pers_Digital} |Diff|= 0.153, Z_K-S = 0.632, p = 0.819.

The second analysis strategy was a mixed linear model (Field, 2009). The goal was to identify nested effects in participants or experimental conditions by comparing two models in each dependent variable. The difference between the first and the second model was that the second one incorporated the writer's competence as a covariate. The results derived from this analysis should be taken with caution because the sample size requirements exceed the sample sizes of our study: 66 versus 114 (Soper, 2022). With this complementary analysis, we were able to evaluate how much a covariate (writer's competence) increases the explanatory capacity of the model regarding the frequency, duration, and type of pauses (O’Dwyer & Parker, 2014).

Results

Participants took 21.99 min on average (SD = 2.88) to complete the three-source reading task and the integrated writing task. The mean time for the writing task was 16.6 min (SD = 2.97). Of this time, 63.2% (SD = 3.83%) corresponded to pauses. Table 3 presents the descriptive statistic of the frequency and average duration of the pauses in the three moments of the writing process. A correlation matrix between these variables at different stages is depicted by Table 8, Appendix 2). The subsequent analyses focus on pause frequencies, the average duration of pauses, and finally, the percentages of pause types.

Table 3 Mean and standard deviations of average frequency and duration of pauses in the three stages of the writing process

Effect on the frequencies and average duration of pauses

Regarding the frequency of pauses, a difference between the writing stages was detected, F(2, 124) = 3.87, p = 0.023, ${\eta }_{p}^{2}=$0.06. At the end of writing (M = 15.31) frequencies of pauses were significantly higher than the beginning (M = 13.46, p = 0.045), and in the middle (M = 13.20, p = 0.011). No differences were observed between at the beginning and the middle of the writing process (p = 0.74). Effects attributable to the task purpose, the reading medium, or the interaction of these factors to each other or with the writing stage were not detected (ps > 0.11). Once the writer's competence was incorporated as a covariate to the model, the ANCOVA showed that previously observed differences between the three stages were attenuated, F(2, 122) = 2.69, p = 0.071. Effects attributable to the reading medium or interaction between reading medium and task purpose were not found (Fs ≤ 0.79, ps ≥ 0.38). Interactions between writing stages, task purpose and reading medium were not found (Fs ≤ 2.82, ps ≥ 0.063).

So then, we can state that the differences in average frequency pause observed between the stages of writing are associated with the writer's competence. It can be explained because once the effect of the levels of writer’s competence was statistically controlled, the observed differences among the writing stages were attenuated.

In terms of average pause frequency, an interaction between the writing stage and task purpose was observed, F(2, 124) = 6.28, p = 0.003, ${\eta }_{p}^{2}$ = 0.09 (See Fig. 1, Panel A). An exploration of simple effects at each stage found no differences between informing and persuading at the beginning stage (M_Inf = 6.78 vs M_Per = 7.03, F(1, 62) = . 07, p = 0.8) or in the middle stage (M_Inf = 7.02 vs M_Per = 6.76, F(1, 62) = 0.19, p = 0.66). However at the end stage, the persuasive task had a significantly longer average duration of pauses than the informative task (M_Inf = 7.67 vs M_Per = 11.03, F(1, 62) = 8.46, p = 0.005, ${\eta }_{p}^{2}$= 0.12). In addition, an exploration of simple effects through task purpose showed that when the purpose was to inform, no differences were observed among the three stages (ps ≥ 0.34). However, participants had more average pauses at the end-stage than at the beginning or in the middle stages (ps < 0.001).

Fig. 1

Mean and standard errors of pause duration segmented by stage with ANOVA (Panel A) and by controlling the competence effect with ANCOVA (Panel B)

Regarding main effects, differences between the stages were detected, F(2, 124) = 13.08, p < 0.0001, ${\eta }_{p}^{2}$= 0.17. At the end (M = 9.35), the average pause durations were significantly longer than at the beginning (M = 6.91), and in the middle stages (M = 6.89) (ps ≤ 0.001), with no difference between these two stages (p = 0.97). Main effects attributable to the reading medium or task purpose or the interaction between them and the writing stages were not found (Fs ≤ 2.90, p ≥ 0.09). Once the writer’s competence was incorporated into the ANCOVA model, the stage of writing and the communicative purpose of the task continued to interact, F(2, 122) = 5.29, p = 0.006, ${\eta }_{p}^{2}$= 0.08 (see Fig. 1, Panel B). However, the differences between the three stages of the writing process disappeared, F(2, 124) = 1.91, p = 0.15. Main effects attributable to the medium or the communicative purpose of the task, or the interaction between them or the stage, were not found (Fs ≤ 3.74, p ≥ 0.06).

Considering these results, we can state that the interaction between the communicative purpose and the writing stage is unrelated to writing competence. Regardless of how good the writer is, it will always take more time (e. g., average pause) at the end of writing when the purpose is to persuade. However, the writer´s competence was associated with the stages of the writing process.

Effect on the pause type percentage

A percentage analysis of the four pauses was performed. Table 4 depicts a hierarchy where the most prevalent pauses were within and between words. Pauses before and after sentences were less prevalent, with values lower than 0.40%. This trend was observed in the three stages of the writing process.

Table 4 Percentage of pause type in the writing process

A mixed 2*2*3*4 ANOVA was performed first, followed by a mixed 2*2*3*4 ANCOVA with the same variables, where the writer's competence, was a covariate. In both analyses, task purpose and medium reading were the between-subjects factors. The within-subjects factors were the stages of the writing process and pauses related to words and sentences (see Table 4). The dependent variable was the percentage obtained in each of the four pauses.

With a mixed ANOVA, only an interaction effect between writing stages and pause types was identified, F(6, 366) = 10.84, p < 0.001; ${\eta }_{p}^{2}$= 0.15. This effect shows that pauses within and between words; and pauses before sentences were more frequent at the beginning and in the middle than at the end of the writing process. However pauses after sentences were increasing from the beginning to the end stage.

In terms of main effects, we observed differences among pauses types, F(3, 183) = 3,291.51, p < 0.001; ${\eta }_{p}^{2}$= 0.98; and between the writing stages, F(2, 122) = 13.20, p < 0.001; ${\eta }_{p}^{2}$= 0.18. Participants had the highest percentage with pauses within words (47.44%), followed by pauses between words (25.99%). Pauses before sentences (0.27%) and after sentences (0.22%) were less prevalent than pauses associated to words. Finally, in terms of writing stages, at the end the average percentage was lower (M = 17.15) than at the beginning (M = 19.31, p < 0.001) and in the middle stages (M = 18.98, p < 0.001), while no differences between these two stages were detected (p = 0.39). Differences between reading medium, F(1, 61) = 0.35; p = 0.56; task purpose, F(1, 61) = 0.69; p = 0.41; or the interaction of these factors, F(1, 61) = 0.54; p = 0.46 were not observed.

Once the writer's competence was incorporated as a covariate, a mixed ANCOVA showed that the interaction between stage and pause type disappeared, F(6, 360) = 0.45, p = 0.843. In terms of main effects, the differences in pauses type were again observed, F(3, 180) = 155.02, p < 0.001; ${\eta }_{p}^{2}$= 0.72; and no differences among writing stages were observed, F(2, 120) = 0.17, p = 0.841.

In summary, our results indicated that writing stages were the main effect, and the interaction effect between pause types and writing stages was related to the writer’s competence; once the covariate was statistically controlled for, these effects disappeared. However, the difference observed among the four pause types was not affected by the writer’s competence because regardless of the covariate, the effect attributable to the pause types was the same.

Mixed linear models

Two mixed linear models were tested for average pause frequency, pause duration, and pause type percentage. Task purpose and reading medium were independent factors, and the writing stages were repeated measures. The pause type category was added as a third independent factor for the average pause type percentage. Model 2, in all dependent variables, included the writer’s competence as a covariate. As a first step, it was analysed whether the increment of -2LL from Model 1 to Model 2 was significant when the covariate was incorporated. Table 5 shows a summary without (Model 1) and with the covariate (Model 2). Concerning the average pause frequency and the average pause duration, the incorporation of the writer’s competence did not increase the explanatory capacity of the model. However, with the average pause type percentage, the incorporation of the writer’s competence increases the explanatory capacity of the model.

Table 5 Summary of the predictive mixed linear models

Regarding the average pause frequency (Table 9, Appendix 4), and according to Model 1, no factor affected the dependent variable. In Model 2, incorporating the writer's competence made visible a subtle interaction effect between reading medium and task purpose (b = 26.19, t = 2.14, p = 0.027). When the purpose was to inform, the pauses were more significant in print than in digital format. Analyzing the average pause duration, in Model 1 and Model 2 (Table 10, Appendix 5), no factors nor their interactions were statistically significant, ts ≤ 1.63, ps ≥ 0.110. Finally, in terms of average pause type percentage (Table 11, Appendix 6), Pause Type was a significant predictor without (b = 38.36, t = 23.84, p = 0.0001) and with the writer´s competence (b = 40.05, t = 5.58, p = 0.0001). Additionally, a subtle interaction effect among reading medium, task purpose, and pause type was detected when the writer’s competence was added to the model, (b = − 35.99, t = − 2.01, p = 0.047).

The models applied to pause frequency and pause duration showed low predictive capacity. However, an interaction effect between reading medium and task purpose slightly affected the average pause frequency. When the writer's competence was controlled, the average pause type percentage was affected by a triple interaction among reading medium, task purpose, and pause type. Finally, it is possible to state that there is no effect of the writer's competence on the difference detected among the four types of pauses.

Discussion

This study aimed to identify how, in a source-based integration writing task, the reading medium (print or digital) interacted with the communicative purposes of the tasks (inform or persuade) to affect pauses at different stages of writing. At the same time, we sought to determine how this interaction was related to the writer's competence on that writing task. We extracted three keystroke log characteristics related to pauses: average frequency, average duration, and pause types to meet these goals. In addition, we divided the writing process into three stages: beginning, middle, and end-stage. Thus, we studied the main effects of these variables on the writing process; and how these four factors simultaneously interacted. Moreover, these interactions must be considered if writing is studied as a dynamic and complex process that constantly changes.

Initially, in none of the three stages into which the writing process was divided, we have found pure interaction effects between the reading medium and the task communicative purpose, which affected the frequency of pauses, their duration, or the percentage of pause types. Although we are not aware of previous studies that have investigated how these variables—at the same time—interact in the writing process, some studies have shown that these variables, alone or in double interaction, affect writing quality, pauses, or the writing process (Latini et al., 2020; Medimorec & Risko, 2017). These findings led us to assume that these variables might interact concomitantly. However, in our results, pausing behaviour was not affected by the interaction of the studied variables: reading medium, the communicative purpose of the task, and the moments into which the writing process was divided.

As the main effect, communicative purpose alone did not affect any dependent variables (frequency, duration, and types of pauses). However, the communicative purpose of the tasks had a significant interaction with the writing stage in the pause duration. Thus, we observed that the average duration of pauses at the end of the writing process was longer in the persuasive task than in the informative task. According to the literature, this difference between the tasks would be due to the more complex features present in the persuasive task (e.g., more sophisticated vocabulary, more elaborate sentences, among others), which would generate a higher cognitive load (Beauvais et al., 2011; Medimorec & Risko, 2016, 2017; Vandermeulen et al., 2020). Therefore, these differences were expectable. These differences between tasks at the end of the writing process remained, despite statistically controlling for the effect of the writer’s competence. This finding means that, no matter how good students are at writing, they will always take longer at the final part of writing if they must persuade rather than inform. Thus, the results presented here are in line with the findings of previous studies showing that the writing process differs according to the purpose of the task, although these studies do not directly address the stages at which these effects are most noticeable (Beauvais et al., 2011; Medimorec & Risko, 2016, 2017; Vandermeulen et al., 2020).

Related to the effect of reading media (print and digital), we did not observe that the means of presenting the sources were different, nor that they interact with any of the other variables analysed. In other words, reading information on a digital screen or printed does not affect pausing behaviour during integrative writing. Our results here may diverge somewhat from previous studies assessing the effects of reading medium, where differences in sustained attention, concentration, and competence between readings media were identified (Ackerman & Lauterman, 2012; Clinton, 2019; Delgado & Salmerón, 2021; Delgado et al., 2018; Liu, 2005). However, it should be noted that studies comparing differences between media generally do not consider writing, let alone the writing process. Instead, they assess competence on multiple-choice or closed-ended reading tasks (Delgado et al., 2018). Only one recent study has investigated the effects of the reading medium on integrative processing and integrated understanding of an illustrated text (see Latini et al., 2020). However, compared to our results, in the study by Latini et al. (2020), the reading medium was apparent in integrative processing, even when there was no effect on integrated understanding. In this study, those who read from print sources showed a more integrative written process than those who read from digital sources. While this may seem contradictory to our findings, it is necessary to point out that our study differs methodologically from Latini et al. (2020), as our interest was in the writing process (in particular, pausing behaviour) and not only the written product. Furthermore, our study considered other sources of variability, such as the moments during the textualisation process and the purpose of the task, and the interactions that these main effects might generate.

In terms of pause type, previous research has indicated that pauses are associated with cognitive processes present in writing (Alamargot et al., 2007; Alves et al., 2007; Medimorec & Risko, 2017). For example, word-related pauses account for lower-level processes such as lexical access and spelling. In our results, pauses were concentrated on words rather than sentences or paragraphs. This finding is not new considering that words are more frequent than sentences and paragraphs. In our case, this tendency was observed in all three stages. That is, the hierarchy remained constant regardless of the stage of the writing process. Thus, pauses within words appeared most frequently, followed by pauses between words. The first has been associated with lower-level processes, while the other pauses account for higher-level processes, such as monitoring and reviewing (Barkaoui, 2019; Conijn et al., 2019; Medimorec & Risko, 2017; Zhu et al., 2019). Despite this strong hierarchy, pause type interacted with the writing stage, but such interaction disappeared when writing competence was statistically controlled. As mentioned above, we observed that pauses within and between words were more frequent at the beginning and middle than at the end of writing. In our understanding, these pauses denote lexical and orthographic processing rather than monitoring activity of the writing process (Escorcia et al., 2017). In this way, we can say that the writer's competence was associated with lexical and orthographic processing and that when this competence were statistically controlled, the interaction disappeared.

The principal strategy of this study was to incorporate the writer’s competence as a covariate to evaluate how its presence modified the effects initially detected with the mixed ANOVA. The results showed that the effects (main and interaction) in which the writing stages were segmented tended to disappear when the influence of the writer’s competence was controlled. For us, it is an indicator that the pauses during the writing process were associated with the writer’s competence. On the other hand, the fact that the writing stages interacted with the task purpose, especially in the persuasion, are indicative that tasks with more cognitive requirements tended to have more pauses at the end of the writing.

The present study allowed us to characterize the effect of different factors present in a standard hybrid task in university academic situations. The effects of the reading medium and the purpose of the writing tasks during the production of an integration text were analysed, particularly on the behaviour of pauses (average frequency, average duration, and type of pauses). Thus, it could be identified that writing behaviour in an integration task does not vary according to the reading medium or the communicative purpose of the task. Nevertheless, the study established that the task's communicative purpose interacting with the writing process stages revealed potential cognitive processes that would underlie this interaction. Thus, we observed a more time-consuming process in persuasive integration tasks at the end of writing than the informative integration task, which we assume reflects monitoring and revision processes. We also observe that the interaction between communicative purpose and writing stages is maintained when the writer's competence is statistically controlled. Thus, these cognitive processes are not related to the quality of the writing. As for the reading medium, no main effects or interaction effects were found with any of the variables analysed.

A slightly different panorama is observed when the linear mixed model is used. Models 1 and 2 applied to pause frequency and pause duration showed low predictive capacity. However, an interaction effect between reading medium and task purpose slightly affected the average pause frequency. Additionally, when the writer's competence was controlled, the average pause type percentage was affected by a triple interaction among reading medium, task purpose, and pause type. This finding indicates that these factors tend to interact when the writer's competence is statistically controlled among the participants in the four experimental conditions. Finally, it is possible to state that there is no effect of the writer's competence on the difference detected among the four types of pauses; because the effect is stable regardless of whether the covariate is present.

This study represents a step forward in an underexplored line of research. However, its conclusions should be taken with caution, given its limitations. First, the recording of keystrokes is still a technique that does not allow us to draw precise conclusions about the cognitive processes. For example, our interpretation of pausing behaviour at the end of the textualisation phase is supported by previous studies (e.g., Alves & Limpo, 2015; Medimorec & Risko, 2017). However, this behaviour does not necessarily reflect an overall review or follow-up (Baaijen et al., 2012). Therefore, this technique could be complemented by using verbal aloud protocols or the analysis of screen recordings. Another limitation of this technique is that a fixed threshold was used to capture the pause (2000 ms). This threshold could affect the analyses given the length of the writing task (100–150 words). Therefore, it would be advisable to use different pause thresholds (≥ 500 ms and ≥ 500 and ≤ 1000 ms) or use a more extended writing task. A final limitation of this technique is its susceptibility to other factors that we did not delve into in our studies, such as prior experience or writing training (Conijn et al., 2019). Although we assume that such familiarity and experience were randomly distributed in our study and that the assignment of participants to the experimental conditions was also random, we believe that other studies could measure and evaluate the effect of these two variables on writing tasks.

Second, task design may have affected the results. For example, students who used digital media and read from print sources completed the writing task on a computer. Thus, while students who read from print media switched media (print to digital), those who used digital media continued in the same medium (digital to digital). This design might have affected the results favouring the digital media condition. It is known that the print medium has an advantage over the digital medium in terms of reading performance (Delgado & Salmerón, 2018, 2021; Delgado et al., 2018). However, this advantage might have been neutralized in the present study when people switched to a digital format during the writing task. Another alternative, suggested by a reviewer, and with which we fully agree, is that the absence of differences between the reading media could be because of taking notes in all groups. Because taking notes may have activated high-level processes in tasks initially aimed at triggering reading processes. Further studies might consider assessing the effect of notetaking as an attenuating variable for differences between reading media.

Third, the way the dependent variables of the writing process were analysed may not have been entirely sensitive to experimental manipulation. Our studies found no evidence that reading medium and task purpose independently affected or interacted with the pause type, frequency, or duration. However, it could be possible to affect the temporal structure of how these pauses appeared in the writing process. One way to detect this effect would be through time series analysis techniques, such as recurrence quantification analysis (RQA) (Castillo et al., 2015; Lira-Palma et al., 2018; Marwan et al., 2007; Wallot, 2017; Wallot & Grabowski, 2013, 2019; Zbilut & Webber, 1992). Using an elementary example (See Table 6), we could assume that the two types of communicative purposes are identical in terms of types of pauses (WW, BW and BS) and that they represent equal proportions (1/3 each). However, the way these appear could be different.

Table 6 Example of how three kinds of pauses could appear in different periods as a function of task purpose, even when there is no difference in terms of proportions

In the same way, if the duration of the pauses is analysed (See Table 7), we could find that the two types of communicative purposes are identical in terms of mean and standard deviation (M = 400; SD = 173); however, we cannot deny that the temporal order of these pause duration is different.

Table 7 Example of how pause durations can appear in different periods as a function of task purpose, even when there is no difference in terms of mean and standard deviation

In this way, from traditional statistical analyses, it would be concluded that the independent variables and their interactions have no effect. However, RQA (categorical or continuous data) would allow estimating values ​​such as entropy, determinism, and other measures based on how the pause types or their duration are organized. In this context, our study's experimental manipulation probably has effects at the temporal structure level, even though it does not work in terms of proportions, means, and standard deviations.

Despite the limitations, our results have several implications for education and research. In educational practice, this study exposes the differences in cognitive load present in the writing process given the communicative purpose and stage. Thus, our findings reinforce the idea of monitoring in the final part of a writing task. This support could come from teachers or the students themselves through an instructional design that facilitates this overall writing revision. Another implication for educational practice is the absence of main effects and interaction of the reading medium in the writing task. Our study observed no differences between those who read from print and those who read on the computer in writing integration. This finding is not minor, as students in higher education do not always have access to print sources. Finally, this study could be interpreted as an extension of the cognitive theory of multimedia learning (Mayer, 2019). This theory states that learning is caused by the instructional method rather than the medium in which the instruction is mounted. Thus, further inquiry is required into instructional methods and task design on the writing process and writing competence.

Conclusion

Our study investigated the writing process in a multi-source integration task. The study analysed variables present in a natural context for a university student: different reading media and different writing tasks. Thus, we investigate how these variables interact (at different stages) in the writing process. How do these variables affect pauses at three different stages of the writing process? Additionally, we sought to determine how this interaction relates to writing competence. In terms of results, we identified that in an integrative writing task, pausing behaviour was not affected by the reading medium, the communicative purpose of the task, or by the interaction of the two variables. We identified that the task's communicative purpose interacts with the stages of the writing process. Specifically, pauses were longer in the persuasive integration task at the end of the writing process than in the informative integration task. It was also observed that participants spent more time at the end of the writing process when controlling for competence effects in the writing task when the purpose was to persuade. These differences were also observed concerning the types of pauses. In conclusion, the study provides different clues about the effects of variables in a situation of reading from multiple sources and integrative writing.

Appendices

Appendix 1. Evaluation rubrics

A. Written competence characteristics

Description of competence by level

Level 4. The text is easy to understand, with an exposition of ideas focused on a topic that follows a logical organization. Demonstrates precise handling of the rules of written expression and adequately uses cohesion resources. Likewise, it adjusts to the register and tone required for the communicative situation.

Level 3. The text is easy to understand. The ideas are built around a central theme following a logical organization. However, there are some problems in the handling of the rules of written expression. However, these do not affect the organization of the text or the fulfillment of the communicative purpose.

Level 2. The text is understood. In most of the text, the ideas are organized around a central theme following a logical order. There are some problems in the handling of the rules of written expression. These affect the organization of the text and/or the fulfillment of the communicative purpose.

Level 1. The text is not easy to understand. There is an organization around a theme, but it is not clear and sometimes disorderly. That is, the logical sequence is scarce. There are also problems in the handling of the rules of written expression. These affect the organization of the text and/or the fulfillment of the communicative purpose.

B. Genre-specific characteristics

Description of competence by level

Level 4. The response to the question is optimal. The communicative purpose is met. The selection of information is consistent and accurate throughout the text.

Level 3. The answer to the question is accurate. The communicative purpose is met, but there are minimal aspects that are not clear. The information presented is coherent and accurate throughout the text.

Level 2. The answer to the question is elementary. The communicative purpose is met, but some aspects are not evident. The information presented is coherent and accurate, but it is not maintained throughout the text.

Level 1. The answer to the question is weak. The communicative purpose is minimally fulfilled, or the purpose is unclear. Despite this, the information presented is coherent and accurate, but it is not maintained throughout the text.

Appendix 2

See Table

Table 8 Bivariate correlation matrix

8.

Appendix 3. Example of texts produced

Example Low-performance evaluation (in Spanish)

Participant e3_18_Ta

La finalidad de acuerdo a mi punto de vista que posee la educacion superior es muchas veces convertise en un profesional de una carrera que es del gusto de la persona que esta estudiando.

Pero la educacion superior es mucho mas que un proceso de convertise en un profesional son muchas veces los sueños que posee una persona de ser alguien en la vida. Si se toma en cuenta la finalidad entorno a un pais la finalidad es crear profesionales para que cumplan una funcion en un trabajo especifico y aporte en el pais.

En cuanto al proposito social es basicamente aportar de acuerdo a la profesion que estudio y convertirse muchas veces en un numero mas de profesionales que aportan en una sociedad.

Example High-performance evaluation (in Spanish)

Participant e3_28_Ta

La educación superior posee como finalidad principal la creación de nuevos profesionales capaces de dar un aporte integro a la sociedad. Esta educación va más allá de la educación básica o de la educación media.

Una gran mayoría de las personas tiene como meta el lograr llegar a la educación superior. Ser un gran profesional. Más que nada por una presión social, dado el hecho de que existe una mala visión o una especie de estigma social por quien no logra llegar a ella y quedar solo con su cuarto medio.

En la sociedad chilena se ve muy marcado este tema, puesto que, en gran cantidad, una persona que posee un título de educación superior es visto y calificado más apto para un trabajo que una persona que solo posee su enseñanza media. A pesar de que cabe la posibilidad que la persona que no posee su título superior sepa mucho más del trabajo que la propia persona que si posee el suyo.

Appendix 4

See Table

Table 9 Mixed linear models for average pause frequency, where Writer’s competence is a covariate (Model 2)

9.

Appendix 5

See Table

Table 10 Mixed linear models of average Pause Duration, where Writer’s competence is a covariate (Model 2)

10.

Appendix 6

See Table

Table 11 Mixed linear models for Pause Type Percentage, where Writer’s Competence is a covariate (Model 2)

11.