Teacher Engagement with Online Formative Assessment in EFL Writing During COVID-19 Pandemic: The Case of China

Abstract

Although previous studies have examined the impact of online formative assessment on second language learners’ writing development, scant attention has been paid to how writing teachers engage with online formative assessment and the influencing factors. By exploring three English-as-a-foreign-language (EFL) writing teachers’ engagement with online formative assessment during COVID-19 in three universities in China, the qualitative case study identified three types of teacher engagement: disturbing, auxiliary, and integral, embodied by their varied emotional, physical-cognitive, and social investment in the formative use of information communication and technology in writing assessment. These individualized engagements were primarily influenced by teachers’ beliefs, digital literacies, and learning/teaching experiences, and mediated by relevant contextual and technological factors. Implications are provided for supporting EFL writing teachers’ engagement with online formative assessment.

Introduction

Given the pivotal role of formative assessment in facilitating student learning and effective classroom practice (Black & Wiliam, 2009) and the ubiquity of information and communication technology (ICT), online formative assessment (computer-based/IT-enabled formative assessment)—i.e., the use of ICT for the purpose of formative assessment in an online environment—has drawn increasing attention (Gikandi et al., 2011). The outbreak of COVID-19 has made online formative assessment more pressing, since many universities have to switch from face-to-face (f2f) to online instruction, and online teaching, learning, and assessment has become “the new norm” (Todd, 2020 p. 5). To facilitate students’ learning and teachers’ professional development during COVID-19 and beyond, it is important for teachers to engage with online formative assessment (Rahim, 2020), and for researchers to examine teachers’ assessment experiences in fully online contexts (Mimirinis, 2019).

In second language (L2) writing, while teachers are advised to treat formative assessment as the central pillar of classroom writing assessment (Lee, 2017), the use of online formative assessment in writing classrooms is under-explored (White, 2019), especially from teachers’ perspective. Little is known about writing teachers’ engagement with online formative assessment especially during crises like COVID-19. This study seeks to fill this void by investigating three English-as-a-foreign-language (EFL) writing teachers’ experience with online formative assessment in China. The findings can shed useful light on the implementation of online formative assessment in EFL writing and writing teacher education in similar contexts.

Literature Review

Online Formative Assessment in L2 Writing

Formative assessment refers to the classroom assessment practices when “evidence about student achievement is elicited, interpreted, and used by the teachers, learners, or their peers to make decisions about the next steps in instruction” (Black & Wiliam, 2009 p. 9). Focusing on improving teaching and learning via assessment, formative assessment is conceptualized as five key strategies (Black & Wiliam, 2009): (1) clarifying and sharing learning criteria for success; (2) engineering effective learning tasks that elicit evidence of student understanding; (3) providing feedback that moves learners forward; (4) activating students as instructional resources for one another; and (5) activating students as the owners of their own learning (p. 8).

Previous research has shown that formative assessment helps improve students’ writing and can be integrated into L2 writing by providing diagnostic teacher feedback and putting students at the center of learning in a supportive, process-oriented writing classroom (Lee, 2017; Wang et al., 2020). Recently, the advancement of ICT has made online formative assessment possible (Gikandi et al., 2011), and fostered an increasing integration of ICT in formative assessment in L2 writing (e.g., Zhang & Hyland, 2018). Researchers have confirmed the benefits of ICT (e.g., rapid communication of ideas and datafication of learning processes) in facilitating formative assessment (Shute & Rahimi, 2017). They also suggest teachers use ICT to enable effective online formative assessment by (1) providing formative and immediate feedback; (2) engaging students with critical learning processes (e.g., collaborative interactions and self-regulation); and (3) promoting equitable education by facilitating responsive teaching and assessment, and supporting progressive learning (e.g., Gikandi et al., 2011).

Although the development of ICT and online formative assessment has opened up new possibilities for writing teachers (Williams & Beam, 2019), the implementation of online formative assessment in L2 writing classrooms remains scarce (White, 2019). Of the limited studies, researchers have mainly focused on the technological aspects and examined ICT tools suited for formative assessment and their influence on students’ writing, such as Google Docs, automated writing evaluation (Zhang & Hyland, 2018), and Microsoft Word tools (Lee, 2017).

Although the existing research has evinced the effectiveness of online formative assessment in enhancing the teaching and learning of writing, such work has overlooked the role of teachers–-the key agents in enacting principles of online formative assessment (Mimirinis, 2019). Not much has been done to examine how L2 writing teachers engage with online formative assessment. Such information is important, given that assessing writing represents a large but daunting part of L2 teachers’ work (Lee, 2017), and online formative assessment is assuming more prominent roles during and beyond COVID-19 (Rahim, 2020).

Teacher Engagement

Teacher engagement refers to the extent to which teachers are invested or committed to their work (Klassen et al., 2013). It indicates teachers’ degree of interest and energetic investment to employ relevant skills and activities to their teaching (Louis & Smith, 1992). Teacher engagement is a multidimensional construct comprising a global level of energetic investment across one’s professional role and the closely-related physical–cognitive, emotional, and social sub-dimensions (Klassen et al., 2013). Particularly, the physical–cognitive engagement includes teachers’ involvement and investment in teaching-related activities; emotional engagement concerns teachers’ affective responses to work; and social engagement highlights teachers’ concern for students. In this study, teacher engagement with online formative assessment is understood as the extent to which teachers physically-cognitively, emotionally and socially value, and participate in online formative assessment in L2 writing.

Teacher engagement, according to the socio-technical theory, results from teachers’ interactions with the institutions and tools to which they have access, since teachers’ attitudes and behaviors depend on the concurrent configuration of the individual, institutional, and technological subsystems of their socio-technical system (Whiteoak, 1997). Mediated by various individual, contextual, and technological factors, such as teacher beliefs, institutional policies, and affordances of ICT, teachers tend to take on different stances on online formative assessment. These stances and factors, however, are under-explored, with the previous research primarily concentrating on teacher engagement with formative assessment in f2f environments. For instance, Marshall and Drummond (2006) distinguished the “spirit” and the “letter” kinds of engagement with formative assessment. While teachers in the former valued learner autonomy and exercised agency to assist student learning, those in the latter stuck to rigid procedures and did little to promote students’ autonomy. Focusing on Norwegian physical education teachers, Leirhaug and MacPhail (2015) found that while teachers demonstrated high-level engagement with formative assessment, their engagement was constricted by the conventional emphasis on grading and insufficient institutional support. Studies on L2 writing also reveal that writing teachers’ understandings, practices, and engagement regarding formative assessment vary, influenced by their teaching experiences and beliefs, schools’ assessment policies, and high-stakes tests (e.g., Lee, 2017; Wang et al., 2020). Recently, researchers have also begun to explore factors influencing first-language teachers’ engagement with formative assessment in technology-enhanced writing classrooms. Particularly, Krishnan et al. (2020) study showed that teachers’ engagement with formative assessment for online collaborative writing was shaped by various individual and contextual factors, such as teacher beliefs and curriculum guides. It, however, did not examine the engagement types of and technological factors shaping teacher engagement, which warrant further attention.

Although previous research offers useful knowledge about L2 writing teachers’ engagement with formative assessment, limited research has investigated L2 writing teachers’ engagement with online formative assessment. It remains unclear how L2 writing teachers engage with online formative assessment while negotiating the possible individual, contextual and technological factors. Our study aims to address this gap by investigating three EFL writing teachers’ experience during COVID-19.

Methodology

Research Questions

This study attempts to answer the following questions:

• 1. How do EFL writing teachers engage with online formative assessment during COVID-19?

• 2. What individual, contextual, and technological factors have influenced EFL writing teachers’ engagement with online formative assessment during COVID-19?

Context and Participants

In China, formative assessment constitutes an important part of the language education reform agenda (e.g., Wang et al., 2020). There has also been a vigorous promotion of comprehensive integration of ICT in EFL teaching since the release of Action Plan of Education Information 2.0 (Ministry of Education, 2018), which encourages teachers to promote information literacy and innovate instructional practices to facilitate learning online. Owing to the close-down of universities during COVID-19, EFL teachers had to switch to the online environment, and use ICT to teach and assess writing in the spring semester of 2020.

Against this background, purposeful sampling was used to recruit teachers. Based on the multidimensional representation of teacher engagement (Klassen et al., 2013) as well as strategies for online formative assessment (e.g., Gikandi et al., 2011) and those for implementing formative assessment in L2 writing (e.g., Lee, 2017), we designed a questionnaire (Appendix 1) and sent it to teachers in five universities which emphasized formative assessment to identify information-rich participants for the study. Among the 17 questionnaires returned, six teachers reported high-level engagement; seven teachers medium; and four teachers low. Based on the questionnaire results, we purposively invited three teachers – Manyun, Xiaofang, and Yuchen (pseudonyms) from three different universities to participate in this study to obtain an in-depth understanding of teacher engagement. While they had implemented formative assessment in writing classrooms (e.g., involving students in writing assessment), they reported different levels of engagement with online formative assessment in EFL writing. Table 1 presents their background information.

Table 1 Participants’ background information

Data collection and analysis

After obtaining ethical approvals, one 90-min, semi-structured interview (Appendix 2) was conducted with each teacher to get an emic view of their engagement with online formative assessment in EFL writing and influencing factors at the end of the spring semester of 2020 (July). The interviews were conducted in Chinese to allow them to articulate thoughts freely, and were audio-recorded and transcribed by the first researcher.

Additionally, videoed lessons and instructional materials were collected to triangulate the interview data and demonstrate teachers’ investment in and practice regarding online formative assessment in EFL writing classrooms. Since the three participants had recorded all or some of their lessons during the 18-week spring semester, the researchers invited them to share their videoed lessons to see how they implemented online formative assessment in EFL writing during COVID-19. Also, the participants provided instructional materials (e.g., PowerPoint files) which exemplified their writing assessment practice. In total, Manyun submitted 14 videoed lessons and 16 writing drafts with teacher written feedback; Xiaofang provided 2 videoed lessons and 5 PowerPoint files; and Yuchen submitted 20 videoed lessons and 7 PowerPoint files.

A qualitative, inductive approach was employed to analyse the data (Miles et al., 2014). The first two researchers read and/or watched each participant’s interview transcripts, videoed lessons, and instructional materials many times to familiarize the data, and coded independently by following the coding scheme (see Appendix 3 for detail) to identify each teacher’s physical–cognitive, emotional, and social engagement with online formative assessment and the individual, contextual, technological factors in their socio-technical systems, resulting in preliminary codes, such as “limited exploration of ICT tools,” “frustration,” “learner autonomy,” “digital literacy,” “institutional support”, and “development of ICT.” They then worked together to juxtapose and compare the initial codes to tease out each teacher’s engagement type and influencing factors, resulting in three key themes concerning teacher engagement with online formative assessment: online formative assessment as disturbing, auxiliary, and integral. Excerpts relevant to the emergent codes and themes were extracted from the interview transcripts, videoed lessons, and instructional materials, transcribed and/or translated where necessary. The preliminary data interpretations were sent back to the participants for member checking, and their comments were used to refine the final results.

Findings

Mediated by the individual, contextual, and technological factors within their socio-technical systems, the participants demonstrated three types of engagement with online formative assessment in EFL writing: disturbing, auxiliary, and integral.

Manyun: Online Formative Assessment as Disturbing

Manyun received training in formative assessment in translation, but never in EFL writing. Having implemented formative assessment for over two years in f2f writing classrooms in a university of foreign languages where students generally have an upper-intermediate English proficiency level, she felt that “the online environment disturbed its implementation during COVID-19” (Interview). Overall, she demonstrated “a low-level engagement with online formative assessment in EFL writing” (Interview), characterized by a suspicious attitude towards ICT, a limited exploration and use of ICT in writing assessment, and a de-emphasis on students’ independent learning.

Admitting that there existed many ICT tools for promoting formative assessment in writing, Manyun doubted “the necessity and benefits of doing online formative assessment” (Interview). Manyun’s scepticism was related to her learning experience: “My teachers never used ICT to assess writing…and I felt comfortable with the traditional f2f assessment” (Interview). It was also influenced by the insufficient university-supported teacher training: “No training has been provided to push me to integrate ICT in formative writing assessment in our university” (Interview).

Overall, Manyun had limited knowledge about using ICT in EFL writing and “rarely worked effectively with ICT in writing assessment” (Interview), demonstrating a low physical–cognitive engagement. Being unenthusiastic about exploring ICT for formative writing assessment, she mainly employed Microsoft tools and CCtalk (https://www.cctalk.com)—a live education platform allowing teachers to deliver lessons—to give teacher feedback to implement online formative assessment. When compared with the pen-and-paper assessment in f2f classrooms, she “gave more detailed and lengthy annotated comments on students’ electronic drafts, as the space allows [her] to do so” (Interview). Nevertheless, such lengthy written comments and detailed oral explanations did not seem to move students forward.

My feedback did not work equally well as in f2f classrooms in improving students’ writing…Due to the absence of f2f interactions, students did not take online learning and assessment seriously and participate in writing activities assiduously. (Interview)

In this regard, ICT’s inability to provide f2f-like interactions lowered students’ participation in writing assessment, thereby impeding their learning. Such technical limitation made Manyun query the effectiveness of online formative assessment and emotionally disengaged with it.

While realizing that students’ low participation was detrimental to learning, Manyun believed that “little could be done to influence students’ participation online” (Interview). Consequently, she “involved students less in writing assessment… and seldom used ICT to help students learn English writing independently during COVID-19” (Interview), showing an increasing de-emphasis on students’ autonomy. For instance, although she believed that peer assessment could benefit students’ writing and autonomous learning, she stopped using it.

Without f2f interactions, it is difficult to involve all the students in peer assessment and ensure that they benefit from it. So I did not use it this semester. (Interview)

The lack of f2f interactions in the online environment thus made Manyun lose faith in peer assessment, and consequently, lessened her social engagement with online formative assessment, to which peer assessment is essential.

Overall, due to her learning experience and limited knowledge of integrating ICT in writing assessment, students’ low participation in online learning, insufficient institutional support, and ICT’s inability to facilitate f2f-like interactions, Manyun was suspicious of and reluctant to embrace online formative assessment during COVID-19.

Xiaofang: Online Formative Assessment as Auxiliary

Xiaofang received professional training on formative writing assessment and practised it in writing classrooms in one research-oriented university with students of generally upper-intermediate and advanced English language proficiency. During COVID-19, she manifested an overall auxiliary stance on and “a medium level of engagement” (Interview) with online formative assessment, featured by mixed feelings about the use of ICT in writing assessment, extensive efforts to find appropriate ICT tools for formative assessment, and the use of ICT-facilitated assessment activities to monitor students’ learning.

Different from Manyun’s scepticism about ICT, Xiaofang recognized its value and promoted it in writing assessment.

ICT makes writing assessment more timely and flexible, since students and I can chat whenever we have problems. It also helps create a supportive learning community for students. (Interview)

While Xiaofang was positive in engaging with online formative assessment, she felt “challenged and frustrated” at first, as she was uncertain about “what ICT tools and tasks would be better…and how to effectively transform prior f2f assessment practices to the fully online context” (Interview).

Although Xiaofang’s unfamiliarity with ICT tools made her feel insecure, she was motivated to find appropriate ICT tools for writing assessment, because “the university emphasizes ICT-assisted instruction…and organized workshops to help teachers explore the formative use of ICT during COVID-19” (Interview). Such institutional support and agentive exploration led to her increasing physical–cognitive engagement with online formative assessment. For instance, she explored and successfully transformed writing conferences from f2f to the online mode with the help of QQ Messenger (https://www.imqq.com/), which allows for instant messages, voice, and video chatting. She commented,

Students contacted me online whenever they had difficulties in understanding my feedback, making teacher feedback more timely, transparent, and effective. (Interview)

Likewise, she followed her colleague’s suggestion to share students’ writing and give immediate feedback through Tencent Docs (https://docs.qq.com/), a free platform that allows users to create, share, and edit documents online. Taking one academic writing lesson as an example, Xiaofang asked students to describe one bar chart on employment pressure (Fig. 1) and commented on students’ writing samples via Tencent Docs, which allowed her “to share students’ writing and give them timely oral feedback as in f2f classrooms” (Interview).

Fig. 1

A snapshot of Tencent Docs

Xiaofang: Jiawei used a double negative sentence to show his finding and found related data to support it. This is good…. However, the description is all about commonality. Can you find any differences between groups and compare them? (Videoed Lesson)

Although Xiaofang sometimes felt frustrated at her limited ability to use ICT tools, ICT enabled her to conduct writing conferences and provide timely feedback to move learners forward by removing the spatial and temporal constraints. Such experience increased her positive reactions to online formative assessment: I felt so contented when I saw my students inspired.

Similar to Manyun, Xiaofang found that the lack of f2f interactions in the fully online environment lowered students’ motivation to write and hindered their learning. Thus, she endeavoured to create a supportive classroom culture and monitor student learning by “giving more encouraging teacher feedback to motivate students to write and revise” and “using ICT to include more assessment activities and provide more timely and frequent feedback” (Interview). For instance, Xiaofang integrated more assessment activities into writing instruction to motivate students to preview and review reading materials concerning writing with Moso Teach (https://www.mosoteach.cn/), a mobile-based teaching management platform that allows teachers to upload learning materials, create activities like discussion and quizzes, and check students’ learning. Such efforts to use ICT-facilitated assessment activities to motivate and monitor students’ learning were influenced by her belief that “writing assessment is not to evaluate how well students can write but to motivate them to learn” (Interview).

Despite her concern for students’ learning and growing ability to use ICT in writing assessment, Xiaofang seldom utilized the assessment information generated by ICT tools to foster students’ independent learning. She reflected,

I am not a teacher with high digital literacy. I know Moso Teach can provide students’ learning information, but I don’t know how to use such information to improve teaching and students’ independent learning. (Interview)

Although the learning information provided by ICT, such as analytical reports of students’ performance can be helpful for online formative assessment, Xiaofang’s limited digital literacy hindered her from unleashing its potential, which frustrated and hampered her social engagement with online formative assessment.

To conclude, while Xiaofang believed in the auxiliary role of ICT in writing assessment and became more capable of using ICT for formative assessment with the institutional support and agentive exploration, her limited digital literacy prevented her from facilitating students’ autonomous learning. Consequently, she took an overall simplistic approach to online formative assessment by using ICT primarily to monitor student learning as an “administrator” and “motivator” (Interview), and demonstrated a mixed feeling of frustration and contentment towards it.

Yuchen: Online Formative Assessment as Integral

Having integrated formative assessment and ICT in EFL writing for over six years in one university of foreign languages where ICT-assisted instruction is vigorously promoted and students have an overall upper-intermediate level of English language proficiency, Yuchen believed that “online formative assessment is integral to writing assessment” (Interview). She showed “a high-level engagement with online formative assessment during COVID-19” (Interview), characterized by her positive attitude towards ICT, active use of ICT-facilitated assessment activities, and extensive efforts to foster learner autonomy.

Like Xiaofang, Yuchen was positive towards integrating ICT in formative assessment at the emotional level.

I am excited about ICT, as it can provide scientific learning information to make writing instruction more responsive and create an anxiety-free classroom. (Interview)

Recognizing the benefits of ICT, along with the university-supported workshops on integrating ICT in EFL instruction, Yuchen actively explored ICT tools suitable for formative writing assessment and engaged students with various platforms at the physical–cognitive level, such as Pigai and Tencent Docs.

Pigai (http://www.pigai.org/) is an automated writing evaluation platform providing holistic scoring, end comments and corrective feedback on writing. Handy as this platform claims, Yuchen merely used it to collect writing assignments before COVID-19. The utter shift to online assessment pushed her to probe more of its formative aspects through self-initiated explorations and university-supported workshops. For instance, she explored Pigai’s error analysis function.

The error analysis function summarizes students’ language errors with relevant examples. With such data, I can accurately identify students’ typical errors and provide more focused, individualized instruction. (Interview)

Through datafication of students’ learning, Pigai enabled Yuchen to “get an accurate picture of students’ language errors” and “make teacher feedback more responsive to students’ needs” (Interview). For example, in one lesson, Yuchen presented the error analysis sheet (Fig. 2) and included exercises to address their language errors.

Fig. 2

Error analysis sheet in Pigai (original in Chinese)

Yuchen: The most frequent error is subject-predicate disagreement…. These sentences (e.g., “People’s activities is the most reason why they died out”) are taken from your writing. What are the problems and how can we improve them? (Videoed Lesson)

Admitting that “Pigai cannot give feedback on content and genre features” (Interview), Yuchen also incorporated Tencent Docs in writing assessment. For instance, in one lesson on expository essays, she organized students to provide and share peer feedback via Tencent Docs. She also provided immediate feedback on students’ feedback to conduct peer feedback training, e.g., “Picklejar noted that paragraphs 2 and 3 have no topic sentence and more concrete examples should be used. Very specific and constructive!” (Videoed Lesson). She commented,

Allowing teachers to collect peer feedback within a short time and students to read others’ feedback, Tencent Docs exposes students to more feedback to improve writing and allows me to use authentic examples to do peer feedback training. (Interview)

Thus, the affordances of ICT tools (e.g., rapid communication of ideas) improved the efficiency of formative assessment in EFL writing, which “increased [Yuchen’s] enjoyment in engaging students with online formative assessment in EFL writing” (Interview).

Similar to Manyun and Xiaofang, Yuchen also observed that students were unwilling to participate in writing activities in the fully online environment. To tackle it, Yuchen “tried to select suitable ICT tools to motivate students” and “guided students to use ICT to improve writing as independent writers and learners” (Interview). For instance, to foster learner autonomy, she instructed students to search for online resources (e.g., Khan Academy) to improve writing based on the error analysis information provided by Pigai, and used Tencent Docs to provide peer feedback training to enhance students’ assessment literacy. Her concern for students’ learner autonomy and high social engagement with online formative assessment were influenced by her belief in student-centred writing assessment.

Assessment is about students themselves. Teachers should adopt appropriate tools to actively involve students in writing assessment, and enhance their assessment literacy and independent learning. (Interview)

Yuchen’s integral stance on online formative assessment was also related to her interest in ICT, institutional support, and the development of ICT tools.

I am very interested in ICT. The university also encourages us to attend ICT training, share useful tips with colleagues, and explore ICT tools to improve our digital literacy...The technological companies also update ICT tools frequently. These all help me engage with online formative assessment. (Interview)

The account suggests that Yuchen’s high engagement with online formative assessment was the result of individual, contextual, and technological factors. Shaped by her socio-technical system, Yuchen treated ICT as an integral part of formative writing assessment and used ICT-facilitated activities to foster independent learners.

Discussion

Based on the three EFL teachers’ experiences of using ICT in formative writing assessment during COVID-19, this study reveals three types of engagement with online formative assessment and the individual, contextual, and technological factors influencing teacher engagement. The findings overall corroborate previous findings that teachers, influenced by their socio-technical systems, tend to display varied stances on formative assessment (e.g., Marshall & Drummond, 2006). The study also contributes to the existing literature by offering a systematic account of EFL writing teachers’ engagement with online formative assessment from the emotional, physical–cognitive, and social aspects, and adding new knowledge about how they used ICT for the purpose of formative assessment.

This study extends the “spirit” and “letter” kinds of engagement with formative assessment (Marshall & Drummond, 2006) by identifying three types of teacher engagement with online formative assessment, which were embodied by teachers’ varied emotional, physical–cognitive, and social investment in ICT in formative writing assessment. The first type is disturbing engagement, displayed by Manyun who was suspicious of ICT in EFL writing assessment. Her negative reactions not only led to limited exploration of ICT-facilitated activities, but also a decrease in formative assessment practices (e.g., peer assessment). Similar to the “letter” stance on formative assessment (Marshall & Drummond, 2006), such engagement paid little attention to promoting learner autonomy–-the core of formative assessment (Black & Wiliam, 2009).

In contrast, Xiaofang and Yuchen who manifested the auxiliary and integral types of engagement were emotionally more positive about ICT, and worked more intensively with ICT tools in writing assessment to monitor and facilitate student learning at the physical–cognitive and social levels. Believing in the auxiliary role of ICT, Xiaofang tried to find appropriate ICT tools and design ICT-facilitated activities to monitor students’ learning. Yet, she was uncertain about ICT and showed mixed feelings about online formative assessment (“frustrated” and “content”). She simply employed ICT tools to transform the prior writing assessment practices in f2f settings online and motivate students to write. While such auxiliary engagement has not been reported in the existing literature on teacher engagement, it lends support to the previous research (e.g., Lee, 2017) that many teachers, though feeling positive about formative assessment, may not integrate them intensively in writing assessment practices to develop learner autonomy due to the contextual constraints. Nevertheless, Xiaofang’s mixed feelings were not shared by Yuchen. Yuchen not only felt excited about the formative use of ICT in writing assessment, but also managed to promote students’ independent learning through intensive use of ICT-facilitated activities (e.g., peer feedback training). Altogether, the three types of engagement suggest that teacher engagement with online formative assessment in EFL writing was multidimensional (Klassen et al., 2013) and varied among teachers.

The study also showed that teacher engagement with online formative assessment in EFL writing was mediated by various contextual, technological, and individual factors, which supports the socio-technical perspective on teacher engagement (Whiteoak, 1997). First, in line with previous studies on formative assessment (e.g., Krishnan et al., 2020; Wang et al., 2020), contextual factors, such as institutional support and collegial communication influenced teachers’ engagement. When compared with Manyun who was discouraged by the insufficient university-supported teacher training, Xiaofang and Yuchen worked in universities where multiple opportunities were provided for teachers to participate in training on writing assessment and ICT, learn from colleagues, and enhance expertise in online formative assessment. The study also unravelled the under-explored technological influences on teacher engagement. Consistent with the existing literature (e.g., Gikandi et al., 2011), the affordances of ICT (e.g., rapid communication of ideas) contributed to formative assessment practices in EFL writing (Xiaofang and Yuchen). Nevertheless, different from William and Beam’s (2019) finding that ICT helps engage students in writing, the teachers in this study reported that ICT could decrease students’ participation in writing due to the absence of f2f interactions, which further influenced their engagement with online formative assessment.

Particularly noteworthy is that compared with the contextual and technological factors above, the individual factors were found to play a more decisive role in teacher engagement. Influenced by their beliefs, digital literacies, and/or learning and teaching experiences, teachers would respond to similar situations in varied ways, leading to different levels of engagement. For instance, Manyun, influenced by her own learning experience, held pessimistic views of ICT and cited students’ low participation as a reason for her low engagement with online formative assessment in EFL writing. By contrast, acknowledging ICT’s role in facilitating formative writing assessment and moving students forward, Xiaofang included more ICT-facilitated assessment activities to monitor students’ learning. However, her efforts were impeded by her limited digital literacy, especially inability to use learning data ICT provides to enhance independent learning. For Yuchen who had been implementing ICT and formative assessment for years, she was more knowledgeable about ICT and worked actively on it to transform students into independent learners. Her integral and intensive use of ICT to foster learner autonomy was also related to technology companies’ efforts to update ICT tools and her belief in student-centred writing assessment. Such views of teacher expertise, ICT development, and students contributed to her high engagement with online formative assessment. Taken together, teacher engagement with online formative assessment in EFL writing was realized through the interactions of teachers with their contexts and ICT tools within their socio-technical systems (Whiteoak, 1997), and the individual factors played a more decisive role.

Implications and Conclusion

Focusing on teacher engagement with online formative assessment in EFL writing during COVID-19, this study responds to Mimirinis’s (2019) call for exploring teachers’ assessment experiences in fully online contexts. By identifying three types of teacher engagement and the specific individual, contextual, and technological factors, it enriches our limited knowledge of how EFL writing teachers engage with online formative assessment during COVID-19 and beyond.

Several implications can be generated to support EFL writing teachers’ engagement with online formative assessment in similar contexts. To start with, since individual characteristics like teacher beliefs and digital literacy profoundly shape teacher engagement, attention should be paid to fostering EFL writing teachers’ optimistic views of ICT and expertise in integrating ICT in writing assessment. Teachers can take initiatives to challenge their beliefs about ICT and learn the means and ends of online formative assessment in EFL writing. Meanwhile, teacher training programs can engage teachers in classroom research and invite them to share their assessment experiences, which may transform their understandings of and attitudes towards online formative assessment. Secondly, given the institutional influence on teacher engagement, universities can provide multiple in-service teacher training opportunities to challenge writing teachers’ possible stereotypes about ICT and improve their expertise in using ICT in writing assessment (Williams & Beam, 2019). Thirdly, as Yuchen noted, technology companies also influence teacher engagement. To alleviate the possible technological barriers, companies can make ICT tools more responsive to teachers’ needs and provide clear instructions on how to integrate them in writing. Lastly, since teacher engagement with online formative assessment in EFL writing depends on the complex nexus of contextual, technological, and individual factors, a coordinated approach needs to be established to unite different stakeholders’ efforts and maximize the formative use of ICT in EFL writing classrooms.

The study is not without limitations. First, the questionnaire used to select participating teachers, though informed by previous research on online formative assessment in L2 writing and teacher engagement (e.g., Klassen et al., 2013), was not validated mainly due to the small number of responses (17) we received. Although the questionnaire was used only for participant recruitment and its data was not reported in the study, future research may continue to validate the questionnaire and use it for large-scale surveys to add quantitative insights about teacher engagement with online formative assessment in EFL writing. Additionally, in the study, not all the online lessons were video-recorded. For instance, Xiaofang recorded only two lessons, preventing us from obtaining a comprehensive picture of her engagement with online formative assessment. Future researchers can conduct longitudinal studies and make fuller use of lesson observation data. Given that teacher engagement with online formative assessment in EFL writing depends on a variety of contextual, technological, and individual factors, it is also promising to explore the ways to support writing teachers’ engagement with online formative assessment during and beyond the COVID-19 pandemic.

Appendices

Appendix 1: Teacher Engagement with Online Formative Assessment in EFL Writing During the COVID-19 Pandemic

Section One: Personal Background

• 1. Gender:_________;

• 2. Years of teaching experience:_________;

• 3. Have you received any training in language assessment? Yes____; No_____.

• 4. Have you implemented formative assessment (i.e., using assessment to promote learning and improve teaching) in EFL writing classrooms? Yes______; No______.

• 5. Have you tried to use information and communication technology (ICT) to help you assess EFL writing? Yes_______; No_______.

Section Two: Teacher Engagement with Online Formative Assessment in EFL Writing

Instruction: Below you will find a list of statements describing your experiences of implementing online formative assessment–-i.e., using information and communication technology (ICT) for the purposes of formative assessment in the online environment– in EFL writing. Please indicate your personal response to each of these statements by checking the number that best represents your experiences during the COVID-19 pandemic: 0-Never; 1-Rarely; 2-Occasionally; 3-Sometimes; 4-Often; 5-Frequently; 6-Always.

Statement	0 Never	1 Rarely	2 Occasionally	3 Sometimes	4 Often	5 Frequently	6 Always
1. I tried my hardest to find appropriate ICT tools/platforms to assess students’ writing during the COVID-19 pandemic
2. I worked intensively with ICT tools/platforms in writing assessment during the COVID-19 pandemic
3. I used ICT tools and platforms to help me diagnose what students can and cannot do in EFL writing during the COVID-19 pandemic
4. I used ICT tools/platforms (e.g., automated writing evaluation platforms) to give feedback on students’ writing during the COVID-19 pandemic
5. I used ICT tools/platforms to carry out peer assessment in writing classrooms during the COVID-19 pandemic
6. I used ICT tools/platforms to help students identify and address the problems they have in their writing during the COVID-19 pandemic
7. I used the assessment information that ICT tools/platforms provide to improve my writing teaching during the COVID-19 pandemic
8. I was excited about exploring the formative use of ICT tools/platforms in writing assessment during the COVID-19 pandemic
9. I loved using ICT tools/platforms to assess students’ writing during the COVID-19 pandemic
10. I found it helpful to use ICT tools/platforms to assess students’ writing during the COVID-19 pandemic
11. I found it challenging to use ICT tools/platforms to assess students’ writing during the COVID-19 pandemic
12. I felt frustrated while using ICT tools/platforms to assess students’ writing during the COVID-19 pandemic
13. I valued the information that ICT tools/platforms provide about the problems and strengths of students’ writing during the COVID-19 pandemic
14. I felt happy while engaging students with online formative assessment in EFL writing during the COVID-19 pandemic
15. I was aware of my students’ feelings in writing assessment during the COVID-19 pandemic
16. I connected well with my students in writing classrooms via ICT tools/platforms during the COVID-19 pandemic
17. I created a supportive classroom culture with the help of ICT tools/platforms during the COVID-19 pandemic
18. I was committed to using ICT tools/platforms to enhance students’ ability to assess English writing during the COVID-19 pandemic
19. I was committed to using ICT tools/platforms to helping students to learn English writing independently during the COVID-19 pandemic
20. I guided students to use ICT tools/platforms to self-assess and monitor their English writing
21. I encouraged students to use the learning information provided by ICT tools/platforms to think about how to learn English writing best
22. Please use one sentence to describe your experience of implementing online formative assessment in EFL writing during the COVID-19 pandemic.
23. Overall, your engagement with online formative assessment in EFL writing during the COVID-19 pandemic was high____; medium_____; or low_____.
24. We’d like to learn more about your experiences of online formative assessment in EFL writing during the COVID-19 pandemic. Are you willing to attend a 90-min interview? Yes_____ (your email address:___________________); No________

Appendix 2: Interview Protocol

• 1. Could you give a brief introduction of yourself (e.g., teaching experience, major writing teaching experience, research experience, and training experience)?

• 2. How did you usually assess EFL writing in the traditional f2f learning environment?

• 3. How do you understand formative assessment and online formative assessment in EFL writing?

• 4. How did you assess EFL writing during the COVID-19 pandemic (e.g., assessment focus, teacher feedback, peer feedback, rubric, and AWE)? How was your assessment this semester different from the writing assessment in the traditional f2f instructional environment?

• 5. What roles do you think ICT plays in assessing writing? What ICT tools have you used to teach and assess EFL writing? How do you integrate them in your EFL writing classroom? Why do you use them?

• 6. How did you feel when you tried to implement formative assessment in the online environment this semester?

• 7. What role do you think your students played in the writing assessment this semester? What did you do to establish a relationship with them in EFL writing classroom and help them learn independently?

• 8. Have you perceived any differences and similarities when you implemented formative assessment in EFL writing in f2f environments and online environments? Compared with the traditional f2f mode, what advantages and disadvantages do you think online formative assessment have?

• 9. What factors facilitated and/or hindered the implementation online formative assessment in your writing classrooms this semester?

• 10. Could you use one sentence to describe your engagement with online formative assessment in EFL writing this semester?

Appendix 3: Coding Scheme

Category	Description	Example
Physical–cognitive engagement	Participant indicated the extent to which they attended to and invested in online formative assessment during COVID-19	The use of Tencent Docs to give timely feedback (Xiaofang, Videoed lesson)
Emotional engagement	Participants demonstrated their affective responses to online formative assessment	I was suspicious of the necessity and benefits of doing online formative assessment (Manyun, Interview)
Social engagement	Participants indicated their concern for students especially learner autonomy in online writing assessment	I guided students to use ICT to improve writing as independent writers and learners (Yuchen, Interview)
Individual factors	Participants indicated how their beliefs, experiences and digital literacies influenced their use of online formative assessment in EFL writing	I don’t know how to use the learner information provided by ICT to foster students’ independent learning (Xiaofang, Interview)
Contextual factors	Participants indicated how the institutional and socio-cultural factors affected their use of online formative assessment	No training has been provided to push me to integrate ICT in formative writing assessment in our university (Manyun, Interview)
Technological factors	Participants indicated the affordances and constraints of ICT in online formative assessment	Pigai allowed me to get an accurate picture of students’ language errors (Yuchen, Interview)