Mediated by the individual, contextual, and technological factors within their socio-technical systems, the participants demonstrated three types of engagement with online formative assessment in EFL writing: disturbing, auxiliary, and integral.
Manyun: Online Formative Assessment as Disturbing
Manyun received training in formative assessment in translation, but never in EFL writing. Having implemented formative assessment for over two years in f2f writing classrooms in a university of foreign languages where students generally have an upper-intermediate English proficiency level, she felt that “the online environment disturbed its implementation during COVID-19” (Interview). Overall, she demonstrated “a low-level engagement with online formative assessment in EFL writing” (Interview), characterized by a suspicious attitude towards ICT, a limited exploration and use of ICT in writing assessment, and a de-emphasis on students’ independent learning.
Admitting that there existed many ICT tools for promoting formative assessment in writing, Manyun doubted “the necessity and benefits of doing online formative assessment” (Interview). Manyun’s scepticism was related to her learning experience: “My teachers never used ICT to assess writing…and I felt comfortable with the traditional f2f assessment” (Interview). It was also influenced by the insufficient university-supported teacher training: “No training has been provided to push me to integrate ICT in formative writing assessment in our university” (Interview).
Overall, Manyun had limited knowledge about using ICT in EFL writing and “rarely worked effectively with ICT in writing assessment” (Interview), demonstrating a low physical–cognitive engagement. Being unenthusiastic about exploring ICT for formative writing assessment, she mainly employed Microsoft tools and CCtalk (https://www.cctalk.com)—a live education platform allowing teachers to deliver lessons—to give teacher feedback to implement online formative assessment. When compared with the pen-and-paper assessment in f2f classrooms, she “gave more detailed and lengthy annotated comments on students’ electronic drafts, as the space allows [her] to do so” (Interview). Nevertheless, such lengthy written comments and detailed oral explanations did not seem to move students forward.
My feedback did not work equally well as in f2f classrooms in improving students’ writing…Due to the absence of f2f interactions, students did not take online learning and assessment seriously and participate in writing activities assiduously. (Interview)
In this regard, ICT’s inability to provide f2f-like interactions lowered students’ participation in writing assessment, thereby impeding their learning. Such technical limitation made Manyun query the effectiveness of online formative assessment and emotionally disengaged with it.
While realizing that students’ low participation was detrimental to learning, Manyun believed that “little could be done to influence students’ participation online” (Interview). Consequently, she “involved students less in writing assessment… and seldom used ICT to help students learn English writing independently during COVID-19” (Interview), showing an increasing de-emphasis on students’ autonomy. For instance, although she believed that peer assessment could benefit students’ writing and autonomous learning, she stopped using it.
Without f2f interactions, it is difficult to involve all the students in peer assessment and ensure that they benefit from it. So I did not use it this semester. (Interview)
The lack of f2f interactions in the online environment thus made Manyun lose faith in peer assessment, and consequently, lessened her social engagement with online formative assessment, to which peer assessment is essential.
Overall, due to her learning experience and limited knowledge of integrating ICT in writing assessment, students’ low participation in online learning, insufficient institutional support, and ICT’s inability to facilitate f2f-like interactions, Manyun was suspicious of and reluctant to embrace online formative assessment during COVID-19.
Xiaofang: Online Formative Assessment as Auxiliary
Xiaofang received professional training on formative writing assessment and practised it in writing classrooms in one research-oriented university with students of generally upper-intermediate and advanced English language proficiency. During COVID-19, she manifested an overall auxiliary stance on and “a medium level of engagement” (Interview) with online formative assessment, featured by mixed feelings about the use of ICT in writing assessment, extensive efforts to find appropriate ICT tools for formative assessment, and the use of ICT-facilitated assessment activities to monitor students’ learning.
Different from Manyun’s scepticism about ICT, Xiaofang recognized its value and promoted it in writing assessment.
ICT makes writing assessment more timely and flexible, since students and I can chat whenever we have problems. It also helps create a supportive learning community for students. (Interview)
While Xiaofang was positive in engaging with online formative assessment, she felt “challenged and frustrated” at first, as she was uncertain about “what ICT tools and tasks would be better…and how to effectively transform prior f2f assessment practices to the fully online context” (Interview).
Although Xiaofang’s unfamiliarity with ICT tools made her feel insecure, she was motivated to find appropriate ICT tools for writing assessment, because “the university emphasizes ICT-assisted instruction…and organized workshops to help teachers explore the formative use of ICT during COVID-19” (Interview). Such institutional support and agentive exploration led to her increasing physical–cognitive engagement with online formative assessment. For instance, she explored and successfully transformed writing conferences from f2f to the online mode with the help of QQ Messenger (https://www.imqq.com/), which allows for instant messages, voice, and video chatting. She commented,
Students contacted me online whenever they had difficulties in understanding my feedback, making teacher feedback more timely, transparent, and effective. (Interview)
Likewise, she followed her colleague’s suggestion to share students’ writing and give immediate feedback through Tencent Docs (https://docs.qq.com/), a free platform that allows users to create, share, and edit documents online. Taking one academic writing lesson as an example, Xiaofang asked students to describe one bar chart on employment pressure (Fig. 1) and commented on students’ writing samples via Tencent Docs, which allowed her “to share students’ writing and give them timely oral feedback as in f2f classrooms” (Interview).
Xiaofang: Jiawei used a double negative sentence to show his finding and found related data to support it. This is good…. However, the description is all about commonality. Can you find any differences between groups and compare them? (Videoed Lesson)
Although Xiaofang sometimes felt frustrated at her limited ability to use ICT tools, ICT enabled her to conduct writing conferences and provide timely feedback to move learners forward by removing the spatial and temporal constraints. Such experience increased her positive reactions to online formative assessment: I felt so contented when I saw my students inspired.
Similar to Manyun, Xiaofang found that the lack of f2f interactions in the fully online environment lowered students’ motivation to write and hindered their learning. Thus, she endeavoured to create a supportive classroom culture and monitor student learning by “giving more encouraging teacher feedback to motivate students to write and revise” and “using ICT to include more assessment activities and provide more timely and frequent feedback” (Interview). For instance, Xiaofang integrated more assessment activities into writing instruction to motivate students to preview and review reading materials concerning writing with Moso Teach (https://www.mosoteach.cn/), a mobile-based teaching management platform that allows teachers to upload learning materials, create activities like discussion and quizzes, and check students’ learning. Such efforts to use ICT-facilitated assessment activities to motivate and monitor students’ learning were influenced by her belief that “writing assessment is not to evaluate how well students can write but to motivate them to learn” (Interview).
Despite her concern for students’ learning and growing ability to use ICT in writing assessment, Xiaofang seldom utilized the assessment information generated by ICT tools to foster students’ independent learning. She reflected,
I am not a teacher with high digital literacy. I know Moso Teach can provide students’ learning information, but I don’t know how to use such information to improve teaching and students’ independent learning. (Interview)
Although the learning information provided by ICT, such as analytical reports of students’ performance can be helpful for online formative assessment, Xiaofang’s limited digital literacy hindered her from unleashing its potential, which frustrated and hampered her social engagement with online formative assessment.
To conclude, while Xiaofang believed in the auxiliary role of ICT in writing assessment and became more capable of using ICT for formative assessment with the institutional support and agentive exploration, her limited digital literacy prevented her from facilitating students’ autonomous learning. Consequently, she took an overall simplistic approach to online formative assessment by using ICT primarily to monitor student learning as an “administrator” and “motivator” (Interview), and demonstrated a mixed feeling of frustration and contentment towards it.
Yuchen: Online Formative Assessment as Integral
Having integrated formative assessment and ICT in EFL writing for over six years in one university of foreign languages where ICT-assisted instruction is vigorously promoted and students have an overall upper-intermediate level of English language proficiency, Yuchen believed that “online formative assessment is integral to writing assessment” (Interview). She showed “a high-level engagement with online formative assessment during COVID-19” (Interview), characterized by her positive attitude towards ICT, active use of ICT-facilitated assessment activities, and extensive efforts to foster learner autonomy.
Like Xiaofang, Yuchen was positive towards integrating ICT in formative assessment at the emotional level.
I am excited about ICT, as it can provide scientific learning information to make writing instruction more responsive and create an anxiety-free classroom. (Interview)
Recognizing the benefits of ICT, along with the university-supported workshops on integrating ICT in EFL instruction, Yuchen actively explored ICT tools suitable for formative writing assessment and engaged students with various platforms at the physical–cognitive level, such as Pigai and Tencent Docs.
Pigai (http://www.pigai.org/) is an automated writing evaluation platform providing holistic scoring, end comments and corrective feedback on writing. Handy as this platform claims, Yuchen merely used it to collect writing assignments before COVID-19. The utter shift to online assessment pushed her to probe more of its formative aspects through self-initiated explorations and university-supported workshops. For instance, she explored Pigai’s error analysis function.
The error analysis function summarizes students’ language errors with relevant examples. With such data, I can accurately identify students’ typical errors and provide more focused, individualized instruction. (Interview)
Through datafication of students’ learning, Pigai enabled Yuchen to “get an accurate picture of students’ language errors” and “make teacher feedback more responsive to students’ needs” (Interview). For example, in one lesson, Yuchen presented the error analysis sheet (Fig. 2) and included exercises to address their language errors.
Yuchen: The most frequent error is subject-predicate disagreement…. These sentences (e.g., “People’s activities is the most reason why they died out”) are taken from your writing. What are the problems and how can we improve them? (Videoed Lesson)
Admitting that “Pigai cannot give feedback on content and genre features” (Interview), Yuchen also incorporated Tencent Docs in writing assessment. For instance, in one lesson on expository essays, she organized students to provide and share peer feedback via Tencent Docs. She also provided immediate feedback on students’ feedback to conduct peer feedback training, e.g., “Picklejar noted that paragraphs 2 and 3 have no topic sentence and more concrete examples should be used. Very specific and constructive!” (Videoed Lesson). She commented,
Allowing teachers to collect peer feedback within a short time and students to read others’ feedback, Tencent Docs exposes students to more feedback to improve writing and allows me to use authentic examples to do peer feedback training. (Interview)
Thus, the affordances of ICT tools (e.g., rapid communication of ideas) improved the efficiency of formative assessment in EFL writing, which “increased [Yuchen’s] enjoyment in engaging students with online formative assessment in EFL writing” (Interview).
Similar to Manyun and Xiaofang, Yuchen also observed that students were unwilling to participate in writing activities in the fully online environment. To tackle it, Yuchen “tried to select suitable ICT tools to motivate students” and “guided students to use ICT to improve writing as independent writers and learners” (Interview). For instance, to foster learner autonomy, she instructed students to search for online resources (e.g., Khan Academy) to improve writing based on the error analysis information provided by Pigai, and used Tencent Docs to provide peer feedback training to enhance students’ assessment literacy. Her concern for students’ learner autonomy and high social engagement with online formative assessment were influenced by her belief in student-centred writing assessment.
Assessment is about students themselves. Teachers should adopt appropriate tools to actively involve students in writing assessment, and enhance their assessment literacy and independent learning. (Interview)
Yuchen’s integral stance on online formative assessment was also related to her interest in ICT, institutional support, and the development of ICT tools.
I am very interested in ICT. The university also encourages us to attend ICT training, share useful tips with colleagues, and explore ICT tools to improve our digital literacy...The technological companies also update ICT tools frequently. These all help me engage with online formative assessment. (Interview)
The account suggests that Yuchen’s high engagement with online formative assessment was the result of individual, contextual, and technological factors. Shaped by her socio-technical system, Yuchen treated ICT as an integral part of formative writing assessment and used ICT-facilitated activities to foster independent learners.