1 Introduction

Computational thinking (CT) is an essential skill that is increasingly important in today’s digital world and is closely related to digital competency and 21st century skills. Wing (2014) defined CT as “the thought process to formulate a problem, express solutions in a form that can be effectively carried out by an information-processing agent”. Besides, Wing (2017) emphasized that nowadays coding is equally essential compared to reading, writing, and arithmetic skills. Therefore, CT is a basis of computational problem-solving skills which incorporates computer and human to solve complex problems.

With its significance, many countries worldwide have introduced CT as part of their formal national education (Heintz et al. 2016). Students can learn, and practice CT through coding which is a new form of expression and a new context for learning (Grover et al. 2013; Djambong and Freiman 2016; Kwon 2019). Scratch is one of the most widely used coding environments in primary education (McGill et al. 2020). It is a visual block-based programming language, designed especially for learners between 8 to 16 years old, to learn fundamental computational concepts by connecting  graphical blocks of computer codes in a manner similar to how they snap Lego bricks in the physical world. This mechanism can help minimize the cognitive load of remembering commands, syntax, and punctuation.

However, it is challenging to effectively conduct hands-on coding activities for many students in a regular classroom (Tissenbaum et al. 2018; Sentance and Csizmadia 2017). Students are individually different in terms of knowledge background, cognitive abilities, learning pace, and learning styles. Neglecting individual differences and providing similar learning experience for every student may not lead to desired outcomes (Cook et al. 2018). Hence, teachers are required to pay attention to individual progression, to diagnose individual strengths and weaknesses, and to provide assistance when needed.

Considering a traditional school setting, the following two challenging factors: a high student-teacher ratio, and a limited instructional time, have added extensive complexity for teachers to deal with an overwhelming range of student diversity. Assignments are a useful formative assessment instrument to determine for each student, the topics (s)he can master well and the topics (s)he struggles; hence helping teachers to early identify at-risk students. However, assignment grading is a time-consuming task which prevents teachers to frequently implement at scale (Alves et al. 2019). Regardless of available diagnosis information, precise intervention is hard to implement. As a result, at-risk students would get lost and lose learning interests, and thus leading to ineffective outcomes and educational waste (i.e., time, money, and resources that produces no change in outcomes) (Basawapatna et al. 2015).

This paper proposes ScratchThAIFootnote 1, a conversation-based learning support framework for computational thinking development as the main contribution (Katchapakirin and Anutariya 2018). The framework not only supports teachers to conduct hands-on coding activities in the classroom, but also assists students for outside classroom practices. It adds a virtual conversational chatbot, automated assessment, and adaptive features to the existing learning environment, i.e., Scratch coding websiteFootnote 2. The chatbot motivates students to further practice CT problems, and provides timely personalized learning assistance. Automated assessment gives instant feedback to adjust their understanding. Adaptive instruction (i.e., exercises and learning materials) is suggested to intervene and fill the student’s knowledge gap. With a cloud-based design, teachers can monitor through a dashboard how satisfactorily the whole class and each individual student progresses both in-class learning and outside classroom practices. Furthermore, the framework observes student-system interaction during a practical session which is hard to accomplish in practice (Katchapakirin and Anutariya 2019). Coding progression, learning preferences, learning behaviors, and learning activities are maintained as learner profile for teachers to manage and provide precise intervention.

Based on the developed ScratchThAI framework, the paper also presents a pilot study which explores the framework’s effectiveness in enhancing student outcomes, as well as in supporting teachers to improve their instruction and to better facilitate students.

The paper is organized as follows. Section 2 discusses important challenges faced in CT education worldwide, and in Thailand, reviews existing works and identifies the research gaps, Section 3 elaborates ScratchThAI framework, Section 4 presents its pilot study in a primary school, and Section 5 concludes.

2 Literature review

2.1 Challenges in CT education worldwide and in Thailand

Introducing CT as part of national basic curriculum in many countries worldwide has imposed various challenges, which may differ from country to country depending on their education systems, infrastructure, available resources and supporting systems, as well as teacher and student related factors, etc. Table 1 summarizes important challenges, categorized into the following three aspects: teachers, students, and environment, and faced by different countries.

In Thailand, CT was introduced as part of the national standard curriculum in 2017. Katchapakirin and Anutariya (2019) identified important challenges encountered by various Thai primary schools after the implementation of 3-year roll-out of the first Thailand Computing Science Curriculum in 2018. The research confirmed that the following challenges related to teachers, students and environment were priority ones: (1) teachers’ lacking CT fundamental knowledge and confidence in teaching CT, (2) mixed-ability class in terms of computing literacy, mathematical and/or problem-solving skills as well as learning and achievement gaps, and (3) infrastructure problem and limited school hours as compared to the curriculum’s expected outcomes.

In this study, we focus on supporting CT education in young learners under these three challenges: P1: Mixed-ability students in the same class, P2: High student-teacher ratio and P3: Learning-hour limitation.

Table 1 The CT development education challenges
Full size table

2.2 Existing work in teaching and learning support system for CT development

This section reviews recent personalized and adaptive systems. Tikva and Tambouris (2021) provided a broad overview of teaching and learning support systems for CT development through programming in K-12 education. AutoThinking (Hooshyar et al. 2019; Hooshyar et al. 2021) is an adaptive CT game to instill primary and secondary students with three important CT concepts: sequence, conditional, and loop and CT skills. Adaptivity within a gameplay includes adaptive hints, feedback, and tutorials. CTSiM (Basu et al. 2014, 2016) contextualizes CT with Science and Ecology for middle school students. It embeds a virtual agent to provide feedback to struggling students, and keeps them motivated through predefined agent-student conversations. AgentSheets and AgentCubes (Repenning et al. 2010) introduces an adaptive scaffolding framework with CT pattern graphs. The framework can track, interpret students’ computational artifacts in games and science simulations and present in knowledge gaps using CT pattern graph. Flowchart-based Intelligent Tutoring System (FITS) (Hooshyar et al. 2015) is an adaptive intelligent tutoring system that utilizes flowchart development to help students conceptualize their solution design. The system provides adaptive guidance by dynamically defining Sub-flowcharts for scaffolding low-performance students. Although the system can facilitate conceptualization, it cannot visualize the result of the designed solutions. REACT (Koh et al. 2014) is a Cyberlearning tool for computer science education with a teacher dashboard that visualized learning progression of students in real-time, allowing teachers to give support to students who significantly need help. The system merely monitors students learning and informs the teachers of the students who are in trouble but does not give feedback directly to the students.

Although various existing research reviewed above can provide personalized and adaptive features to assist learners in CT skill development, Sentance and Csizmadia (2017) found that a computing course still had the largest achievement gaps among students when compared to other subjects. In addition, Montiel et al. (2021) reported a present need for frameworks to support teaching CT in the classroom. That is, there was a lack of the following important capabilities in existing teaching-learning support tools to effectively assist a teacher to conduct hands-on coding activities under the three important challenges in a regular classroom: (i) learning progression tracking, (ii) diagnosis and early identification of at-risk students with slow learning, low performance, or disengagement, and (iii) collection and analysis of learning activities and learning behavior for both in-class and outside-school-hours.

This study aims to fill such gaps by employment of technology-enhanced learning to deal with CT development to enhance student learning and teacher teaching quality. Table 2 compares existing research and the proposed research based on the supporting CT skill development.

Table 2 Comparison of existing research and the proposed research to support CT skill development
Full size table

3 Proposed research

This section presents the research questions, the research design overview and the design and development of the proposed framework, ScratchThAI, which employs innovative technology-enhanced learning (TEL) as well as validated educational approaches to promote computational thinking development for young learners under the complex Thai school context influenced by the three important challenges: P1 (mixed-ability students in the same class), P2 (high student-teacher ratio) and P3 (learning-hour limitation)

The paper formulates the following research questions:

R1::

How can we construct a TEL-based framework for CT skill development in young learners and dealing with the three important challenges?

R2::

How effectively can the proposed framework improve learning performance and enhance teaching-learning activities?

R3::

What are the students’ and teachers’ perceptions toward the proposed framework’s usage experience regarding the learning motivation and engagement as well as the learning flexibility dimensions?

3.1 Research design overview

Based on the defined research questions R1-R3, this section proposes ScratchThAI framework and elaborates its design overview using influenced educational approaches and informed-empirical studies as a theoretical basis Fig. 1 illustrates for each CT challenge, the adopted design strategies & TELs, the implemented ScratchThAI modules and the evaluation criteria.

Fig. 1
figure 1

Research Design Overview

Full size image

Concerning the challenge P1 (mixed-ability students), ScratchThAI employs the learner-centered approach and personalized learning approach to address the challenge where students are able to learn at their own knowledge level, pace and style (Bjork and Bjork 2011; Deunk et al. 2018; OECD 2012; Schleicher 2016; Peng et al. 2019). Personalized & Adaptive Technology (T1), and Automatic CT Assessment (T2) are the key enabling technologies that can help improve students’ progress and engagement (Campos et al. 2012; Hattie and Timperley 2007; Marwan et al. 2020).

To overcome the challenge P2 (high student-teacher ratio), important strategies include: (i) Tracking of at-risk students, (ii) Provision of assistance to teachers, and (iii) Provision of instructional support to students. The enabling TELs are T1, T2, T3 and T4 where Conversational Agent Technology (Chatbot) (T3) can be adopted as a personal teaching assistant and Cloud-based & Web Technology (T4) can improve the system usability and availability for both teachers and students (Hailikari et al. 2008; Ssharratt 2017).

The challenge P3 (learning-hour limitation) is overcome by the provision of flexible learning in time and place strategy. With Internet access, the Cloud-based and Web technology (T4) allows ubiquitous learning, and enable teachers to monitor students’ progress both in and outside the classroom practices.

The proposed ScratchThAI framework is designed and constructed to provide learning experiences tailored to individual’s needs. Students can learn CT concepts and practice online coding anywhere, anytime. Teachers can track learning progress and performance of the whole class as well as of individuals through the dashboard and can design suitable intervention within limited school hours. Details are described in Section 3.2.

Figure 1 further denotes that to address the research questions R2-R3, the research performs both quantitative and qualitative analysis with the following defined evaluation criteria: (1) Learning Performance Improvement, by determining pretest posttest scores, (2) Teaching and Learning Enhancement by determining the number of students who can complete all assigned exercises as well as the average learning time and teaching time to complete each exercise, (3) Student Satisfaction and Perception and (4) Teacher Satisfaction and Perception. Details are elaborated in Section 4.2.

3.2 System use cases and architecture

Based on the design overview elaborated earlier, ScratchThAI framework is constructed. Figure 2 illustrates the system architecture of ScratchThAI, which supports the following four student use cases:  U1: Request Personalized Practices, which assigns exercises based on the current student’s performance and the teacher’s predefined lesson plan, U2: Request Learning Material, which prompts a list of learning materials such as Scratch block information and CT concepts, U3: Practice and Submit Code, which allows a student to perform a coding practice, submit the code and obtain the automatic grading and feedback, and U4: View Student Progression and Performance, which displays the student’s individual progression and performance report. Furthermore, it supports two teacher use cases: U5: View Dashboard, for teachers to display the whole class progression and learning status, and hence enabling them to identify and monitor potential at-risk students easily. U6: View Report, to support teachers to view learning progression and performance of each individual student in details.

Fig. 2
figure 2

ScratchThAI system architecture

Full size image

The system architecture organizes the six designed modules of ScratchThAI into three layers: User Interface, Precision Education, and Data Layers, and decomposes into components denoted by rectangle shape. Components with blue solid line represents the completely developed software. While gray solid line components represent utilized existing tools i.e., MIT Scratch, Chatbot Engine. The components denoted by a dashed line border means future components which are parts of future work.

3.2.1 User interface layer

This layer consists of two modules: (i)ScratChatbot & CT Practice, and (ii)Visualization Modules. The ScratChatbot& CT Practice Module comprises four components: ScratChatbot, Chatbot Engine, Connector, and and CT Practice Environment i.e., MIT Scratch Environment in this study. The ScratChatbot component, designed as a browser extension (aka. add-on or plugin), allows seamless integration with the MIT scratch environment. Figure 3 shows the ScratchThAI extension, circled by green color, the ScratChatbot UI for the student use cases U1 – U3 and MIT Scratch Environment. The bot-student conversations are in a kid-friendly manner through simple language, avatars, emoticons, and images to attract primary-level students. To send a message, students can simply select one of the dynamic buttons/messages suggested based on the current context. In this layer, the Chatbot Engine component identifies students’ intentions, extracts relevant information. The Connector component provides interoperability with the other modules in the Precision Education Layer.

Fig. 3
figure 3

Screenshots of ScratChatbot user interfaces to support use cases U1 – U3

Full size image

The Visualization Module consists of Dashboard and Report components to supports the teacher use cases U5: View Dashboard and U6: View Report. As shown in Fig. 4, the dashboard presents the real-time class progression for a teacher to identify potential at-risk students having slow progress or low performance. To be specific, the bar graph shows the whole class progression with respect to the number of practice exercises completed by each student and grouped by the exercise difficulty levels (Level 0 to Level 5). The teacher could, thus, notice that the students st110005 and st140027 progressed slowlier than peers, i.e., completing only 2 exercises of Level 0. The coding practice logs then show in detail, for each individual student, the in-progress exercise, practice history, elapsed time, and start time; allowing the teacher to further inspect that the students had difficulty in EX105 exercise and had been practicing for 20 minutes. Thus, the teacher could provide a timely assistance and a closer monitor on the students’ learning performance. Figure 5. shows learning performance reports of specific students with respect to their CT concept understandings. From such reports, the teacher could see that the students st140040 and st140048 had limited understanding in the loop concept, while the student st140048 also struggled with the condition concept.

Fig. 4
figure 4

Teacher dashboard: real-time class progression and practice history (Use Case U5)

Full size image
Fig. 5
figure 5

Individual student learning performance reports (Use Case U6)

Full size image

3.2.2 Precision education layer

This layer has three modules: (i)Diagnosis, (ii)Treatment & Prevention, and (ii)Learning Insight & Analytics. The first module, Diagnosis, focuses on automatically diagnosing students’ CT skill development during the learning to provide precise and reliable information for adaptive instruction and possible intervention. It has two components: Programming Assessment and Problem-Solving Assessment. The first component automatically checks and grades a student’s programming code which corresponds to particular CT concepts by evaluating the code against a teacher’s predefined correct coding specification. The specification can be described using a combination of correct coding patterns, must-have or must-not-have blocks, and a maximum number of blocks allowed in the students’ code. The output score is computed as a weighted summation of the coding similarity score, which uses Jaccard’s similarity coefficient, together with the coding satisfaction score. The Problem-Solving Assessment is a future component which evaluates the student’s CT thought process, comprising: problem decomposition, abstraction, pattern recognition, and algorithmic thinking for a more precise and comprehensive diagnosis.

The Treatment & Prevention Module comprises two components: Personalized Assignment and Material Support to support an adaptive learning path for students. The first component suggests an appropriate personalized practice for a student, based on the teacher’s lesson plan and the student’s learning status and progression which includes his/her understanding level/scores for each CT concept, measured by the Diagnosis Module, described earlier. The other component, Material Support, facilitates the learning of challenging topics by allowing students to ask ScratChatbot for learning materials related to specific CT concepts or Scratch coding blocks. The materials are available in various formats for students to choose according to their preferences including textual explanations, infographics, coding examples, and video clips. 

The Learning Insight & Analytics Module comprises two components: Collector and Analyzer. The former is responsible for logging all learning activities and related data for a data-driven approach, which are maintained in the Data and Knowledge Layer and can be categorized into (1) Coding practice logs: source code evolution, the number of runs, edit distance from run by run, and time spent on each exercise; (2) Learning activity and behavior logs: practice timestamps, frequency of practices, materials learned, time spent on each material, and any web addresses students open while practicing; (3) ScratChatbot conversation logs. The Analyzer component is responsible for in-depth analyzing the collected logs using learning analytics and educational data mining techniques to gain more insight into individual variability in learning behavior, learning style, and learning preference of each student. Hence, proper intervention and prevention can be provided as the aims of the precision education approach. Note that, this Analyzer component is part of future work.

3.2.3 Data and knowledge layer

This layer maintains Data Store and Knowledge Base for the system and comprises the following:

  • D1: Student profiles, maintaining students’ performance & learning progression history

  • D2: Coding practice logs

  • D3: Learning activity and behavior logs

  • D4: ScratChatbot conversation logs

  • K1: Predefined conversation flow

  • K2: Lesson plan

  • K3: Learning objects and metadata

  • K4: Programming assessment specification

  • K5: Personalized assignment strategy

A relational database and a document store are used for this layer. In the future, we also plan to incorporate ontology and knowledge graph to formally model the system’s knowledge base.

3.3 System implementation

For this section, we describe the implementation of ScratchThAI system architecture in this study. Table 3 summarizes techniques, programming languages, tools, and production detail for the components in the system architecture.

Table 3 ScratchThAI Implementation
Full size table

4 ScratchThAICamp: Adopting and evaluating ScratchThAI in computing science classroom

ScratchThAICamp, a 2-day scratch coding camp, was organized to perform a pilot study of adopting ScratchThAI system as part of the Grade 5’s computing science course in a provincial primary school in Thailand. The study explored the effectiveness and practicability in applying ScratchThAI to promoting student engagement, improving learning performance, and to enhancing teaching-learning activities. In particular, the study was conducted to answer the defined research questions R2 and R3, using the defined evaluation criteria as introduced in the Section 3.1.

4.1 Lesson plan, material and assessment design

The camp was designed to introduce CT concepts and coding experience using Scratch. It consisted of four important CT lessons: sequence, event handling, conditional statement and looping. For the designed lessons, fundamental knowledge of the four CT concepts, basic block programming concepts as well as related mathematics knowledge, e.g., geometry coordinate and angle, were developed and provided as supporting learning materials in various formats, e.g., textual explanations, infographics, and video clips. In addition, practice exercises were designed with six difficulty levels (Level 0 to Level 5) and were defined as either mandatory or optional. An optional exercise would be assigned as an extra practice by the ScratChatbot to the students who could not master certain concepts and are required to repeat the learning. This personalized alternative exercise was determined and suggested by ScratchThAI’s adaptive feature. In addition, students could interact with the ScratChatbot to get helps and request for materials or explanations related to unclear CT concepts at any time. As a result, students could learn at different paths and paces depending on their prior knowledge and abilities to learn new concepts.

To measure a student’s prior knowledge, a pretest was utilized and comprised 20 multiple-choice questions. A student could choose “I do not know” or leave blank if they did not know the answer. To assess the learning outcomes, a posttest was taken as a summative assessment. The posttest was identical to the pretest for reliability and validity.

Note that all learning materials, exercises and assessment specifications for this pilot study were co-developed and validated by ten computing science teachers from three primary schools in Thailand.

4.2 Evaluation design

The camp, was organized in June 2020 at a provincial primary school in Thailand with a face-to-face learning experience. A total of 45 5th-graded students participated in this camp and was divided into two groups: an experimental group and a control group. The former practiced their CT skills with ScratchThAI, while the latter learned in an ordinary environment. Figure 6 illustrates that the camp was designed with eight learning activities: pretest, tutorials, coding practices, posttest and interviews with students and teachers.

Fig. 6
figure 6

Pilot study activities

Full size image

To answer the defined research questions R2 and R3 regarding the three important dimensions: learning performance improvement, teaching and learning enhancement, and student/teacher perceptions toward their ScratchThAI usage experience, corresponding quantitative and qualitative research methods were employed as elaborated below.

For performance improvement analysis, a quasi-experimental research with nonequivalent control group pretest-posttest design was adopted. It compared the performance improvements of both within groups and between groups. The score differences between the pretest and posttest within group could identify how scores changed from pretest to posttest. On the other hand, the analysis of gain scores could show the effectiveness of practicing with ScratchThAI. Hence, the hypothesis statements for the study are:

  • \(HS_{1}\): The mean for posttest scores is greater than the mean for pretest scores in the experimental group.

  • \(HS_{2}\): The mean for posttest scores is greater than the mean for pretest scores in the control group.

  • \(HS_{3}\): The mean of gain scores for the experimental group is greater than the mean of gain score scores for the control group.

To analyze how ScratchThAI can better enhance and support teaching and learning, the study compared interesting descriptive analytic between students of the two groups on their learning behavior, number of practices and time taken to accomplish an exercise, extra personalized learning support needed. 

Lastly, to perform an initial investigation on the student/teacher satisfaction and perceptions toward the adoption of ScratchThAI, two qualitative techniques were utilized. First, a focus group discussion with the students of the experimental group was conducted to enable students to share their opinions or on top of their peers’ views on the following predefined topics: (1) Learning Motivation and Engagement, (2) Learning Flexibility, Efficiency and Effectiveness, and (3) Additional Features Suggestions. Second, a semi-structured interview was performed to elicit the perspectives from the teachers who observed the entire learning activities of both groups related to the following points: (1) Learning Behaviors, Motivation and Engagement, (2) Learning Flexibility and Learning Improvement, (3) ScratchThAI Adoption by Teachers, and (4) Limitations and Suggestions.

4.3 Results

4.3.1 Learning performance improvement

Figure 7 illustrates the distribution of the pretest, posttest and gain scores of the experimental group and the control group. Table 4 summarizes key descriptive statistics, the resulting normality data distribution tests, and the obtained t-Tests using the significance level of 0.05, which can be interpreted as follows:

  • For HS1, the mean of the posttest scores of the experimental group was greater than the mean of the pretest score (p-value = 0.00 \(< \alpha \)).

  • For HS2, the mean of the posttest scores of the control group was not significantly higher than the mean of the pretest scores (p-value = 0.06 \(> \alpha \)).

  • For HS3, the mean of the gain scores of the experimental group was significantly higher than the mean of the gain scores of the control group (p-value = 0.03 \(< \alpha \)).

Hence, from the pilot study, it was found that practices with ScratchThAI could lead to learning performance improvement in computational thinking ability (HS1, HS3) and impact on CT education (effect size= 0.8).

Fig. 7
figure 7

Histogram of the pretest, posttest and gain scores of both experimental group (E) and control group (C)

Full size image
Table 4 Learning performance improvement: hypothesis testing
Full size table
Table 5 Teaching-learning enhancement: descriptive statistics
Full size table

4.3.2 Teaching and learning enhancement

Based on the learning logs collected and maintained automatically by ScratchThAI for students of the Experimental Group as well as the data recorded manually for the Control Group, Table 5 compares interesting descriptive statistics between the two groups with regard to their learning behavior, practice accomplishment, personalized learning support needed. The table shows that there were the total of 10 mandatory and 4 optional exercises (2 of which were challenging ones) as part of the designed lesson plan with 6 difficulty levels covering the important 4 CT topics.

Important findings drawn follow. For the control group, the learning activities were led by a teacher. That is, based on the designed lesson plan, the teacher selected practice exercises and assigned to the whole class using the same teaching-learning pace. Thus, fast-learning and average-learning students must wait for some slow-learning students to finish their works, and then the whole class would move together to the next topic/exercise. In addition, the teacher had to grade the student submissions manually to check their understanding of the learning topics.

On a contrary, students in the experimental group had more flexibility in learning at different pace and did not need to wait for their peers to accomplish the practices. Therefore, fast-learning students could progress their practices more efficiently, while slow-learning students could get help from the teacher directly or from the system (in term of extra practice or extra learning materials). In addition, based on the automatic code checking and grading functionality of the system, the learning of the whole class progressed much faster, since it did not need to wait for the teacher to manually check the correctness of each student’s code. That is, the table shows that for the topics of the level 1 and above, students of the control group took much longer time to accomplish than the experimental group. Furthermore, the teacher of the control group had to skip some mandatory exercises, e.g., sequence/Ex104, condition/Ex109/Ex110 due to slow class progression.

The number of exercises practiced by students of the experimental group (control group) were: 10 (resp. 7) mandatory, 2 (resp. 0) optional, and 2 (resp. 2) challenge exercises. In total, the number of students from the experimental group who could successfully accomplish all 10 mandatory exercises and the two challenge ones was 13 (dropout rate 41%), whereas that of the control group was 5 (dropout rate 76%).

4.3.3 Student satisfaction and perceptions

The students of the experimental group participated in a focus group discussion to share and discuss their satisfactions and perceptions regarding their experience in learning and practicing with ScratchThAI. Important findings are threefold:

  1. 1.

    Learning Motivation and Engagement: With ScratchThAI, the students were motivated, inspired, excited and had less pressure to obtain feedback and to get their exercises graded by ScratChatbot. Most students preferred to learn with ScratchThAI over the traditional style as well as to ask for help and to request for learning materials from ScratChatbot rather than by searching and reading books by themselves. All students agreed that ScratchThAI could motivate and engage them to practice and to put more effort in learning with the goal to achieve higher scores and to rank higher in the class leaderboard.

  2. 2.

    Learning Flexibility, Efficiency and Effectiveness: Most students agreed that through the Request Learning Materials (U2) function, ScratchThAI could support them to understand the fundamental CT concepts as well as related topics such as geometry, coordinate, and angle better. They also had flexibility to select the types of learning materials they preferred at their own pace and their learning style. Considering coding practices, since ScratChatbot could assign students to practice additional exercises on the topics that they could not complete or did not pass, the students agreed that this iterative practice with immediate feedback could effectively strengthen their competency and understanding.

  3. 3.

    Additional Features: Students suggested to have a feature to re-submit and ask for more exercises based on their interests to improve their overall scores and their ranks in the leaderboard. Additional gamification features could also help engage and motivate them more.

4.3.4 Teacher satisfaction and perceptions

From the close observation and from the pilot study during the entire learning activities, the teachers recorded and reported the following important points:

  1. 1.

    Learning Behaviors, Motivation and Engagement: In the experimental group, students spent some time at the beginning to understand the mechanism of ScratChatbot for learning material request, coding practice, exercise submission, and personalized practice request. After that they were motivated to complete their personal learning plan, stayed engaged and interacted actively with the ScratChatbot. Moreover, with ScratchThAI’s score announcement, the leaderboard and the immediate feedback obtained after each practice submission, the experimental group students were motivated to self-learn, while the control group students had to wait for the teacher assistance and code verification when they faced learning difficulty.

  2. 2.

    Learning Flexibility and Learning Improvement: The teachers observed that ScratchThAI helped promote effective and flexible learning in the following ways: First, students interacted with ScratchThAI to progress their learning at their current knowledge level, and to practice coding exercises from the basic level to a more advanced one. Furthermore, ScratchThAI provided personalized learning assistance for diverse students to improving their learning through its additional practice assignment, automatic assessment and immediate feedback; hence, enabling mixed-ability students in the same class to learn at different paces. In other words, fast learners could continue with their exercises and quickly increase their progress, whereas slow learners or struggling students could get help either from ScratchThAI or from the in-class teachers.

  3. 3.

    ScratchThAI Adoption by Teachers: Teachers agreed that adopting ScratchThAI as a supporting learning tool in the computing science subject could not only enhance students learning experience, but also reduce teachers workload, and enable them to track and develop students’ learning performance individually. Moreover, since ScratchThAI can promote flexible learning regarding time and place, it can overcome the problem of limited school-hours. Students can use ScratchThAI at home as part of the assigned homework. Furthermore, several co-curricular activities can be organized.

  4. 4.

    Limitations and Suggestions: Even though ScratchThAI could alleviate the problem of mixed-ability students in both computational thinking and related mathematical skills, one concern was that the learning and achievement gaps may expand if low-performing and low-engaging students do not give extra effort and/or teachers do not provide timely intervention to assist their learnings.

4.4 Discussion

Interesting findings from the pilot study show that compared to the traditional learning method, ScratchThAI could improve learning performance, enhance teaching-learning activities, gain satisfaction and obtain positive perceptions from both teachers and students under the aforementioned challenges i.e., P1 (mixed-ability students in the same class), P2 (high student-teacher ratio) and P3 (learning-hour limitation). Survey feedback along with the observed learning behavior have pointed to the following factors:

  1. 1.

    ScratchThAI helps enhance teaching-learning activities for mixed-ability students by shifting the classroom from a teacher-led one into a more learner-focused one, where each student has more flexibility in learning at their own pace. Hence, fast learners can progress their learning and practices more efficiently, while weaker learners can seek help from the teacher or from the system. In addition, since the system can check the correctness of student codes automatically, the learning of the whole class can also progress faster. That is, the number of exercises and topics learned in ScratchThAI classroom were higher than that of the traditional Scratch classroom.

  2. 2.

    ScratchThAI helps teachers to manage a classroom of students with different learning abilities, to monitor progresses and performance, as well as to identify learning gaps. With ScratchThAI, teachers can classify students by their learning abilities and identify required assistance accordingly. Good performance students can self-learn new concepts through the system and gradually gain more understanding through exercises. On the other hand, at-risk students and their learning and achievement gaps will be identified so that teachers can effectively plan interventions and provide specific assistance within limited school hours.

  3. 3.

    The integrated ScratChatbot keeps students engaged and motivated to practice. Based on its design specifically for young learners in the digital generation, who are familiar with technology, ScratchThAI provides interactive learning through engaging chatbot and gamification, i.e., progressive exercises and friendly competition.

  4. 4.

    As the effort in Q&A and in giving and grading assignments are offloaded to ScratchThAI, teachers can focus on course material preparation, paying attention to students’ progression and individual assistance offering.

5 Conclusion

This paper designed and proposed ScratchThAI, a conversation-based learning support framework, which adopts relevant educational approaches and related technology-enhanced learning (TEL) in its design and architecture to address the three critical CT development challenges in Thailand, namely, mixed ability students in the same class, high student-teacher ratio and limited school hours. Specifically, it incorporates the following approaches and design elements: (i) learner-centered and personalized learning approach, (ii) tracking of at-risk students, (iii) provision of assistance to teachers to support and reduce teachers workload, (iv) provision of instructional support to students, and (v) flexible learning support in time and place. The key enabling TELs employed and developed include T1: personalized and adaptive technology, T2: automatic CT assessment/grading, T3: conversational agent (chatbot), and T4: cloud-based and web technology. By adopting ScratchThAI as part of Grade 5’s computing science course in a primary school in Thailand, the results revealed that ScratchThAI could alleviate the aforementioned challenges with the following supports: (i) achieving better learning performance, (ii) enhancing teaching and learning activities, (iii) promoting student engagement and motivation, (iv) positive students’ and teachers’ satisfaction.