1 Introduction

Self-Regulated Learning (SRL) skills are seen as increasingly relevant and essential for the 21st Century [42, 46] as they are significantly correlated with measures of academic performance [55]. SRL is the meta-cognitive process where a student uses self-assessment, goal setting, and the selecting and deploying of strategies to acquire academic skills [55]. By supporting these skills students may be able to learn more effectively, even outside of the tutoring session. However, many schools lack the resources to provide a one-on-one tutor support for SRL skills, and it is difficult to achieve long-term changes in SRL skills with Intelligent Tutoring Systems (ITS) [26].

Our aim is to support the long-term adoption of SRL skills so that they are adopted as a general behaviour; long-term behaviour change is also one of the aims in Socially Assistive Robotics (SAR) [33] research. Human–Robot Interaction (HRI) researchers believe that social robotic tutors can be used to assist in developing skills that can transfer to long-term behaviour change [51] or generalise to other scenarios and contexts [5]. Robots are increasingly being used to provide motivating, engaging and personalised support to learners [33]. Robotic tutors have been able to increase learning gain by providing personalised hints [30] or personalised problem selection [13]. Most research in the area has investigated the use of robotic tutors to increase students’ learning gains and engagement. While short-term studies (e.g. a single session) are relatively common [29], long-term explorations (e.g. over weeks or months) of the effects of robotic tutors are still limited in number and scope.

To date robots have not aimed to supported SRL; the benefits of a personalised robotic tutor may motivate and engage students to utilise SRL process in the learning activity. We adopt an OLM as the basis for the personalisation as this is a simple and intuitive way of displaying to the learners their developing skills [7]. An OLM allows us to ensure that the learner has all relevant information on which to base their reflections and SRL processes upon. An OLM can support SRL by promoting reflection to raise awareness of understanding or developing skills, which can help planning and decision-making [8]. The robotic tutor’s scaffolding of SRL skills is also enhanced by a memory of previous interactions with the learners, which enabled the tutor to give a summary of developing skills to further support reflection and enhance the social interaction.

Our primary research question is: does long-term adaptive SRL scaffolding make a lasting improvement on learners’ SRL skills and learning gain? We aim to contribute to the field of long-term robot interaction, specifically in the area of memory and adaptation where our robotic tutor remembers aspects of the past interactions with users, which has been identified as an unexplored area [29]. We describe a study where a robotic tutor provides different levels of personalised SRL scaffolding to primary school children. The autonomous robotic tutor’s behaviour builds upon information provided to a student in an OLM. We compare personalised adaptive SRL support to personalised domain support and investigate longer term effects of such feedback. This enables us to investigate how effective the SRL scaffolding is compared to motivating, engaging and personalised domain support alone.

This study demonstrates how a robotic tutor can support the transfer of SRL skills by using memory and a summary of developing skills over a longer-term interaction. We can see that the effects are not necessarily short lived or due to a novelty effect. We also show that the social robotic tutor is effective in scaffolding SRL in a real world school environment. The results show that, when a robotic tutor personalises and adaptively scaffolds SRL behaviour using an OLM, greater indication of SRL behaviour can be observed over the control condition, where the robotic tutor only provides domain support and no SRL scaffolding. We also see some evidence of transfer of SRL skills within the activity to a different task, and also changes to the self-report of the learners’ attitudes to self-regulation.

2 Related Work

2.1 Socially Assistive Robotics and Educational Robots

SAR is a field in which robots aim to provide motivational, engaging, social, personalised, and long-term support to people [9, 11,12,13, 33, 51]. The field of SAR lies at the intersection of the fields of Assistive Robotics and Socially Interactive Robotics [12]. In Assistive Robotics a robot’s main function is to support people through physical contact, often for purposes of physical rehabilitation [12]. In Socially Interactive Robotics, a robot’s main function is to have social interaction with people; the goal is to create a engaging interaction rather than offer physical support [12]. The goal with SAR is for a robot to support people through non-physical social interaction [12], as such this brings together a broad range of research to ensure that this non-physical support is relevant to the needs of the target population [51].

Increasingly throughout the world there are projects that explore the use of socially assistive robots as tutors or educational agents in educational applications (examples include EMOTE,Footnote 1 EASEL,Footnote 2 L2TORFootnote 3 and CoWriterFootnote 4 in Europe; the “Socially Assistive Robotics: An NSF Expedition in Computing”Footnote 5 [44] in the U.S.A.; and the CRR ProjectFootnote 6 [50] in Japan). Motivations include providing learners with engaging learning experiences; providing learners with motivational, personalised, and long-term support; and providing teachers with new teaching tools. This means that there is an increasing amount of research that investigates how robots can be of benefit in an educational context. When comparing the presence of a robotic tutor to a virtual agent or on-screen feedback, there is a preference for the robotic embodiment with reference to social presence [25], enjoyment [54], trust [21], performance [17], and learning gain [31]. A key benefit of a robotic tutor is that it can motivate students to engage in the learning activity [30].

It is believed that the formation of a socio-emotional relationship between the learner and robot is paramount to facilitating a good learning experience [22]. The social capabilities of the robot play a large part in the interaction and can be quite complex and counterintuitive. For example, it has been shown that a robotic tutor that is too social may negatively impact learning [23]. An alternative approach is to reverse the social dynamic and teaching roles so that the learner is the one to teach a robot peer [13, 49]; this can lead to a strong social bond [49] and motivate the learner to learn more words [50]. There are a number of studies that investigate long-term interaction with robots in education and show high levels of social acceptance and engagement [29, 49], which suggests that in the longer-term these robots will be able to support learning gain.

There is increasing interest in how HRI can personalise or adapt to the learner. A robotic peer that requests help on problems estimated to optimise information gain can lead to greater learning gains [13]. A robotic tutor that personalises hints based on a student’s puzzle solving skills can lead to a more successful interaction with reduced problem solving time and a more motivated learner [30]. Prompts personalised to a specific level of detail based on the ability and performance of the learner can be more effective and less frustrating  [15]. Robot behaviours adapted to a learner’s engagement can increase recall levels [48]. There is increasing interest and amount of proposed research in exploring how personalisation can make HRI more effective by adapting difficulty levels [38], responding to affective states [22, 39], learning styles [9], and help-seeking behaviours [37]. Yet, there is no work looking at how HRI can impact on SRL or meta-cognition in an educational context.

2.2 Scaffolding SRL and OLM

It is important to encourage or scaffold SRL processes as students may not always be meta-cognitively or motivationally active during the learning process [2]. As a consequence students will not use or develop SRL skills. If students lack SRL skills they will struggle to learn in the future, particularly if the learning task requires independent learning, is open-ended, or not well defined.

2.2.1 Personalised SRL Support with ITS and HRI

Teachers are important in support of reflective and meta-cognitive processes [40]. Scaffolding is support or feedback that is given in a timely manner to help a learner achieve a goal that they may not have without that support [10, 27]. SRL scaffolding from a teacher can take the form of hints, feedback, and motivation [2]. When teachers scaffold SRL with a personalised or adaptive approach it can lead to a learner adopting better SRL skills as compared with conditions where fixed or no scaffolding is offered [3].

Scaffolding of SRL can be part of the feedback provided by an intelligent tutoring system (ITS). To achieve this the system has models of the domain/ subject, the learner, and a pedagogical model; these components allow personalisation to the learner [8, 47]. There is increased interest in moving from ITS that supports domain learning to ITS that supports meta-cognition and SRL [14, 26, 42]. ITS that support meta-cognition can increase meta-cognition and learning outcomes [26]. Research indicates that real-time monitoring and adaptive or personalised scaffolding of help seeking behaviour within an ITS can improve student’s help seeking behaviour in the system [37, 41].

2.2.2 Moving from External Regulation to SRL

Much of the SRL support is external in nature i.e. where the learner does not initiate the SRL behaviour. With such external regulation the learner may not internalise the SRL skills. For a learner to gain more autonomy in using SRL skills a tutor or ITS tries to reduce the scaffolding or support for those skills [42]. Roll suggests that an OLM is one method to prompt reflection to support co-regulation between the system and a user  [42]. An OLM frequently takes the form of a series of skill meters [6, 32, 34]. Previous studies suggest that an OLM can help students better allocate efforts [6] and improve problem selection [35]. Self-assessment is the ability for a learner to accurately assess their knowledge and skills in relation to their actual performance; this ability is very important in learning [52]. OLM, when used to support reflective self-assessment activities, can have a positive effect on self-assessment and on learning outcomes [24, 32, 36].

Table 1 Procedure timetable
Full size table

3 Methodology

Our aim is to investigate how personalised SRL scaffolding via OLM with a social robotic tutor impacts learning gain, SRL processes, and transfer of long-term SRL skills. To achieve this we conducted a longer-term study with 2 conditions. In the control condition the robotic tutor provides personalised domain support. In the SRL Scaffolding condition the robotic tutor provides SRL scaffolding, in addition to the domain support, based on the learner’s skill levels, task performance, and rules for appropriate SRL behaviour.

24 primary school participants individually interacted with the robot in 4 sessions—1 session each week, over a period of 1 month (detailed in Table 1). The robot is fully autonomous and begins by providing the introduction and summary, offers support to the learner throughout the learning task, and provides a wrap-up at the end of each session.

We are motivated to employ a robotic embodiment, as the physical presence, behaviour, and social interaction of socially assistive robots can engage and motivate users to develop desirable skills and abilities. Robots can be used to tackle the problem of offering children enough one-to-one support [44]. We can also use robots to offer engaging long-term support or guidance [29]. To be effective in long-term interactions the robot should be able to display an awareness of, and respond to, the user’s affective state and also adapt to the individual’s preferences in order to build and maintain a good social interaction [29]. Another way to build a bond between the robot and learner is to use memory to recall past activities [28]. The robotic tutor is able to utilise the learner model as a memory of the learners’ development in the current and previous interactions. This enables the robotic tutor to provide summaries of the learners’ development through the current session, in the form of a wrap-up at the end of the session and the learners’ development over all previous sessions in the form of a summary at the beginning of a session. We believe that this wrap-up and summary is able to support the learners’ reflection. This enables us to see if the skills developed by a learner can transfer to long-term behaviour change [51] or generalise to other scenarios and contexts [5].

3.1 Participants

There were 24 (10 female, 14 male) participants of mixed ability. The learners were aged between 10 and 12 and attended the same primary school in the U.K.

3.2 Experimental Setup

The robotic tutor was an Aldebaran Robotics NAO torso and was fully autonomous during the activity. The activity runs on a 27 inch touch screen table. The learners were standing to enable them to comfortably reach all areas of the touch screen. The robot was positioned on a stand opposite the touch screen in order for it to be at a similar height to the learner. The setup is shown in Fig. 1.

Fig. 1
figure 1

Longer-term study scenario

Full size image

3.3 The Task

The autonomous robotic tutor supported individual learners in a geography task, namely map reading. The task enables the learner to exhibit SRL skills and processes, i.e. self-monitoring, goal setting, and help seeking. The learner is in total control of the task at all times, which is in line with the methodology of scaffolding and moving from external regulation to self-regulation where we allow the learner to practice and develop their SRL skills. The activity was designed to test compass reading, map symbol knowledge, and distance measuring competencies.

The task gives basic feedback when an answer is given; the area of the task that displays the objectives flashes green if the answer given is correct or red if the answer given is incorrect. The learner was provided with three tools to assist them with the activity. They had the option to open a map key, use a distance tool, display a compass on screen, and to view previous clues in a scrap book.

There were 4 sessions in this longer-term study. In session 1 there were 6 activities: cardinal directions (easy compass); inter-cardinal directions (harder compass); distance in meters; distance in km (harder distance as it requires conversion from m to km); symbol; and all skills (compass, distance, and symbol combined). In session 2 we also added a map trail activity that combines all skills. In session 3 we added a wind farm activity that requires the learner to use distance and symbol skills to solve the problem of where to place a wind farm. For completeness, there was no new activity in session 4. This procedure is detailed in Table 1. Tasks were introduced gradually so that the learner was not overwhelmed by choice in the beginning and could become familiar with the activity.

3.4 Learner Model

We build a learner model as the basis for the robotic tutor’s domain support in addition to the SRL scaffolding behaviour and OLM skill meters used in the SRL condition. The model of the learner’s map reading competencies is created using constraint based modelling. This is an approach whereby competency values are calculated by checking the learner’s actions against a set of relevant constraints [35]. Distance and direction are evaluated based on the learner identifying a point on a map that is at a particular distance and/ or direction from a starting point. Symbol knowledge is tested by selecting a particular symbol from a choice on a map. It is possible for the learner to provide a partially correct answer by meeting the distance constraint but breaking the direction and symbol constraint; this is reflected in the model with distance competency increasing and the direction and symbol competency decreasing. To ensure that the competency values are current, we use a weighted average so that recent evidence is given a higher weighting than older evidence in determining the overall level of the competency.

3.5 Appartus

The robot acted in the role of a robotic tutor and social agent. In both conditions the robot provides domain tutoring, introduces the learning task, tools, and performs idle motions throughout the session. The touch screen table provides a collaborative context where a social interaction may take place, this approach is used with social robots in the “Sand-tray” system [4]. The touch screen table allows us to avoid technical limitations with speech recognition and natural language processing to achieve autonomous interaction; the robot is able to detect the learners’ actions in the learning scenario, and is able to respond to the learner actions, i.e. positive beeps for a correct answer, sympathetic beeps for an incorrect answer, and directing its gaze to the area that the learner is interacting with.

There are a number of events that can trigger the robot to execute a behaviour, these are: Answer attempt, when the learner answers a step in the activity; Timeout, when there has been no robot or learner activity in the preceding 15 seconds; and Tool selection, when the learner selects a tool to use. When one of these events occurs the system evaluates if the robot should execute a behaviour according the condition (Table 2). To avoid repetition or the robot talking too much we have alternative phrases for the robot behaviours and an utterance is not executed if that utterance has been delivered by the robot recently.

3.5.1 Domain Tutoring

In both conditions the robotic tutor offers domain tutoring. The aim of domain tutoring is to provide support to the learner throughout the task, to keep them motivated, and keep them progressing through the learning task. The pedagogical support has been developed based on UCD studies and a review of literature detailed in our previous publication [19]. We present a brief summary of the robot’s behaviours in this context. When the learner answers correctly the robot will provide positive feedback in the form of a positive beep.

When the learner answers incorrectly the robot will in the first case repeat the instructions or keywords for the objective. If the learner continues to make mistakes the robot will provide more detailed domain support. The help provided is based on the learner model and the performance that step. More detailed help is given if the learner model value is low and there have been multiple incorrect attempts. More motivational support will be provided if the learner model is high and there have been few errors, such as the robot saying “almost” or “You are really close”. The robot will then attempt to “pump” [14] the learner to encourage them to reflect and expose knowledge by highlighting the aspect of the step that is incorrect, e.g. “So how far is that distance?”. The robot will also give more detailed hints of how to solve the problem if required.

3.5.2 SRL Scaffolding

In the SRL scaffolding condition the robot offers SRL scaffolding in addition to the domain support. The aim of scaffolding SRL skills is to enable a student to develop their skills by reflecting on their current abilities, to identify strengths and weaknesses so that they can effectively plan their learning through selecting appropriate strategies, goals, activities, and using the tools and resources available. We provide adaptive scaffolding whereby the learner is prompted to use SRL skills at appropriate points in the activity based upon the learner’s state [3, 26]. To support this we have created an ideal model of SRL for our learning context. Such an approach has been used in another meta-cognitive tutoring systems that focus only on when a student should ask for help [1, 37].

Table 2 Robot behaviour and triggers in each condition
Full size table
Table 3 Robot behaviour and triggers in each condition continued
Full size table

We observed in previous studies that teachers scaffold SRL skills by drawing attention to the learner’s developing competencies, then encouraging reflection on why the competencies are changing, and using this as a basis to suggest appropriate tools, goals, and strategies for the learner [19], e.g. teachers would encourage the students to work on the more simple activities until they are proficient before encouraging the student to work up to more advanced activities. We use this approach as the basis for the robotic tutor’s behaviours. The robot uses the OLM to prompt the learner to reflect on their developing skills, and also as a basis to suggest other SRL skills, e.g. the robot will prompt the learner to reflect on their skills and mastery of the current activity, to prompt the learner to develop appropriate task strategies, and to work on an activity of an appropriate difficulty level for learning. We hope that the SRL scaffolding from the robotic tutor and OLM will improve learners’ ability to self-assess their performance. This meta-cognitive skill also allows the student to assess the difficulty of the problem that they are working on, and to decide to continue with the activity or move on [36]. We detail the scaffolding procedures in Table 2.

OLM: In the SRL scaffolding condition the OLM shows skill meters for each competency and is visible at all times in the top left of the screen. Changes to the skill meters are made visible with animation and there are indicators to show the previous values [32]. The learner can inspect a history of the most recent 10 pieces of evidence for each individual competency by clicking on the corresponding skill meter. For example, if the learner expands the skill meter for distance then they will see evidence broken into north, east, south, west, e.g. they may see that they have met the north and south constraints correctly but not the west and the east constraints. This enables the learner to see exactly in which aspect of the competency their strengths and weaknesses lie. The OLM should enable the student to plan their learning by helping them identify knowledge gaps, based on this they can then fill their knowledge/ skill gaps by selecting an appropriate activity or tool.

Scaffolding Appropriate SRL Behaviour: Based on out previous studies we have identified a number of appropriate SRL behaviours in the learning task [19]. The robot’s SRL scaffolding behaviours to promote good SRL behaviour are detailed in Robot behaviour and triggers in each condition continued Table 3.

Students with higher levels of SRL skills should work on problems of the appropriate difficulty. If the problem is too easy they should move on, if the problem is too hard they should either seek to address the issue or switch to an easier activity to master the skills required, this will show an awareness of appropriate goal setting for either process or outcome goals. A high SRL skill level student may also take more time over each question as they will work on questions that are at the edge of their ability. The high SRL students will not necessarily have more questions correct on average, or even in total, as they should be pushing their zone of proximal development. The ability to reflect and self-assess knowledge and skills is linked with the number of attempts to get a question correct, learners with high levels of SRL skills should realise quickly that they have an issue, and promptly correct the problem, so a high level of SRL skills will have a low average number of attempts at each problem.

High levels of SRL should also lead to appropriate tool use. High SRL students will not necessarily have high tool use, as they may not need a tool at all; however, when a learner with high levels of SRL uses a tool it should be appropriately targeted. Students using SRL skills will be aware of the OLM increasing and decreasing and how incorrect questions will affect this. Consequently, students should be aware when there is a gap in skill or knowledge and work to address the problem quickly.

Summary and Wrap-up As this study is longer-term we also have the possibility to provide a wrap-up of the completed session and a summary of previous sessions when introducing the later sessions in the study. In the SRL scaffolding condition the robot uses these points to discuss the learners’ developing skills in terms of their competencies, tool use, performance, and mastery of the activities. It is hoped that this will prompt the learners to reflect more on their developing skills and performance.

3.6 Conditions and Hypothesis

We have devised two conditions to explore our overall hypothesis. In all cases the robotic tutor is present and provides domain tutoring and gives an introduction to the task and the tools. The different robot behaviours and the events that trigger them are summarised in Tables 2 and 3.

Control Condition In this control condition the robot provides domain support as described in the above section (Sect. 3.5.1). The learner does not receive SRL support or have access to the OLM to try and limit the amount of meta-cognitive support. However, there is some support for motivation as part of the domain tutoring, the learner is also informed if the answer that they have provided is correct or incorrect.

SRL Condition In this condition the autonomous robotic tutor personalises and adapts its SRL scaffolding based on the learner’s skill levels, task performance, and rules for appropriate SRL behaviour for the current state of the learner (i.e. the ideal model). This is considered an adaptive or dynamic SRL scaffold, as it provides feedback on meta-cognitive errors, such as using an inappropriate tool or continuing with an activity that is too easy or too challenging [26]. There is additional SRL or reflective support, given at the beginning and at the end of each session, where the robot gives feedback on the learners skill levels, tool use, and mastery of activities.

We hypothesise that a personalised and adapted scaffolding of SRL processes via OLM will lead to higher learning gain and improving SRL processes over a robot that only supports domain skills. There may not be a huge increase in learning gain in the adaptive SRL condition over domain tutoring as the learners will still have personalised domain support. However, we do hypothesise that more SRL behaviours will be evident and this should lead to better allocation of effort and more effective learning. Our specific hypotheses are as follows:

Hypothesis 1 (H1) Adaptive SRL and domain scaffolding will lead to a greater increase in learning gain than domain scaffolding alone.

Hypothesis 2 (H2) Adaptive SRL and domain scaffolding will lead to more appropriate SRL learning behaviour than domain scaffolding alone.

Hypothesis 3 (H3) Adaptive SRL and domain scaffolding will lead to an increase in self report of SRL motivation than domain scaffolding alone.

We would expect to see these effects in less able students to a greater degree than in more able students that might already have strong domain knowledge and good SRL skills, which would be similar to findings in OLM research [34].

3.7 Procedure

The study was conducted in the participants’ primary school. Prior to the study a presentation was given to all students providing an overview of the study. The students were asked to complete a pre-activity domain test and SRL self-report questionnaire. The participants individually interacted with the robot in 4 sessions—1 session each week over a period of 1 month. In each session the autonomous robotic tutor introduced the learning task and explained how the task and tools work. Each student then carried out the task which was limited to 20 min in each session. At the beginning and end of each session the students were asked to rank their distance, direction, and symbol skills. After all 4 sessions each student was then asked to complete a post-activity domain test and a questionnaire with questions regarding their SRL skills. This procedure is detailed in Table 1.

4 Data Analysis

In this section, we describe the metrics we use to evaluate the effectiveness of the robot employing domain and SRL scaffolding. The students were randomly split across conditions. We have additionally broken down the analysis to investigate differences between more able and less able students based on the mean of the pre-test domain score, as was done in other OLM research [34, 36]. The breakdown of ability is presented in Table 4.

Table 4 Participant details
Full size table

4.1 Adherence to Ideal Model of SRL

We are interested in how well learners adhere to the ideal model of SRL. We investigate the rates of adhering to or deviating from the ideal model of SRL, and how these change over the sessions. We hope to see an increase in adherence to the model and a reduction in deviation over the sessions as this will show us that the learner is developing SRL skills. If the learner does not deviate from the ideal model of SRL then the robotic tutor will no longer be co-regulating SRL and the learner will be fully self-regulated. This provides an insight into SRL behaviour change over time.

4.2 SRL Indicators in Task Performance Data

The indicators we have extracted from the logs aim to measure SRL behaviours in this learning scenario. Independent samples t tests were performed on the indicators.

The Number of questions answered gives an indication of how long a learner spends on each question; a learner that is reflecting more or making use of tools will complete fewer questions. Also a student that is stretching themselves and working on items that they have difficulty with should take longer to provide an answer.

Ability in Session 3 Activity We introduced a task in session 3 that was intended to test the transfer of SRL skills into a slightly different context. The task requires the learner to choose the location of a new wind farm and requires the learner to apply the distance, direction, and symbol skills in a slightly different and more advanced context, the learners that are successful here will have more SRL skills.

4.3 Self-Assessment Accuracy

Before and after each session we asked the learners to rank their skills for each competency in the activity. Students with high SRL skills are able to make more accurate self-assessments. To measure self-assessment accuracy we can use the Absolute Accuracy Index [32, 45, 53] which represents how closely a learner’s self-assessment matches their actual performance. We show the formula to calculate this in Fig. 2. “N” is the number of competencies, “c” is the learners’ self-assessment, and “p” is the actual performance in the competencies. The closer this figure is to 0 the higher the self-assessment accuracy.

Fig. 2
figure 2

Absolute accuracy index

Full size image

4.4 SRL Questionnaire Scores

We asked the learners to complete the SRQ-A [43] prior to and after the study. This is an instrument that is designed to measure SRL skills of children in an academic context [43]. The instrument is composed of four sub-scales: external regulation, introjected regulation, identified regulation, and intrinsic motivation. It is possible to combine the scores of the sub-scales to create an RAI score. To form the RAI for this instrument the formula in Fig. 3 is used to combine the sub-scale scores.

Fig. 3
figure 3

Relative autonomy index (RAI)

Full size image

4.5 Learning Gain

Learning gains were calculated using Normalised Learning Gain [16], based on the difference between the pre-activity domain test and the post-activity domain test, the calculation is presented in Fig. 4. In both the pre and post test the learners were asked 20 similar questions that cover compass reading, distance measurement, and map symbols. A t test was used to determine whether there was any statistically significant difference between the Normalised Learning Gain of the groups.

Fig. 4
figure 4

Normalised learning gain

Full size image

5 Results

The results presented here are derived from the analysis of the log data, domain pre-test and post-test, and questionnaires.

Fig. 5
figure 5

Adherence to ideal SRL model: average moves to a more difficult activity when an activity is mastered in each session

Full size image

5.1 Adherence to Ideal Model of SRL

Adherence to Ideal SRL Model: moving to a more difficult activity when an activity is mastered The chart in Fig. 5 shows the average number of times the learners in each condition follow the ideal model of SRL by moving on to a more difficult activity when the current activity is mastered in each session. We see that the SRL condition learners on average have a higher adherence to the model and this increases over the sessions. The learners in the CONTROL condition have a lower adherence to the ideal model and do not improve throughout the sessions.

Fig. 6
figure 6

Deviation from ideal SRL model: average count of continuing with an activity that is mastered in each session

Full size image

Deviation from Ideal SRL Model: Continuing with an Activity that is Mastered The chart in Fig. 6 shows the average number of times the learners in each condition deviate from the ideal model of SRL and continues to work in an activity that they have mastered in each session. A high number here means that the learner is deviating from the ideal model of SRL spending time practising an activity that they have already mastered. We see that the learners in the SRL condition deviate from the model less frequently than the learners in the CONTROL condition. This shows that the learners in the SRL condition consistently move on more quickly to a more difficult activity when they have mastered the activity they are working on. We do see an improvement in the CONTROL condition which shows that the learners are showing some awareness of their developing skills and move on eventually.

Fig. 7
figure 7

Deviation from ideal SRL model: average count of changing activity where there are issues with knowledge/ skill in each session

Full size image

Deviation from Ideal SRL Model: Changing Activity Where there are Issues with Knowledge/Skill The chart in Fig. 7 shows the average number of times the learners in each condition deviate from the ideal model of SRL by changing the activity when there are issues with knowledge/ skill in each session. This means that the learner moves to a different activity when they have a low learner model level or have a specific issue with a part of an activity. We see that in both conditions this deviation is rare and the deviations reduce over the sessions. The learners in the CONTROL condition do appear to deviate less that the learners in the SRL condition but this is also related to the CONTROL condition learners spending a greater amount of time in each activity practising when they have already mastered the activity.

Fig. 8
figure 8

Deviation from ideal SRL model: average count of moving from an incomplete activity in each session

Full size image

Deviation from Ideal SRL Model: Moving on from an Incomplete Activity The chart in Fig. 8 shows the average number of times the learners in each condition deviate from the ideal model of SRL by moving on from an incomplete activity in each session. This is a deviation from the ideal model as the learners have not learnt everything that the activity has to offer. We see the learners in both conditions deviate fewer times on average as they go through the sessions. In general the SRL learners deviate from the model less frequently.

In summary we see that in general in the SRL condition there is greater adherence to the ideal SRL model as the learners in the SRL condition are more likely to move on to more difficult activities when they have mastered the current activity, they deviate from the model less by spending less time in an already mastered activity, and they do not typically change activity until they have learnt all they can from it and resolved all issues with knowledge/ skill.

5.2 SRL Indicators in Task Performance Data

The indicators we have extracted from the logs aim to measure SRL behaviours in this learning scenario. Independent samples t tests were performed on the indicators.

Fig. 9
figure 9

Total questions completed in each session: all learners (left), less able (centre) and more able students (right)

Full size image

Number of Questions Answered In session 1, the SRL condition learners (overall, less able, and more able) answer significantly fewer questions than the CONTROL condition learners. In session 2, the SRL condition learners (overall and more able) answer significantly fewer questions than the CONTROL condition learners. In session 3, there are no significant differences due to the new wind farm task that was introduced that week. However, we see that there were more questions for the SRL condition, this may be because the nature of the new task was more supported by the SRL condition tutor. We therefore expected the number to be low and the learners effectively transfer some of their SRL skills as hoped (see below for further discussion). Again in session 4, the SRL condition learners (overall and more able) answer significantly fewer questions than the CONTROL condition learners. This indicates that in general the learners in the SRL condition are spending more time reflecting on each question.

In Fig. 9 we see that in all sessions other than session 3 there are significant differences between the conditions. In the SRL condition learners answer significantly fewer questions than the CONTROL condition.

Ability in Session 3 Activity We see in Fig. 9 that in the SRL condition learners were less negatively impacted by the change in activity. This indicates that they are able to successfully transfer some of their SRL skills into this new activity type.

5.3 Self-Assessment Accuracy

We see in Fig. 10, that in general the self-assessment accuracy of the learners in the SRL condition improves over time. The closer this figure is to 0 the higher the self-assessment accuracy. This is due to the more able students becoming more accurate with their self-assessments.

Fig. 10
figure 10

Self-assessment accuracy in each session

Full size image
Fig. 11
figure 11

Difference in identified regulation: all learners (left), less able (centre) and more able students (right)

Full size image

5.4 SRL Questionnaire Scores

We see that less able students in the SRL condition have a significant increase in identified regulation (Fig. 11). This sub-scale addresses questions concerning the types of motivations for self-regulation that we aim to foster in the students e.g. a willingness to learn and improve.

Fig. 12
figure 12

Relative autonomy index paired: all learners (left), less able (centre) and more able students (right)

Full size image

When we look at the paired t test for the RAI (Fig. 12), we see a significant increase in the RAI score between the pre and post-test for the CONTROL condition, \(\hbox {t}(11)=-2.86\); \(\hbox {p}=0.01\), and also a significant increase in the RAI score in the SRL condition, \(\hbox {t}(11)=-2.39\); \(\hbox {p}=0.02\). So we see an increase in both conditions. In the CONTROL condition the more able students also have a significant increase, \(\hbox {t}(6)=-2.78\); \(\hbox {p}=0.02\). In the SRL condition the less able students have a significant increase, \(\hbox {t}(5)=-2.75\); \(\hbox {p}=0.02\). In summary we see that the tutoring in our environment appears to improve the SRL attitudes in general.

5.5 Learning Gain

We use a t test to compare the normalised learning gain between conditions, the results are shown in Fig. 13. We see that in both conditions there is an increase between the pre-test and post-tests, however no significant differences were found.

Fig. 13
figure 13

Normalised learning gain: all learners (left), less able (centre) and more able students (right)

Full size image

6 Discussion

There is evidence to support our hypothesis that a more personalised and adapted scaffolding of SRL processes via OLM lead to improving SRL processes than domain tutoring alone.

H1 is not supported as we see an increase between the pre and post-test domain scores for the students in both the SRL condition and the CONTROL condition. The increase in both conditions is likely due to the domain support offered by the robot in both conditions.

H2 is supported as we see that the learners in the SRL condition show more SRL indicators, e.g spending more time on each question. We also see that in general in the SRL condition there is greater adherence to the ideal SRL model than the learners in the CONTROL condition. The learners in the SRL condition are more likely to move on to more difficult activities when they have mastered the current activity, they deviate from the model less by spending less time in an already mastered activity, and they do not typically change activity until they have learnt all they can from it and resolved all issues with the knowledge/ skill. We also see that over the sessions the adherence to the model increases and the deviations decrease for the SRL condition learners which shows that the robots scaffolding behaviours will not be triggered as frequently and the learner is moving away from co-regulation of SRL skills to become more self-regulated.

H3 is not supported as we see an increase in the relevant areas of the SRQ-A questionnaire scores in both conditions. We measured four types of motivation relating to academic self-regulation; intrinsic, identified, introjected, and external. The sources of the motivation for intrinsic and identified regulation are more internal, and the source of the motivation for introjected and external are more external. We see increase in the self reported motivation for the overall RAI and the identified regulation, which are the types of motivation that are most important for SRL.

We have not found large differences in results between conditions. We believe that this is due to the control condition still providing personalised adaptive support, an opportunity to practice SRL skills, and motivation to the learner throughout the longer-term study. We believe that had we also included the OLM in the control condition that the results would have been even closer. In previous work we have compared different combinations of SRL support and OLM presence, where we found that an OLM without robotic support does raise the learners awareness of difficulties [20], which is in line with other OLM research.

We may have been able to see a larger difference between conditions by comparing the SRL condition to a control condition that provided less personalised support, however we do not feel that this would be a fair comparison as the importance of personalised support has been seen in many studies as detailed in the related work section. The most similar study would be that of a social robotic tutor was able to effectively support appropriate help-seeking behaviour by adapting to the learners help-seeking behaviour compared to a control condition that offered help on demand [37]. We have also seen in our previous research [20] that non-adaptive or less personalised support is not as effective.

The activity was designed to allow the development of SRL skills and even without specific SRL support, learners appear to have been able to use the opportunity to test and develop their SRL skills. This has been reflected in the results from the SRQ-A that the robotic tutoring in our environment appears to improve SRL attitudes in general. This shows the importance of activity design for the support of developing SRL skills in a learner. However, we do not believe that the environment alone is always enough to encourage the development of SRL skills. In our previous research and SRL research in general we have seen that it can be difficult for learners to engage SRL skills so motivational support from a robotic tutor is still required to keep a learner engaged. For example in our previous research in this learning environment, if the robotic tutor does not provide motivational or domain support the learner is likely to remain in a single activity and not push themselves to learn [20].

In future work we would like to address the limitation of small sample size to see if it would be possible to see statistical significance between groups. We would also like to increase the duration of the study and investigate if the SRL skills could transfer to another domain.

7 Summary and Conclusions

We believe that in this study we have seen the benefits of personalised adaptive SRL scaffolding compared to traditional adaptive domain tutoring. Learners in the SRL condition show more signs of transfer of SRL skills within the task. In both conditions we see improved attitudes to SRL as detailed in the SRQ-A analysis. In the near-term transfer within the task we see the learners in the SRL condition were better able to cope with the introduction of the wind farm activity which asks them to use the geography and SRL skills in a slightly different and more demanding activity. We also see that the SRL tutoring appears to help the less able students increase motivation to engage in SRL practises.

We also show that the social robotic tutor is effective in scaffolding SRL and motivating students in a real world school environment. The key factors of this approach that we believe can lead to long-term behaviour change are using studies based in the real world school environment, using physical robotic embodiment, basing pedagogical and social behaviours on human teachers, and using behaviours that adapt to the learner to provide personalised support. We see that both conditions are quite close in results of our data analysis, we believe that this is due to the control condition still providing adaptive support, an opportunity to practice SRL skills, and motivation to the learner throughout the longer-term study. While the changes to the learners SRL behaviour are subtle these findings are encouraging as it means that the inclusion of SRL scaffolding does not detract from the interaction.

The increased SRL skills that we have seen in this study should help the learners to learn more effectively in the future. We believe that the approach of providing engaging, personalised support for SRL and meta-cognitive skills via a physical robotic tutor can be applied to other learning environments. These findings are important for researchers developing education technology, as supporting learners to become independent with good SRL skills is a key requirement for educators in the 21st century, as we have seen first hand in our teacher interviews [18].