Introduction

In the course of societal megatrends such as globalization, demographic change, or digitization, new impulses for schools have emerged in recent years, which require new approaches to designing learning environments and promoting sustainable learning processes (OECD, 2022). Lifelong learning is therefore increasingly becoming the focus for a more equitable and sustainable society, with the educational system playing an essential role (Bolhuis, 2003; UNESCO, 2022). Self-regulated learning (SRL) processes are essential for lifelong learning and are widely regarded as the cornerstone of education in the twenty-first century (Anthonysamy et al., 2020; Lüftenegger et al., 2012; Taranto & Buchanan, 2020; Wigfield et al., 2011). This raises the question of how schools as educational organizations should enable students` SRL to cope with the challenges of social transformative processes. Fueled by the digitalization of educational systems (Beller, 2013; Scherer et al., 2019), SRL is increasingly supported by digital media, such as learning platforms, learning management systems, chatbots, or apps (Broadbent & Poon, 2015; Broadbent et al., 2020). From an educational science perspective, the primary goal is to support the learning and educational process of learners, ideally tailored to their individual learning process, to optimally accompany and improve learning (Azevedo & Gašević, 2019), also referred to as Adaptive Learning Technology (ALT). While the quantity of research on ALT has significantly increased in recent decades (Martin et al., 2020), and its usage in pedagogical practices is becoming more widespread (Broadbent & Poon, 2015; Molenaar et al., 2021), questions still exist about the technical and pedagogical requirements of ALTs. These requirements are necessary to provide learners with personalized and timely support throughout their learning process (Martin et al., 2020). Our paper makes a unique contribution by enhancing the capabilities of adaptive learning as a technology for regulating individual learning. This is accomplished by exploring the potential of integrating an ALT with artificial intelligence, specifically natural language processing (NLP). We thereby address three identified gaps in previous research. First, there is a call for more studies like ours, as “NLP technologies have the great potential to provide and advance precision education and personalized learning” (Chen et al., 2020, p. 15). Second, we not only employ NLP in the context of ALT, but also base all NLP decisions and configurations on a sound, qualitatively informed theoretical and empirical framework that draws from research in educational studies (Zawacki-Richter et al., 2019). Finally, we tackle a growing concern that questions the perceived preference for using quantitative methods in ALT. Scholars instead suggest a more integrative combination of NLP and qualitative methods, such as interviews (Cheligeer et al., 2022; Guetterman et al., 2018), to design and implement meaningful and effective ALTs to foster SRL.

Adaptive learning technology

ALT is a collection of educational software tools and systems that utilize data analytics and machine learning algorithms to offer tailored learning experiences to individual students, taking into account their particular needs, strengths, and weaknesses (Imhof et al., 2020; Kurilovas et al., 2015; Nakic et al., 2015; Pelletier, 2022). Therefore, ALT`s aim to personalize education for better student learning by identifying their learning gaps, recommending relevant content, and assessing their progress until a learning objective is met (Kerr, 2016; Petersen et al., 2017; Xie et al., 2019). The concept of adaptive learning has a long tradition in learning research and is closely related to other concepts such as personalized learning, student-centered education, individualization, and differentiated instruction (Schmid et al., 2022). Throughout the history of education, skilled educators have been known to adapt to the unique needs of their students by making adjustments such as altering the method of presenting information, adjusting the academic level, changing the order of tasks, and providing customized guidance and feedback (Petersen et al., 2017). Due to the rapid development and use of information and communication technology in schools and classrooms (Scherer et al., 2019), the potential of ALT has increasingly expanded and provided a wide range of new opportunities in school practice and research (Martin et al., 2020).

How is adaptive learning technology connected to self-regulated learning?

SRL involves a hierarchical, adaptive process in which learners analyze a task, set goals and plans, and use strategies to achieve them with motivational and affective factors being critical to initiating and sustaining goal attainment. The learning process and goal attainment are monitored through metacognitive strategies and the effective use of self-regulatory strategies, which vary based on the task and context (Greene et al., 2021). Continuous diagnosis of students’ SRL competences is crucial, and is accomplished through teachers coaching and providing feedback related to SRL (Butler & Winne, 1995; Karlen et al., 2020; Klug et al., 2011; Nicol & Macfarlane‐Dick, 2006; Tempelaar, 2020). Gradually shifting responsibility from teachers to students is a key aspect of supporting SRL. This involves moving from externally regulated learning to co-regulated learning, and ultimately to students’ SRL (van Beek et al., 2014; van de Pol et al., 2010). ALTs can provide learners with greater flexibility in practicing SRL by assisting them in responsibly utilizing the opportunities presented in learning situations (Park et al., 2023; Winne, 2017).

Researchers traditionally used cognitive or constructivist models to explain learning with ALTs, but recently, some have advanced metacognitive and SRL models to describe the complex mediating processes involved in students’ learning with technology (Azevedo & Gašević, 2019; Winne & Azevedo, 2014). For example, Molenaar and colleagues refer to the COPES Model (Winne & Hadwin, 1998) when they state that “in adaptive learning technologies […] part of the control and monitoring loop is taken over by the technology.” (Molenaar et al., 2021, p. 2). The influence of ALT on successful SRL has been widely demonstrated (e.g., Aleven et al., 2017; Molenaar & Van Campen, 2016). Forsyth et al. (2016) were able to show that ALT can be used to identify weaknesses in students’ SRL and improve them through the use of clear goals. To do this, they used an Automated Grading Learning System that helped learners evaluate their learning and allowed instructors to provide timely and individualized feedback. Furthermore, it has been shown that ALTs can have a positive impact on students’ academic performance and motivation (Faber et al., 2017) as well as on the successful adaptation of learning behaviors (Molenaar et al., 2021). Azevedo et al. (2012) stipulated that a crucial aspect of ALTs is their ability to facilitate students’ SRL through scaffolding, fostering, and supporting cognitive, affective and metacognitive processes.

Despite the research demonstrating the impact of ALTs on SRL, there is a substantial body of interdisciplinary evidence suggesting that learners often show signs of dysregulated learning (Azevedo & Feyzi-Behnagh, 2011). In essence, this describes a situation where learners, due to maladaptive framework conditions in the classroom or delayed feedback—not utilize these processes to regulate their own learning or adaptively modify their behavior. This challenge is closely linked to what Xie and colleagues describe as “readiness” (2019, p. 13), which emphasizes the importance of considering the experiences and contexts that enable students to be motivated and capable of learning in class. For example, research has shown that although a personalized creativity learning system provides customized learning paths by utilizing richer data sources such as user input and questionnaire responses during learning, the level of user effort required is too high (Lin et al., 2013) Integrating context-aware user data collection techniques into technology-enhanced learning is critical for achieving real-time context-aware adaptive learning (Xie et al., 2019).

Natural language processing

NLP refers to “the computational examination of texts’ linguistic properties” (Crossley et al., 2016, p. 7) that can handle large amounts of text data that are being produced, among others, within ALT. Generally, NLP has been suggested to provide a systematic approach to enable computer-based scaffolding (Raković, et al., 2021). The underlying notion describes a process whereby students interact with a piece of ALT software, which provides them with formative feedback, informing their SRL, and then gradually being faded out. In their conceptual work, Graesser and McNamara (2010) refer to these types of software solutions as “pedagogical agents” (p. 3) that can encourage students to consider and implement particular SRL strategies and plans. For these pedagogical agents to work efficiently, the authors stipulate that statistical measures in computational linguistics, such as NLP, can greatly contribute to streamlining the process of identifying matches and gaps in students’ SRL, regarding predetermined and suggested self-regulatory strategies. Araka and colleagues (2022) suggested that NLP provides a valuable toolkit that can aid teachers to monitor and influence students’ learning paths within bound SRL trajectories. For example, the integration of NLP into ALTs has demonstrated the enhanced recognition and support of emotion regulation strategies and scaffolding among students (Azevedo et al., 2022). The application of NLP has proven its potential to provide more precise individualized support for learners, exemplified through the use of chatbots (Sáiz-Manzanares et al., 2023). Within this context, Gabriel and colleagues (2022) indicate that complementing more qualitative instruments and methodological approaches with NLP can greatly contribute to revealing and supporting the dynamic nature of SRL. Similarly, Crossley and colleagues (2016) employed NLP, in combination click-stream data and test results, to investigate students’ completion rates in massive open online courses. In their study, the authors used a range of different NLP measures, including sentiment analysis to effectively monitor and evaluate students’ task performance. Based on their promising results, the authors concluded that NLP can provide innovative and valuable information in the context of ALT. Even more so, they call for increased use of NLP in order to develop more rigorous models for SRL that are independent of domains and specific contexts. Raković and colleagues (2021) depart from a similar notion and argue that NLP and the analysis of linguistic features constitute valuable resources to better understand and possibly anticipate students’ SRL decisions. In another study, they used data from a large introductory biology course and examined how evaluation and adoption, informed by NLP, can shape and influence students’ SRL (Raković et al., 2022). Among others, the authors used opinion mining (Liu, 2010) and Part-of-Speech tagging (POS) (Chiche & Yitagesu, 2022) to develop a computational system that evaluated students’ metacognitive evaluations and plans, and then prompted reflective responses, depending on the input of the students. Their findings provide insights and support for the notion of NLP adding a valuable dimension to better understand and inform SRL, particularly among underachieving students.

Research questions

ALT is prominently featured as an important development in educational technology in the 2018 Horizon report (Becker et al., 2018). ALTs supported by computational techniques such as NLP are continually evolving to provide adaptive and personalized support to learners (Moreno-Guerrero et al., 2020; Mousavinasab et al., 2021). Despite the potential benefits of incorporating context-aware user data collection techniques in technology-enhanced learning, there remains a challenge in addressing pedagogical issues related to the entire learning-teaching process, which are often overlooked (Zhang & Aslan, 2021). In this context, scholars like Zawacki-Richter and colleagues (2019) argue that current ALT applications and approaches are often-times lacking a foundation in pedagogical practice. This is underlined by the scarcity of empirical studies that utilize ALT in pedagogical settings (Cavanagh et al., 2020; Imhof et al., 2020). According to Kabudi et al. (2021), despite evidence of ALT-enhanced learning interventions, challenges persist in effectively addressing learners’ abilities and issues. As a result, learners perceive ALTs as not being tailored to their individual needs, leading to critical, dismissive or even dysfunctional attitudes towards these systems. Moreover, adaptive mobile systems do not exist as they demand educational researchers to have up-to-date IT skills and require building from scratch (Xie et al., 2019).

Successful learning with ALTs depends on learners being able to adaptively regulate their cognitive and metacognitive behaviors during the learning process. As such, challenges, particularly in the advancement of SRL, can arise due to this dependence (Azevedo & Gasevic, 2019; Winne, 2017). But according to Azevedo and Feyzi-Behnagh (2011), it is often the case that learners are not capable of adaptively adjust their learning behavior. Therefore, there is a need to make ALTs even more effective and accessible just-in-time in promoting SRL (Azevedo et al., 2017). Hence, in this paper the overarching research question is as follows:

What is the potential of combining NLP to improve real-time contextual adaptive learning within an ALT to support learners` SRL?

More specifically, this question can be subdivided into the following component parts:

  1. 1.

    How can an ALT be designed to effectively consider learners’ needs and preferences regarding their SRL?

  2. 2.

    How can NLP be utilized in an ALT to effectively gather relevant information from learners to provide adaptive support in SRL?

Method

Context of the study

The goal of this project is to collaborate with students and teachers in the development, testing, and adaptation of a tailored digital tool, aiming to offer adaptive and personalized support to both groups engaging in SRL. In order to achieve this objective, we initiated a one-year co-design process (Penuel et al., 2007, 2022) in collaboration with a high school, as depicted in Fig. 1. The primary objective of this phase was to develop the tool by actively considering the needs and requirements of the learners. This involved closely collaborating with the target groups. At the core of the process was a co-design team consisting of students, teachers, and researchers. Over the course of one year, this group convened in two workshops to collaboratively assess and analyze the gathered data, drawing meaningful conclusions for the project. Initially, emphasis was placed on assessing the tool’s usability, including its functionality, design, and learner interaction. The co-design process was divided into three main phases (McKercher, 2020):

  1. 1.

    Establishing preconditions, immersion and coordination

  2. 2.

    Exploring, designing, and

  3. 3.

    Testing, refinement, implementation and learning.

Fig. 1
figure 1

Co-design process overview

Full size image

As depicted in Fig. 1, the project started in October 2022 with the initiation of version 0 of the digital tool known as “studybuddy”. This iteration evolved from a pre-existing prototype that had been earlier developed at the university. Studybuddy serves as an ALT, aimed at fostering SRL by providing learners with tailored and timely SRL strategies (Mejeh et al., in press).

In the first phase various types of data were collected. Quantitatively, data on students’ trait behaviors related to their SRL were collected using a pre-post design questionnaire. Additionally, learner trajectory data were gathered over a 5-week period to examine learners’ state behavior in relation to their SRL. Qualitatively, individual interviews were conducted to perform profile analyses of individual SRL behaviors. Additionally, group interviews were conducted with teachers and students to evaluate the usability of the digital tool and determine the need for SRL support at a broader level. At the co-design group level, a questionnaire on interests and goals was administered in preparation for the first workshop in the second phase of the project.

During the second phase, the research team examined both quantitative and qualitative data obtained from pre- and post-surveys, daily surveys, group discussions, and interviews. Workshop 1 was conducted with the co-design group to analyze the outcomes of the tool’s implementation in classrooms, establish goals, gather ideas, and conduct a SWOT analysis. Following the workshop, the research team assessed the outcomes and developed prototypes for version 0.1 of the digital tool. The co-design group provided feedback on the prototypes, and once the selection of prototypes and elements was finalized, the technical implementation of version 0.1 commenced.

The third phase consists of the testing and refinement of studybuddy version 0.1. Based on the findings from the initial two phases, we made the decision to deviate from our original approach and make adjustments to our data collection methods. Rather than conducting SRL pre-surveys, daily short questionnaires, and interviews and group discussions once again, we opted to involve more students as advisors in the co-design process. For this purpose, we actively sought out students who were interested in working in small groups to test specific enhancements of the digital tool, such as evaluating dashboards. To engage these student consultants, the research team initiated contact and assigned them small tasks (e.g., “What do you think of the new color scheme of the dashboard?”). This approach allowed us to gather valuable feedback from the students and involve them directly in the refinement of the digital tool. The feedback received from the advisors was carefully evaluated and formed the foundation for the second workshop with the co-design group. During this workshop, the feedback was thoroughly discussed and assessed, leading to the development of several prototypes for the digital tool. The co-design group collaboratively decided on a specific prototype that served as the basis for version 1.0 of the tool. Once the prototype was chosen, the development of version 1.0 commenced.

Data base

The study was conducted in collaboration with a Swiss high school (Canton Bern). The digital tool was utilized in three classes throughout a one-year co-design process, involving a total of 69 students who used studybuddy. Out of these 69 students, 25 voluntarily agreed to participate in interviews. The qualitative data collection during the co-design process encompassed three types of interviews: group discussions, stimulated recall, and individual interviews. The individual interviews specifically concentrated on students’ SRL and their usage of studybuddy. This is the reason why these interviews were selected for analysis in this study (for the interview guide, see Appendix A). A subset of 6 students willingly took part in the individual interviews, constituting a convenience sampling approach (Robinson, 2014), which is commonly employed in co-design scenarios for the development of educational technology (e.g., Penuel et al., 2007). The six students (2 female, 3 male, 1 without gender) came from three different classes of the 11th grade. Regarding the incorporation of NLP into ALT, our current primary focus lay in understanding how pertinent information can be assimilated by students and then effectively processed by the digital tool to offer enhanced adaptive support. However, even though their perspective is often essential, the viewpoint of learners is often underrepresented in co-design processes (Garcia et al., 2018a, 2018b; Könings et al., 2011).

The students were between 16 and 17 years old. Overall, this resulted in 210 min of recorded interviews. The audio files were then transcribed using Deepgram, an automated transcription tool that uses artificial intelligence to identify conversational patterns and employs a range of speech-to-text formatting features. Departing from the established guidelines for transcribing interviews as suggested Kuckartz and Rädiker (2019), we decided to configure Deepgram with an essential setup, focusing on punctuation, while neglecting pauses and emphases. Utterances were included in the transcript and taken care of by other NLP features that will be described below. After an initial inspection of the transcripts by the researchers, the transcripts were then imported into MAXQDA to perform a thorough accuracy check to ensure the fidelity of the transcriptions. The study was approved by the Human Research Ethics committee of the first authors university (Ethics approval number: 2022-10-05).

Analysis

In order to analyze how the integration of NLP into an ALT system could look like to provide just-in-time support to students, we employed a three-tier approach, namely using opinion mining, POS, and sentiment analysis. It is important to note that we do not employ this approach in isolation. Instead, we follow the suggestion of scholars like Zawacki-Richter and colleagues (2019), who called for more quantitative approaches in ALT that are clearly informed by qualitative research, such as the indicated interviews from the co-design process, and sound theoretical considerations, as described in the preceding paragraphs of this paper.

Opinion mining (Liu, 2010) refers to the use of computational linguistics to identify and extract subjective information from textual data (Varathan et al., 2017). This information can then be used to create lexicons, e.g., based on individuals’ responses to interview questions or ALT cues, that can then be compared to reference lexicons. In the case at hand, the reference lexicons were based on the identified (meta-)cognitive, motivational, as well as emotional components that form the basis for SRL. Components were identified based on the two most widely used and reputable SRL assessment tools: the Motivated Strategies for Learning Questionnaire (MSLQ) and the Learning and Study Strategies Inventory (LASSI), as outlined by Credé and Phillips (2011) and Fong et al. (2021). For the MSLQ, the German LIST version (Wild & Schiefele, 1994) was employed, while the German WLI version (Metzger, 2017) was utilized for the LASSI, both of which are established adaptations. The potential overlap between individuals’ responses and the SRL references was determined using cosine similarity (Lahitani et al., 2016), which compares the words and phrases in the applicable lexicons with each other, yielding an index ranging from 0 (completely different) to 1 (completely similar). The resulting metric allowed us to determine the degree to which individuals used terminology associated with SRL constructs such as resource management, emotion, motivation and metacognitive strategies. This provided a baseline, indicating the overall degree with which individuals were already actively using SRL-related terminology. It is important to note that the focus of the index, for the purpose of this showcase, is do identify SRL levels within individuals. A comparison between individuals is of course also possible, but at this point not at the center of this work. Next, we zoomed in to more nuanced linguistic aspects of interviewees’ responses (Chen et al., 2020).

Subsequently, based on the overall categorization of the responses, we conducted POS to extract keywords and phrases. The main idea of POS is to assign each word of a text to its proper syntactic tag in the context of its appearance (Chiche & Yitagesu, 2022). This is also referred to as grammatical tagging (Khan et al., 2019) and includes verbs, adjectives, adverbs, and nouns. By implementing POS, we were able to deal with the ambiguity of individual words and better evaluate and contextualize the meaning of word pairs within the sentences that they occurred. For example, opinion mining might have indicated that the overlap of an individual’s response is high with respect to the SRL construct of “motivation”. While proving valuable information, we are not yet able to classify whether the individual is referring to an increase of motivation (e.g. “I became more motivated”) or a lack of motivation (e.g. “I got bored with the topic”). Adding POS to opinion mining provides exactly this type of additional insights that are highly relevant to provide more nuanced and tailor-made feedback and suggestions for that particular individual. For these first two steps, all data was analyzed using the R statistical software package, drawing on the libraries textstem, udpipe (Zeman et al., 2017) and quanteda (Benoit et al., 2018).

Finally, following up on our POS argument and the notion that affective processes can be instrumental for individuals’ SRL strategies (Azevedo et al., 2012), we also conducted sentiment analysis (Manning & Schutze, 1999). More specifically, in the examples provided above, POS can help to identify nuanced word combinations, but does not by itself reveal the difference between “motivated” and “bored”. Adding sentiment analysis to opinion mining and POS therefore adds another valuable layer of information to support the development of more adaptive support in SRL. Sentiment analysis is usually based on the ordinal classification of emotions and opinions (Rosenthal et al., 2017). We used the Linguistic Inquiry and Word Count (LIWC) software (Boyd et al., 2022), which consists of multiple dictionaries that map written text to important psychosocial constructs and theories with words, phrases, and other linguistic constructions. More specifically, given the context of this study, we employed the DE-LIWC2015 dictionaries, as they are focused on the German language and include affective processes. These processes can be subdivided into positive emotions and negative emotions. Words that describe positive emotions include “happy”, “pretty”, and “good”. Negative emotions, on the other hand, are further subdivided into anxiety (e.g., “nervous”, “afraid”), anger (e.g., “hate”, “annoyed”), and sadness (e.g., “grief”, “sad”). The resulting index determines the percentage overlap between the words from the responses and the words from the DE-LIWC2015 dictionaries (Boyd et al., 2022).

Results

Based on the context of this study (see chapter 3.1), we will now describe the conceptualization of the digital tool and the needs of the students and teachers that have emerged during the first two implementation phases. This prompted us to opt for the integration of an AI empowered with NLP during the third phase. This enhancement aims to provide even more adaptive and personalized support to students in their SRL. We then proceed to showcase the outcomes of integrating NLP into the tool.

Studybuddy—a digital tool to support learners self-regulated learning

To support learners SRL in a context-dependent and need-based manner in real-time, we developed and implemented the digital tool studybuddy. Studybuddy is a digital platform available as both a website and an app, designed to promote SRL. The tool offers learners different forms of feedback, such as new assignments or reminders to complete tasks. The main focus of studybuddy is to provide learners with various regulation strategies that help them reflect on their learning process and make adjustments to achieve their learning goals autonomously. In this process, learners receive feedback through a range of notes and strategies in the form of prompts, providing them with direct insight into their learning process. This immediate feedback is intended to contribute to the improvement of SRL, as studybuddy engages directly with learners concerning their learning processes. Utilizing a questionnaire-based approach, SRL-related data is systematically gathered from learners over time, encompassing insights into motivation, emotion, cognition, metacognition, and resource management. This approach employs a standardized 7-point Likert scale, ranging from 1 (strongly disagree) to 7 (strongly agree), to evaluate students’ self-reflection on their SRL. This questionnaire is seamlessly integrated into the digital learning environment as a brief daily survey, administered both in the morning and afternoon of a school day. If a student does not exceed a specific threshold, the app automatically prompts them with an appropriate regulation strategy. For instance, in the context of self-efficacy, learners are asked, “Do you feel optimistic that you will make good progress at studying today?”. If the student selects one of the four lowest values (“strongly disagree”, “disagree”, “somewhat disagree”, “neutral”) on the 7-point Likert scale, a regulation strategy is automatically suggested (for an overview of what the daily reflection on SRL looks like, see the video in Appendix B).

For the most part, the content and functionality of the website is the same as that of the app. The app proves to be essential in that its use allows for direct interaction with the users by sending different prompts. It is also used to display individual learning progress (dashboard function) and can be used as a planning tool, featuring a calendar and a note function.

With the help of studybuddy, learning-related data is collected from learners over time. This means, that learning is divided into different episodes (e.g., a task, a lesson, or a whole school day) that proceed cyclically, whereby—due to a feedback loop—preceding phases influence subsequent phases (Bellhäuser et al., 2022). Accordingly, as to when learners receive adaptive feedback from the digital tool, depends on which learning episodes are defined and how long they are. Figure 2 depicts studybuddy’s four components: an automated prompting system, a digital dashboard, personalized strategies, and a planning tool (for more details see Mejeh et al., in press). In this article, we focus on the automated prompting system and the personalized strategy delivery as part of a larger feedback system.

Fig. 2
figure 2

Overview different functions studybuddy

Full size image

Automated prompting system

Figure 3 illustrates how the automatic prompting system works for learners via push notifications (for an overview of how the automatic prompting system works, see the video in Appendix B). This system is essential for SRL, as it helps learners to adjust their learning strategies and monitor their progress (Nicol & Macfarlane‐Dick, 2006). The system is pre-programmed and prompts can be send to learners at a predefined time (e.g., push notification of the start of the week) or activated individually based on their needs and preferences (e.g., push notification of new learning strategies). This has been shown to be particularly effective for SRL, as it enables learners to make timely adjustments to their learning strategies and monitor their progress (Dabbagh & Kitsantas, 2012). Shute (2008) found that learners who received immediate and specific feedback through an ALT made significant gains in their performance compared to learners who received delayed or general feedback.

Fig. 3
figure 3

Getting a new prompt by studybuddy

Full size image

Moreover, the system can remind students of newly unlocked tasks, keeping them engaged and motivated (Seiler et al., 2018). Notifications are also activated based on certain behavioral patterns within the learning environment, which helps learners to reflect on their learning behaviors and improve their self-regulation skills (Bodily & Verbert, 2017). For instance, when a learner clicks on the "Task completed" button, a notice appears utilizing images and text material to draw their attention to the successful completion of the tasks (Ifenthaler et al., 2021). Additionally, notes become visible in the learning environment at predefined points in time, drawing learners’ attention to the strategy recommendations or strategy collection, which can enhance their learning outcomes (Garcia et al., 2018a, 2018b).

Personalized strategies

Learning Analytics (LA) can help to provide insights into how to personalize feedback and interventions to meet the unique needs of the learners (Baker & Inventado, 2014). As depicted in Fig. 4, this facilitates personalized LA feedback by offering individualized strategy recommendations grounded in personal motivation, emotions, cognition, metacognition, and resource management data (for an overview of how the personalized strategies system works, see the video in Appendix B).The integration of personalized strategy recommendations into a digital learning environment, such as studybuddy, can help to facilitate learners’ adoption and use of these strategies, as well as provide a convenient and accessible resource for their ongoing development (e.g., Dabbagh & Kitsantas, 2012; Järvelä & Hadwin, 2013).

Fig. 4
figure 4

Getting a new strategy suggestion by studybuddy

Full size image

The majority of the strategies used by studybuddy are based on the German version (Metzger, 2017) of the Learning and Study Strategies Inventory (LASSI) (Weinstein et al., 1988). The individual strategies are adjusted and displayed after each collection of learner-related data, both before and after solving tasks. The format takes the form of short videos that use pictographs and text to clearly illustrate the recommended actions (for an overview of how the strategies look like, see Appendix C). For example, for motivation, one strategy is to create a sense of achievement. This involves rewarding oneself to an appropriate degree for achieving intermediate goals or embedding an unpleasant task between the completion of two pleasant tasks. In the context of emotion, learners can engage in “expressive writing”, where they write down all their emotions without worrying about grammar or spelling. This activity allows them to reflect on their emotions, resulting in greater emotional regulation and control. For cognition, learners can use the “slow down” strategy, which involves going through the learning material again and taking small breaks to improve understanding and retention. In terms of metacognition, learners can manage their tasks more effectively by using the "task management" strategy, which involves diversifying their work schedule and packing unpleasant tasks between two more pleasant ones. This strategy helps learners prioritize their time, resulting in greater productivity and reduced stress. Finally, when considering resource management, learners can optimize their workspace using the "flow place" strategy, which involves identifying the best place to focus and making sure they are in that environment. By optimizing their resources, learners can better concentrate on their learning tasks. The individual strategies are adjusted and displayed after each collection of learner-related data (before and after solving the tasks, respectively).

In summary, studybuddy is an ALT characterized by an automated feedback system and personalized strategy mediation, offering enhanced adaptability through its portability. During the initial two phases of the co-design process, it became evident that studybuddy, although already highly adaptive, required further development to increase its adaptiveness. Throughout the development process, three key areas were identified: adaptivity, strategy communication, and dashboard design. Concerning adaptivity, the co-design group proposed several enhancements. These included the ability to customize the timing of push messages, integrating a to-do list feature, establishing a collaborative forum for task editing and discussion, and incorporating artificial intelligence. Regarding strategy delivery, it was apparent that students desired various display formats for SRL strategies, a favorites function, and strategies that align effectively with specific task types. Therefore, to enhance the adaptiveness of the digital tool and better cater to learners’ individual needs, efforts were made to explore the possibilities of utilizing a text recognition program (Deepgram) and an NLP AI. These components were tested using existing interview material from the co-design process.

Opinion mining, part-of-speech tagging, and sentiment analysis

Based on the indicated interviews and the methodological approach described, we now present some preliminary results. First, we employed opinion mining, in order to broadly define potential overlap between the words used by interviewees and the defined SRL strategies. Figure 5 below visualizes the findings. As can be seen, there is quite some variance between the different interviewees, indicating different levels of SRL proficiency. Moreover, it seems that the SRL constructs “Emotion” and “Motivation” are slightly more pronounced than the other ones, already providing valuable insights as to which SRL strategies might require more attention in the context at hand. Based on the initial lexicons, these findings already suggest that by using the similarity measure, an initial rough assessment of individual SRL-related needs can be conducted and is able to capture differences within and across individuals.

Fig. 5
figure 5

Overview of similarity measures across Interviews (grouped by SRL construct)

Full size image

Another visualization of the keywords, particularly the nouns and adjectives used in combination with each other, is depicted in Fig. 6. The aim of this type of POS tagging is to contextualize words within the framework in which they are used. The connections between words (Fig. 6), provides additional insights into how students might consider and use SRL in their daily lives. This, in turn, enables the provision of more fine-tuned and adaptive feedback and suggestions that take into account the individual context in which students engage with SRL.

Fig. 6
figure 6

Overview of identified keywords across all Interviews

Full size image

For instance, clusters of words become evident in the network surrounding key terms such as “video”, “difficulty”, “support”, “strategy”, “matter”, or “situation”. This can provide an initial impression of what is important to learners regarding their SRL and in which context they situate it. However, in its current form, the word network remains somewhat unclear from a purely descriptive standpoint. Therefore, the term "strategy" will be further explored in the subsequent discussion as it assumes a central role in the context of the conducted analyses.

Figure 7 zooms in on the specific part of the interviews that dealt with “strategy”.

Fig. 7
figure 7

Zooming in on POS connections around the concept of strategies

Full size image

Upon closer examination of the term “strategy” in the word network, it becomes evident that the interviewees predominantly mention it in combination with other words that suggest a promotion of SRL through a digital tool. Specifically, the link between “strategy” and terms such as “personal”, “different” “new”, as well as “learning”, clearly indicate that the expected effect of the tool can be assumed based on the participants’ discussions. The connection between the words “strategy” and “learning”, indicating that they have been used in the same sentence and in relation with each other, is particularly noteworthy. This preliminary finding suggests that the tool and the project have been identified by students to support their learning on how to further improve their SRL strategies. Moreover, being able to identify this via NLP highlights the potential of the method, being part of an ALT, to enhance the automated adaptivity of educational technology to the specific needs and developments of students’ SRL. Consequently, it is possible to draw even more nuanced conclusions about how interviewees were talking about their SRL strategies and to what extent they might already use them, or what type of hurdles they encounter.

Finally, when considering interviewees sentiment, they overwhelmingly used positive emotions and words associated with affective processes to describe their SRL strategies, as well as the feedback and the suggestions they got from the ALT (see Table 1).

Table 1 Different feelings expressed by the respondents during the interviews
Full size table

These insights provide valuable information on whether being subjected to feedback and potentially new SRL strategies affect interviewees emotional disposition. In the case at hand, there does not seem to be cause for alarm. However, if the results would have been more pronounced, e.g., in terms of anxiety, the ALT could identify and highlight this, allowing it to respond with additional feedback and other forms of support to better help the interviewee in the situation.

Discussion

In this paper, our objective was to explore the potential of NLP as part of an ALT system to enhance real-time contextual adaptive learning and improve learners’ SRL. Through the considerations around NLP, we aimed to examine the system’s ability to interpret learner input and provide more effective responses.

Practical implications

Departing from our overarching research question we first investigated how an ALT can incorporate the needs and preferences of learners. In this context, we employed a co-design process, which enabled us to identify areas where studybuddy could be more responsive to the individual needs of learners. This collaborative approach allowed us to address the challenges of integrating educational technology into pedagogical practices while ensuring that the tool is customized to meet the specific needs of the school community and all learners involved. In doing so, we have endeavored to address the demand for educational technology to be more closely aligned with pedagogical practice (Kabudi et al., 2021; Zawacki-Richter et al., 2019; Zhang & Aslan, 2021). Regarding this, the possibility to analyze large amounts of data has demonstrated that promoting SRL through ALT systems offers the potential to counteract maladaptive SRL (Azevedo & Feyzi-Behnagh, 2011; Azevedo et al., 2017). Based on feedback from the co-design team, it became evident that there is a specific need to lower the threshold of use for users in the third phase of studybuddy development. The co-design group’s feedback on studybuddy aligns with previous research indicating that poor usability leads to increased cognitive stress and more challenging learning experiences (Mirata et al., 2020). Consequently, we tested Deepgram, as an automatic transcription tool, to further lower the perceived hurdle in actively using studdybuddy as an ALT to support SRL. While we observed some inaccuracies in the automatic transcription, the integration of a speech recognition system holds promising possibilities to provide more immediate, just-in-time feedback based on students’ verbal input.

Theoretical and methodological implications

Following our second research question we showcased a proof of concept on the potential of NLP to enhance ALT. We are thereby able to support the work of previous research on this topic, suggesting that the integration of NLP in ALT can support students’ SRL strategies and scaffolding (Azevedo et al., 2022), as well as reveal and support the dynamic nature of SRL. We are also able to support previous studies that discovered that POS tagging enables a more detailed analysis of the grammatical structure of learners’ contributions, leading to a better understanding of their learning process and providing more precise feedback based on language patterns (Nicoll et al., 2022). Additionally, employing opinion mining and sentiment analysis allows for the capture of emotional and motivational aspects of learners, facilitating tailored feedback on their SRL. Furthermore, we contribute to existing research by underlining the potential of NLP to potentially contribute to design a “warning system” that automatically flags when a student might be experiencing a difficult time (e.g., having stress or experiencing anxiety) (Macfadyen & Dawson, 2010). This opens avenues for gaining insights into learners’ attitudes, engagement, and satisfaction, enabling feedback to be delivered in a timely and relevant manner (Nandakumar et al., 2022). To this end, we follow the work of other scholars that highlighted that NLP promises to be most beneficial, if it is complementary to qualitative methods, such as interviews (Cheligeer et al., 2022; Guetterman et al., 2018). Moreover, we contribute to a growing body of research that analyzes SRL processes through NLP in ALT without solely relying on self-reports (Winne & Perry, 2000) or traditional learning analytics data such as trace data (Schumacher & Ifenthaler, 2018).

One of the main challenges that emerged in our study was the standardized questioning regarding different aspects of SRL, which significantly limited adaptivity for learners. However, integrating NLP provides evidence that the individual dimensions of SRL can be addressed in a more targeted manner. For instance, through POS tagging, it becomes feasible not only to explicitly address time management as a concept and provide appropriate feedback but also to comprehend words in their contextual meaning (e.g., my allocation of time instead of time management). Thus, the digital tool, in its current form, tackles a theory–practice issue by enabling the translation of everyday language into scientific terminology in a user-friendly manner (Garg et al., 2022). Ultimately, we demonstrated the feasibility of developing adaptive and portable systems that can be utilized in pedagogical practice and further refined in collaboration with pedagogical practitioners.

Future steps

Since we are still in the development phase of the digital tool, the question arises as to the next steps in our research project in order to address the identified limitations.

Firstly, our study is based on a relatively small sample size. Once the digital tool has undergone the initial iteration process, it is important to test its effectiveness on a larger and more diverse sample. This will help in gaining a broader understanding of the tool’s impact and its applicability across different contexts. Additionally, it raises the question of how teachers will access and effectively utilize the information generated by studybuddy in the classroom. Secondly, there is a need to expand the dictionaries used for NLP analyses (Berger & Packard, 2022; Fan et al., 2019). This expansion will enhance the accuracy and coverage of the NLP techniques employed, allowing for more comprehensive and nuanced analysis of learner input. Thirdly, the use of the speech recognition program requires further refinement. It is necessary to examine whether improved results can be achieved in the future by processing shorter text inputs without the need for manual verification. In our model of feedback processing (Fig. 8), the next steps in development are summarized as questions, which guide the subsequent stages of the research project. Moreover, Fig. 8 illustrates the integration of NLP into studybuddy.

  1. 1.

    How are the prompts sent?

    • Studybuddy App

    • Studybuddy Website

  2. 2.

    How do learners communicate their current needs?

    • Voice Message

    • Written

  3. 3.

    How is the information processed?

    • Dictation function

    • Automatic transcription

  4. 4.

    How is the information analyzed?

    • NLP synonyms

    • POS tagging

  5. 5.

    What happens with the analysis?

    • Adaptive, relevant SRL strategies for all learners

    • Feedback on learning behavior

    • Adjustments in the dashboard

  6. 6.

    What happens with it?

    • Hints are prompted to learners

    • SRL strategies are prompted to learners

  7. 7.

    What do learners do with it?

    • Adjust their SRL strategies

    • Provide feedback to the tool

  8. 8.

    Continuously collaborating with stakeholders to further develop the tool.

Fig. 8
figure 8

Feedback loop of studybuddy 1.0

Full size image

In summary, the integration of NLP techniques into existing ALTs offers valuable data for in-depth analysis, enabling researchers and educators to gain deeper insights into learners’ behaviors, preferences, and learning patterns. However, it is important to continue making further technical advancements, considering pedagogical implications, and gathering scientific evidence to ensure that these data are meaningful and beneficial for learners.