Adults’ Self-Regulatory Behaviour Profiles in Blended Learning Environments and Their Implications for Design

Abstract

Blended forms of learning have become increasingly popular. However, it remains unclear under what circumstances blended learning environments are successful. Studies suggest that blended learning challenges learners’ self-regulation. Yet little is known about what self-regulatory behaviour learners exhibit in such environments. This limited understanding is problematic since this insight is needed for effective designs. Therefore, the aim of this study was to identify learners’ self-regulatory behaviour profiles in blended learning environments and to relate them to designs of blended learning environments. Learners’ (n = 120) self-regulatory behaviour in six ecologically valid blended learning courses was captured. Log files were analysed in a learning analytics fashion for frequency, diversity, and sequence of events. Three main user profiles were identified. The designs were described using a descriptive framework containing attributes that support self-regulation in blended learning environments. Results indicate fewer mis-regulators when more self-regulatory design features are integrated. These finding highlights the value of integrating features that support self-regulation in blended learning environments.

Appendices

Appendix 1: Questions per Attribute

Attributes	Main question	Sub question
Authenticity	Does the learning environment contain authentic real-world relevance?	Is an authentic context provided that reflect the way the knowledge will be used in real life? Are authentic activities provided? Is there access to expert performances and the modelling of processes? Are there multiple roles and perspectives provided? Is there support for collaborative construction of knowledge? Is articulation provided to enable tacit knowledge to be made explicit? Is authentic assessment of learning provided within the tasks?
Personalization	Does the learning environment contain personalization?	Is the personalization name-recognized? Is the personalization self-described? Is the personalization cognitive-based?
Learner-control	Does the learning environment allow learner control?	Is control of pacing allowed? Is control of content allowed? Is control of learning activities allowed? Is control of content sequence allowed?
Scaffolding	Does the learning environment scaffold support?	Is support tailored to the learner through continuous monitoring? Does the support fade over time? Is there a transfer of responsibilities over time?
Interaction	Does the learning environment entail interaction?	Is learner-content interaction facilitated? Is learner-instructor interaction facilitated? Is learner–learner interaction facilitated? Is learner-interface interaction facilitated? Is vicarious interaction facilitated?
Reflection cues	Does the learning environment contain reflection cues?	Does the reflection-for-action approach apply? Does the reflection-in-action approach apply? Does the reflection-on-action approach apply?
Calibration cues	Does the learning environment contain calibration cues?	Is a strategy applied to guide learners to delay metacognitive monitoring? Is a strategy applied for the provision of forms that guide students to summarize content? Are timed alerts given that guide students to summarize content? Is a strategy applied for helping learners review the ‘right’ information? Is a strategy applied for effective practice tests that provide students with records of their performance on past tests as well as items (or tasks) on those tests?

Appendix 2: Manual for Scoring Attributes

2.1 Authenticity

The use of the word authentic is open to interpretation. A sustainable amount of attempts to define this concept transparently is done (see e.g., Bennet et al. 2002; Herrington 2005; Wesiak et al. 2014). Definitions range from real-world relevance (Wesiak et al. 2014), needed in real-life situations (Sansone et al. 2011) and of important interest of the learner for later professional life (Grimmett and Neufeld 1994) to models that focus on applying conceptual knowledge or skills, such as critical thinking or problem solving (Young 1993). Based on their literature review Van Laer and Elen (2017) defined authenticity as the real-world relevance (both to the learners’ professional and personal life) of on the one hand the learning environment (e.g., Herrington et al. 2003; Petraglia 1998; Roth and Bowen 1995) and on the other hand the task (e.g., Merrill 2002; Reigeluth 1999; van Merriënboer and Kirschner 2001). Guidance question for identifying authenticity in learning environments and learning tasks are:

• 1. Authentic context Is an authentic context provided that reflect the way the knowledge will be used in real life? In designing online learning environments with authentic contexts, it is not enough to simply provide suitable examples from real-world situations to illustrate the concept or issue being taught. The context needs to be all-embracing, to provide the purpose and motivation for learning, and to provide a sustained and complex learning environment that can be explored at length (e.g., Brown et al. 1989; Reeves and Reeves 1997).

• 2. Authentic activities Are authentic activities provided? The learning environment needs to provide ill-defined activities which have real-world relevance, and which present a single complex task to be completed over a sustained period of time, rather than a series of shorter disconnected examples (e.g., Bransford et al. 1990; Lebow and Wager 1994).

• 3. Expert performance Is there access to expert performances and the modelling of processes? In order to provide expert performances, the environment needs to provide access to expert thinking and the modelling of processes, access to learners in various levels of expertise, and access to the social periphery or the observation of real-life episodes as they occur (Collins et al. 1989).

• 4. Multiple roles Are there multiple roles and perspectives provided? In order for students to be able to investigate the learning environment from more than a single perspective, it is important to enable and encourage students to explore different perspectives on the topics from various points of view, and to ‘criss cross’ the learning environment repeatedly (Collins et al. 1989).

• 5. Collaborative knowledge construction Is there support for collaborative construction of knowledge? The opportunity for users to collaborate is an important design element, particularly for students who may be learning at a distance. Consequently, tasks need to be addressed to a group rather than an individual, and appropriate means of communication need to be established. Collaboration can be encouraged through appropriate tasks and communication technology (e.g., discussion boards, chats, email, debates etc.) (e.g., Hooper 1992).

• 6. Tacit knowledge made explicit Is articulation provided to enable tacit knowledge to be made explicit? In order to produce a learning environment capable of providing opportunities for articulation, the tasks need to incorporate inherent opportunities to articulate, collaborative groups to enable articulation, and the public presentation of argument to enable defense of the position (e.g., Edelson et al. 1996).

• 7. Authentic assessment Is authentic assessment of learning within the tasks provided? In order to provide integrated and authentic assessment of student learning, the learning environment needs to provide: the opportunity for students to be effective performers with acquired knowledge, and to craft polished, performances or products in collaboration with others. It also requires the assessment to be seamlessly integrated with the activity, and to provide appropriate criteria for scoring varied products (e.g., Linn et al. 1991; Reeves and Okey 1996; Wiggins 1993).

2.2 Personalization

Personalization is often described as non-homogenous experiences related directly to the learner (Wilson et al. 2007), associated with characters and objects of inherent interest to the learner and connects with topics of high interest value (Cordova and Lepper 1996). Similar to these views on personalization, based on their literature review, Van Laer and Elen (2017) defined personalization as the modification of the learning environment to the inherent needs of each individual learner. Five major questions were raised by the current literature on the use of personalized learning environments (Devedžić 2006; Martinez 2002). These questions are:

• 1. Name-recognition Is the personalization name-recognized? This type of personalization aims at the acknowledgement of the learner as an individual. For example, the learner’s name can appear in the instruction or previous activities or accomplishments that have been collected and stored can later be presented when appropriate.

• 2. Self-described Is the personalization self-described? Self-described personalization enables learners, (using questionnaires, surveys, registration forms, and comments) to describe preferences and common attributes. For example, learners may take a pre-course quiz to identify existing skills, preferences, or past experiences. Afterwards, options and instructional experiences appear based on the learner-provided answers.

• 3. Cognition-based Is the personalization cognitive-based? Cognitive-based personalization uses information about cognitive processes, strategies, and ability to deliver content specifically targeted to specific types (defined cognitively) of learners. For example, learners may choose to use an audio option because they prefer hearing text rather than reading it. Or, a learner may prefer the presentation of content in a linear fashion, rather than an unsequenced presentation with hyperlinks.

2.3 Learner-Control

Learner-control refers to the amount of control learners have over support in BLEs. Different researchers identify different kinds of learner-control. Varying from freedom of task-selection by the learner (Artino 2009), control of learning sequences (sequence control) (Lin and Hsieh 2001), allowing decisions on which contents to receive (selection or content control), allowing decisions on how a specific content should be displayed (representation control) and control over the pace of information presentation (Scheiter and Gerjets 2007). Van Laer and Elen (2017), based on their literature review, defined learner-control as an inclusive approach based on the earlier mentioned different kinds of learner-control. Therefor learner control is a concept where learners have or have not control over the pacing, content, learning activities and content sequence. Four major questions (Williams 1993) occur when describing learner-control in learning environments:

• 1. Control over pacing Is control of pacing allowed (Sims and Hedberg 1995)? These traces suggest that the learners have control over the speed of presentation of instructional materials. Another element considered is the ability to control pacing, is the speed and time at which content is presented.

• 2. Control over content Is control of content allowed (Milheim and Martin 1991)? These traces suggest that the learner is permitted to skip over certain instructional units. This option generally refers to the selection of topics or objectives associated with a specific lesson, although it does not extend to a choice of which content items are displayed. This component of learner control does not focus on the micro level of interaction, in which the learner must make certain choices in response to questions or problems. Therefore, while the learner has control over the content selected for study, the actual presentation of that content has generally remained instructor driven. Thus, there would appear to be two levels of content control—that where the learner chooses a module of study, and that where the presentation and associated display elements are also under learner control.

• 3. Control over learning activities Is control of learning activities allowed (Laurillard 1987)? This includes options for the student to see examples, do exercises, receive information, consult a glossary, ask for more explanation, and take a quiz.

• 4. Control over content sequence Is of control of content sequence allowed? This includes provisions for the student to skip forward or backward a chosen amount or to retrace a route through the material, and options to control when to view such features as content indexes or content maps. Sequence control refers to the order in which the content is viewed, and often is defined in terms of being able to move to and fro among content items, such as those described by Gray (1988).

2.4 Scaffolding

Many different approaches to scaffolding have emerged from the design research on interactive learning environments, and a variety of design guidelines or principles have been proposed (Edelson et al. 1999; Kolodner et al. 2004). Based on their literature review Van Laer and Elen (2017) define scaffolding as changes in the task, so learners can accomplish tasks that would otherwise be out of their reach (Reiser 2004). This definition of scaffolding is reflected by three major questions (Puntambekar and Hubscher 2005):

• 1. Contingency Is support tailored to the learner through continuous monitoring? The support must be adapted to the current level of the learners’ performance and should either be at the same or a slightly higher level. A tool for contingency is diagnostic strategies. To provide this support, one must first determine the learners’ current level of competence. Many authors have acknowledged the importance of diagnosis in relation to scaffolding (e.g., Garza 2009; Lajoie 2005; Swanson and Lussier 2001).

• 2. Fading over time Does the support fade over time? Fading depends upon the learners’ level of development and competence. Support fades when the level and/or the amount decreases over time.

• 3. Transfer of responsibility Is there a transfer of responsibilities over time? Responsibility for the performance of a task is gradually transferred to the learner. Responsibility can refer both to cognitive and metacognitive activities and to learners’ affect. The responsibility for learning is transferred when a student takes increasing learner control.

2.5 Interaction

The nature of interaction in various forms of learning environments has been defined in a variety of ways, based upon the participants’ level of involvement in a specific learning opportunity and the objects of interaction such as other participants or content materials. The nature of interaction is also dependent upon the contexts in which interaction occurs, in a face-to-face situation or at a distance. Van Laer and Elen (2017) describe interaction as the involvement of learners with elements in the learning environment. Five major interaction related questions are taken into account (Woo and Reeves 2007):

• 1. Learner-content interaction Is learner-content interaction facilitated (Hiemstra 1993)? The first type of interaction is interaction between the learner and the content or subject of study. They are often one-way communications with a subject expert, intended to help learners in their study of the subject.

• 2. Learner-instructor interaction Is learner-instructor interaction facilitated (Moore 1989)? The second type of interaction is learners-instructor interaction between the learner and the expert who prepared the subject material, or some other expert acting as an instructor.

• 3. Learner-learner interaction Is learner–learner interaction facilitated (Moore 1989)? The third form of interaction is the inter-learner interaction, between one learner and other learners, alone or in group settings, with or without the real-time presence of an instructor.

• 4. Learner-interface interaction Is learner-interface interaction facilitated (Hillman et al. 1994)? The fourth type of interaction is learner-interface interaction, which describes the interaction between the learner and the tools needed to perform the required task.

• 5. Vicarious interaction Is vicarious interaction facilitated (Sutton 2001)? This final type of interaction takes place when a student actively observes and processes both sides of a direct interaction between two other students or between another student and the instructor.

2.6 Reflection-Cues

Many different definitions of reflection have been proposed over time. Dewey (1958) defined reflection as “active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusion to which it tends” (p. 9). Moon (1999) describes reflection as “a form of mental processing with a purpose and/or anticipated outcome that is applied to relatively complex or unstructured ideas for which there is not an obvious solution” (p. 23). Boud et al. (2013) define reflection as “a generic term for those intellectual and affective activities in which individuals engage to explore their experiences in order to lead to a new understanding and appreciation” (p. 19). All three definitions emphasize purposeful critical analysis of knowledge and experience, in order to achieve deeper meaning and understanding. Van Laer and Elen (2017) define reflection cues as prompts that aim to activate learners’ purposeful critical analysis of knowledge and experience, in order to achieve deeper meaning and understanding. This definition occurs via three major questions (Farrall 2007; Mann et al. 2009):

• 1. Reflection-before-action Does the reflection-for-action approach apply (Farrall 2007)? This type is different from the other two types since it is proactive in nature. For example the instructor asks the learner about his or her personal expectations about an upcoming task.

• 2. Reflection-in-action Does the reflection-in-action approach apply (Farrall 2007; Schön 1987)? This type of reflection takes place while learners are performing a task. Reflective cues are given when the learner is performing a certain task. Cues are given to let him reflect upon if he needs to alter, amend, change what he is doing and being in order to adjust to changing circumstances, to get back into balance, to attend accurately, etc.? Learners must check with themselves that they are on the right track: if I am not on the right track, is there a better way? For example an instructor asks learners to review the actions they are undertaking.

• 3. Reflection-on-action Does the reflection-on-action approach apply (Farrall 2007)? Munby and Russell (1992) describe it succinctly as the “systematic and deliberate thinking back over one’s actions”. Another definition which involves thinking back on what teachers have done to discover how knowing-in-action might have contributed to unexpected action (Hatton and Smith 1995). For example an instructor asks the learner about his or her previous experiences regarding a task that is just finished.

2.7 Calibration Cues

Calibration is defined as the learners’ perceptions of performance compared to the actual performance and perceived use of study tactics and actual use of study tactics (Bol and Garner 2011). Calibration concerns on the one hand the deviation of a learner’s judgment from fact, introducing notions of bias and accuracy and on the other hand metric issues regarding the validity of cues’ contributions to judgments and the grain size of cues (Azevedo and Hadwin 2005). Van Laer and Elen (2017) define calibration cues as triggers for learners to test their perceptions of performance against their actual performance and their perceived use of study tactics against their actual use of study tactics. While identifying calibration cues we focus on five major questions (Nietfeld et al. 2006; Thiede and Dunlosky 1994):

• 1. Cues for delayed metacognitive monitoring Is a strategy applied to guide learners to delay metacognitive monitoring? (Thiede and Dunlosky 1994)This strategy is based on a phenomenon labelled ‘the delayed judgement of learning effect’ that shows improved judgments after a learning delay similar to improved performance associated with distributed sessions over time. For example, learners might be first asked to highlight a text and at a later time evaluate the highlighted content in terms of how well it is understood, how easily is can be retrieved, and how it relates to the learning objective. They are asked to evaluate previously made judgements.

• 2. Forms for summarizing Is a strategy applied for the provision of forms that guide students to summarize content? Summarizing information improved calibration accuracy. It is suggests that the summaries were more effective when forms and guidelines were provided (Wood et al. 1995). For example an instructor gives the learners the task to summarize a specific content component and to review it using a correction key.

• 3. Timed alerts Are timed alerts given that guide students to summarize content? Thiede et al. (2003) state that summarizing information after a delay improved calibration accuracy.

• 4. Review of the ‘right’ information Is a strategy applied for helping learners review the “right” information? (Bol and Garner 2011) Learners have a tendency to select “almost learned” or more interesting content for restudy. If students were to rate test items on judgement of learning and interest they could be provided with feedback indicating that selection of content for restudy based on interest and minimal challenge may not be the best choices. For example an instructor advises the learners to select exercises that are challenging for them.

• 5. Effective practice tests Is a strategy applied for effective practice tests that provide students with records of their performance on past tests as well as items (or tasks) on those tests? (Bol and Garner 2011) Learners should be aware of the change in behaviour they should make. By informing them of the mistakes they already made they might direct further attempts. For example an instructor gives the results of the previous test as a guideline for the completion of the next test.

Appendix 3: Overview of Blended Learning Environments Described

See Fig. 6.

Fig. 6

Overview of the six Blended Learning Environments and the total mean score describe via the framework of the 7 attributes that support self-regulation

Appendix 4: Variables Traced per School

School A		School B
Content	1. Course module viewed (p < .05) 2. Course searched 3. Course viewed (p < .05) 4. List of modules viewed 5. User logged in in course	Content	1. Course module viewed (p < .05) 2. Course viewed (p < .05) 3. Feedback viewed 4. List of modules viewed 5. SCORM started (p < .05) 6. User logged in in course
Content related information	6. Content posted (p < .05) 7. Discussion made (p < .05) 8. Discussion viewed (p < .05) 9. Enrolled on discussion (p < .05) 10. Message made (p < .05) 11. Message modified (p < .05) 12. Note created 13. Note removed 14. Post made 15. Subscription made on discussion 16. Subscription removed	Content related information	7. Discussion created (p < .05) 8. Discussion viewed (p < .05) 9. Note created 10. Note removed 11. Post made 12. Subscription made on discussion 13. Subscription removed 14. User profile viewed
Tasks and assignments	17. Assignment made (p < .05) 18. Assignment saved (p < .05) 19. Assignment sent (p < .05) 20. File uploaded (p < .05) 21. Submissions made 22. Test attempt viewed (p < .05) 23. Test made (p < .05) 24. Test started (p < .05) 25. Test viewed 26. There is an uploaded file 27. User preserved submission	Tasks and assignments	15. Assignment made (p < .05) 16. Assignment saved (p < .05) 17. Assignment sent (p < .05) 18. File uploaded (p < .05) 19. Test viewed 20. There is an uploaded file 21. User preserved submission
Scores and results	28. Score overview viewed 29. Status of assignment viewed (p < .05) 30. Submission form consulted (p < .05) 31. Summary test attempts viewed (p < .05) 32. Test attempt reviewed (p < .05) 33. Test checked 34. User score (p < .05)	Scores and results	22. Score report viewed (p < .05) 23. Status assignment viewed (p < .05) 24. Submission form viewed (p < .05) 25. Test checked 26. User score (p < .05)