Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This international handbook presents cutting-edge interdisciplinary research on metacognition and learning technologies within specific tasks and learning contexts. Current psychological and educational research on learning with advanced technologies provides a wealth of empirical data, indicating that learners of all ages have difficulty learning about complex topics in areas such as science and math. Learning with advanced technologies requires students to analyze the learning situation, set meaningful learning goals, and determine which strategies to use. During learning, students need to assess whether the strategies are effective in meeting the learning goal while they evaluate their emerging understanding of the topic and continuously determine whether any particular learning ­strategy is effective for a given learning goal. In addition, they need to modify their plans, goals, strategies, and effort in relation to internal conditions (e.g., cognitive standards) and contextual conditions (e.g., scaffolding from a human tutor) while using a particular learning technology. Further, depending on the learning task, they need to reflect on their learning. Collectively, these processes involve metacognitive monitoring and ­control, and are sometimes also called self-­regulated learning (SRL).

Traditionally, researchers have used or developed their own discipline-specific frameworks, models, and theories to account for the various metacognitive and self-regulatory processes used by humans while using learning technologies to comprehend complex materials. Recently, several researchers have extended these theories and models by advancing models of metacognition and SRL that describe the influence of mediating processes related to students’ learning of these complex topics and domains. These new models have been advanced to account for the various phases (e.g., planning, metacognitive ­monitoring, strategy use, and reflection) and areas (e.g., cognitive, affect/motivation, behavior, and context) of learning. However, these emerging ­frameworks pose significant conceptual, theoretical, empirical, and educational challenges for understanding students’ learning with advanced learning technologies.

A large variety of learning technologies are becoming widespread at a very rapid pace, such as distributed online or hybrid courses, open online repositories of educational materials, hypermedia environments, games, simulations, virtual worlds, intelligent tutoring systems (ITSs), tutorial dialogue systems, electronic portfolios, and peer review systems. The list goes on and on. As a practical matter, the better we understand how learners learn with these technologies, and what challenges they encounter, the more likely it is that instructional designers and developers of technology-enhanced learning will create learning environments that benefit learners and help them learn better, instead of being just a cheaper delivery vehicle for “old” instructional methods. A particularly enticing perspective is that these learning environments will not only help learners acquire deep conceptual knowledge of complex topics, or robust cognitive skill, but will also help them become better learners across domains by allowing them to acquire, internalize, share (with other human and nonhuman agents), and practice key metacognitive and self-regulatory skills.

The study of self-regulation and metacognition in computer-based learning environments (CBLEs) is timely and important, for a number of reasons. First, it is becoming increasingly clear that the way learners monitor and regulate their learning in CBLEs is a major influence on their learning outcomes. At the same time, CBLEs can be very taxing in terms of the amount of self-regulation that they require. It is important, therefore, that these environments are designed with a good understanding of the challenges that learners face. It is good to see described in this handbook many CBLEs that are designed to scaffold aspects of SRL (e.g., metacognitive knowledge versus metacognitive skills). Even better, many systems are designed to foster important self-regulatory or metacognitive skills and we are beginning to see systems that assess and adapt to learners’ SRL and ­metacognition so as to help them become more effective learners.

CBLEs are excellent platforms to study metacognition and self-regulation for a number of methodological and practical reasons. First, they offer unprecedented opportunities for fine-grained data gathering at a large scale with very frequent “sampling” (i.e., multiple data points in a single minute) over longer periods of time. Often, systems can gather data in an unobtrusive manner, which has many practical advantages. This trend toward unobtrusive, automated data gathering and analysis of log data from systems is very compatible with the recent methodological emphasis on trace-based methodologies for studying SRL. It is also very compatible with the recent theoretical emphasis on event-based approaches and models of SRL. This is not to say that meaningful analysis of trace data or log data from CBLEs to study SRL is straightforward. There are many challenges due to the inherent uncertainty in any process that infers unobservable mental processes from behavioral data. Adding to this fundamental challenge, there is a growing trend toward using interdisciplinary research methods and analytical techniques with multichannel data (e.g., log files, eye tracking, physiological measures) to capture the complex nature of SRL and metacognitive processes. Nonetheless, interesting progress is being made, and it is good to see connections between SRL research and the burgeoning field of educational data mining.

In addition to these methodological reasons, CBLEs are also an attractive platform for studying SRL when viewed from a practical perspective. There is great natural variety in the types of self-regulatory processes that learners may employ in these environments. Therefore, they offer researchers the opportunity to observe and study these processes. As a research strategy, researchers studying SRL or metacognition can vary the design of the environments in order to study the influence of particular strategies. For example, researchers may vary the amount of learner control in an environment as a way of making certain metacognitive monitoring and control strategies more likely or less likely to occur. They may then observe the frequency of these strategies and its relation with learning outcomes. This approach to research may yield interesting insights into how system design, self-regulation, and learning outcomes are related. However, it is important to note that data on SRL and metacognitive processes must be analyzed vis-à-vis the context in which they are collected and analyzed.

The current state of research and educational applications of metacognition and learning technologies poses several challenges that are addressed in this handbook. Theoretically, we document the assumptions and complexity of various models, frameworks, and theories of metacognition and how they relate to our understanding of learning with technologies. This is a critical step in understanding how different fields conceptualize metacognition, the specificity and granularity of these models, the accuracy with which these models can be used to predict learning, and the relation between metacognition and other key learning processes (e.g., cognition, motivation, and affect). Empirically, we summarize the different types of data that researchers collect when they seek to understand the nature of metacognitive processes used during learning with advanced technologies. The foci will be on the methods used to collect, measure, and interpret data on metacognition and learning technologies. This is a critical aspect of the handbook since the inclusion of data from different disciplines will allow researchers to critically examine how various methods and analytical approaches can be used to understand the complex nature and dynamics of metacognitive knowledge and regulatory strategies used during learning with technology. Methodologically, this handbook also addresses how the use of educational technologies enables novel ways of studying metacognition, for example it makes possible a dramatic shift toward capturing, storing, analyzing, and making inferences based on highly detailed behavioral data. The availability of large stores of data also brings with it the challenge of analyzing the data; as such the handbook also contains chapters on novel data analysis techniques. Likewise, novel techniques have been developed and are described for analyzing the metacognitive data stream in a moment-by-moment fashion in order for the system to react adaptively to individual students’ metacognition. Educationally, this handbook serves as a repository of theoretically driven and empirically based examples of ­effective ways that learning technologies can be used to enhance learning for students of all ages and in various tasks and domains. These examples can be used by professionals in science and math education, classroom teachers, industry, etc. This timely volume will present innovative interdisciplinary research and stands to contribute to numerous fields and areas of research and instruction.

Brief Overview of Chapters in Each Section

The two-volume international handbook contains 46 chapters contributed by an international group of leading researchers. We organized the chapters thematically into seven different sections.

To ensure uniformity across chapters, we asked each contributing author or group of authors to address (as much as possible) the following questions found below.

  1. 1.

    Provide an overview of the context in which a particular learning technology is used to study and foster students’ metacognitive or SRL. This should include a brief description of the type of learning technology used (e.g., hypermedia, multimedia, ITS, microworld, hybrid system), the level (e.g., developmental, expert) of the target audience, and the domain or topic being addressed. Describe how the features of the learning technology have been designed to study and support metacognitive processing and SRL (e.g., adaptive help-seeking behavior, explicit scaffolding techniques, questioning techniques, etc.), and their individual and combined role in supporting students’ learning of the task/topic/domain.

  2. 2.

    Provide an overview of the metacognitive (or SRL) theoretical/conceptual framework and the underlying assumptions. This should include the model or framework assumptions, and an explanation of how the particular theory/model addresses students’ metacognitive SRL processes (e.g., which specific phases and areas are being targeted).

  3. 3.

    Describe how effective their existing learning technology is in detecting, tracing, modeling, and fostering learners’ metacognitive and self-regulatory behaviors, by summarizing their empirical findings. This should emphasize the nature of the measurement tools and analytical techniques used in the research.

  4. 4.

    Discuss the implications for the design of metacognitive tools to support metacognition and learning. Which of these components or aspects of metacognition and SRL can and should be modeled and why?

  5. 5.

    Examine the theoretical, methodological, ­analytical, and instructional challenges. For example, discuss limitations of current ­methodologies, theoretical models, analytical methods and assumptions, etc.

The first section focuses on models and components of metacognition. As such, we have five chapters that focus on a diverse set of models and components. A common theme in these chapters is the design and evaluation of specific instructional interventions that are grounded in theoretical work focused on particular models of metacognition, often including monitoring. Thus, in this work, theoretical development and practical application are closely intertwined, which has many advantages. In fact, close ties (and bidirectional influence) between theory and practical applications are found in much of the work reported in this handbook.

The chapter by Griffin, Wiley, and Salas explains an empirically grounded and detailed theoretical framework for understanding the distinction between metacognitive knowledge and metacognitive monitoring. Particular emphasis is placed on the importance of improving the relative accuracy of metacognitive monitoring skills; typical instruction in study strategies may not be sufficient to improve monitoring. The chapter by Kramarski and Michalsky describes the results of eight controlled experimentations examining different conditions for implementation of the IMPROVE self-questioning prompts in Web-based learning environments (Web-LEs) from two perspectives, first for students’ learning in the classroom, and second for preservice teachers’ learning during their professional preparation. The IMPROVE method aims to support key aspects of self-regulation targeting learning processes. By contrast the chapter by Pieschl, Stahl, and Bromme raises two important issues regarding the metacognitive self-regulation of learning with technologies. First, adaptation to the external context is a core component of SRL. Second, learner characteristics play an important role in SRL and adaptation. As such, their empirical work emphasizes epistemic beliefs as an exemplary learner characteristic and they demonstrate the importance of this learner characteristic in terms of the deployment of various cognitive, metacognitive self-regulatory processes. Rawson and Dunlosky provide an overview of the retrieval-monitoring-feedback (RMF) technique, a learning technology designed to promote both durable and efficient student learning of key concepts from course material. This is a carefully designed technique that involves core concepts from cognitive psychology and metacomprehension research. The RMF program uses the student’s monitoring judgments to schedule subsequent practice trials for each item. The technique has shown to yield relatively impressive levels of long-term retention of key concepts and it can be used to support learning for materials from many different topic domains and promises to benefit a wide range of learners. Lastly, the chapter by N. Schwartz and colleagues takes the position that learning and thinking are synergistic actions of the way people develop knowledge to adapt to the world. As such, they propose a conceptualization of metacognition as a closed-loop model of biased competition by proposing that the actions are collateral cognitive operations sharing a unitary outcome of performance, with metacognition functioning as an integral operator in the actions. They propose a model from evidence originating in neuroscience and cognitive psychology to show that metacognitive monitoring and control are reciprocal functions of the same neurologic processes that excite and inhibit, in a recursive fashion, the regions of the brain responsible for two types of activities involved in learning. These are activities involved in processing information relative to the goals of a task and other activities involved in processing the original activities deployed to seek goal attainment. They conclude their chapter by explaining how the model explains the results of research investigating the effects of metacognition on performance in CBLEs.

The assessment and modeling metacognitive knowledge and skills is the focus of the second section of the handbook, which contains five chapters. All chapters describe innovative assessment methods that can be used in conjunction with CBLEs; some of these methods are also applicable in other types of learning environments (i.e., without computers), whereas ­others depend critically on the automated ­logging that CBLEs provide. Interestingly, most work in this section is grounded in SRL or metacognitive theory. As is typical of all sections in the handbook, this section highlights a range of theoretical and methodological perspectives, as well as different types of CBLEs. Interestingly, the section also highlights the use of a range of different types of data in the study of SRL. Many projects featured in this section created automated methods for assessment, which in the future can be used to make CBLEs adapt to individual learners.

This section starts with Baker and colleagues’ chapter on why students “game the system,” a malaptive self-regulatory strategy, in which learners try to circumvent the hard work of learning, somewhat ironically by taking advantage of features of the system that aim to support learning (e.g., using hints to get answers without understanding). This work leverages machine-learned models of student gaming, termed “detectors,” which can infer student gaming from students’ interaction with educational software recorded in log files. These detectors are developed using a combination of human observation and annotation, and educational data mining. They applied the detectors to large data sets and analyzed the detectors’ predictions. They used the detectors to discover and study the factors associated with gaming behavior, which can then be remedied through adaptive scaffolding. The chapter by Greene and colleagues focuses on a pervasive issue that shows that the lack of instructional scaffolding and high degree of user control inherent to most hypermedia-learning environments (HLEs) make them difficult learning environments, especially for learners who lack the ability to ­appropriately self-regulate their learning. In order to address this issue, they introduce a two-tiered (i.e., the micro and macro level) approach to analyzing SRL data derived from think-aloud protocols. This approach turns out to be informative in terms of the domain-, task-specific self-regulatory processes that should be scaffolded in particular HLEs. They also report findings from a number of their research studies that illustrate how analyzing data at both tiers results in a comprehensive ­understanding of how learners self-regulate in HLEs, and how the nature and quality of that self-regulation interact with internal and external conditions. Opfermann and colleagues’ chapter also focuses on the benefits of hypermedia and requirements of hypermedia environments by presenting and detailing about how theories and models of SRL can serve as a framework for their research on the effectiveness of HLEs. In particular, they focus on multilevel componential and theoretical approaches, and analyses of cognitive, metacognitive, learner characteristics and cognitive load interact during learning with HLEs. The chapter by van Gog and Jarodzka discusses the use of eye tracking to assess cognitive and metacognitive processes and cognitive load in CBLEs. They discuss the benefits and limitations of eye tracking for studying such processes during learning and problem solving. In addition, they also provide examples of how eye tracking can be used to improve the design of instruction with CBLEs and discuss opportunities and challenges provided by eye-tracking technology. Finally, Veenman’s chapter ends this section by emphasizing how metacognitive skills are considered to be an organized set of metacognitive self-instructions for the monitoring of and control over cognitive activity. These self-instructions can be represented as a production system of condition-action rules. He discusses how in computerized learning tasks, online traces of learner activities can be unobtrusively stored in log files. He also emphasizes the need to capture the dynamic change in metacognitive processes over time, and how progressive patterns of metacognitive activity can be identified in logged traces through time-series analysis.

The third section focuses on scaffolding metacognition and learning with hypermedia and hypertext. The nine chapters presented in this section highlight the widespread focus placed on the use of nonlinear learning systems by several researchers, of which hypertext and hypermedia are prime examples. In these environments, ­learners typically study a complex web of related and challenging concepts. These environments lend themselves well to the study of SRL and metacognition, as learners working in these environments face a challenging self-regulation problem and exhibit a wide range of self-regulatory ­processes. At the same time, these environments are known to be challenging to learners due to the open-endedness and complexity in both the ­targeted learning materials and the learning environment itself. A common theme in this section is therefore the design and evaluation of various methods to scaffold learners working in complex, nonlinear learning environments. The nine chapters focus on a diverse set of systems and types of scaffolding. As is the case in other sections of the handbook, the work presented in this section has a strong grounding in theories of SRL and metacognition.

The first chapter, by Bannert and Mengelkamp, provides evidence and discusses appropriate scaffolding (e.g., reflection prompts, metacognitive prompts, training and metacognitive prompts) for metacognitive reflection when learning with modern CBLEs. Specifically, it focuses on prompting metacognitive and SRL skills during hypermedia learning. They end their chapter by proposing implications for the design of metacognitive support to improve hypermedia learning. The chapter by Clarebout and colleagues discusses the relationship between metacognition and the use of tools. Being able to determine when the use of a tool would be beneficial for one’s learning is seen as a metacognitive skill. Different assumptions are made with respect to this relationship between metacognitive knowledge (including instructional conceptions) and tool usage. They report on a series of studies in which different instruments were used to measure metacognitive knowledge and metacognitive skills to provide empirical underpinning for these assumptions. Dabbagh and Kitsantas’ chapter reviews research that examined whether tools and features of learning management systems (LMSs), referred to in this research as Web-based pedagogical tools (WBPT), can be used to support and promote specific processes of student SRL, such as goal setting, help seeking, and self-monitoring, in online and distributed learning contexts. Five categories of WBPT are described, including administrative tools, content creation and delivery tools, collaborative and communication tools, learning tools, and assessment tools. In addition, they present findings from several studies and demonstrate how WBPT can be used to support a number of self-regulatory processes, and that college instructors and faculty can use WBPT to design effective learning tasks that promote student SRL. Ge’s chapter presents a Web-based, database-driven cognitive support system for scaffolding self-regulation in the process of ill-structured problem solving. Of particular interest are the mechanisms of question prompts, expert view, and peer review in supporting self-monitoring, self-regulation, and self-reflection during ill-structured problem solving. She summarizes findings from several empirical studies on the effects of various support mechanisms conducted in several different knowledge domains (e.g., instructional design, education, and pharmacy). Her findings show that the cognitive support system has a positive influence on self-monitoring and self-regulation, which subsequently facilitates ill-structured problem-solving processes. The chapter by Lajoie and colleagues focuses on medical students’ metacognitive and self-regulatory behaviors during medical diagnosis using BioWorld, a technology-rich learning environment. The system offers an authentic problem-based environment where students solve clinical cases and receive expert feedback. Their team focuses on the evaluation of key system features (e.g., the evidence table and visualization maps) to determine whether they promote metacognitive monitoring and evaluation. Learning outcomes, based on novice/expert comparisons, are compared to other key measures of medical reasoning and problem solving (e.g., diagnostic accuracy, confidence, and case summaries). They present guidelines to foster key metacognitive and self-regulatory processes in medical problem-solving tasks. Narciss and colleagues’ chapter summarizes the rationale and findings of several studies conducted by her team on rich open-ended Web-LEs as learning technology in higher education. Their Web-LEs include a combination of scaffolds to support cognitive and metacognitive learning activities with university students and across various topics (e.g., introductory psychology). They close their chapter by discussing the limitations, challenges, and implications of using log-file data for investigating SRL with rich Web-LEs. The chapter by Puntambekar and colleagues emphasizes the difficulties experienced by learners when self-regulating their learning in order to make navigation decisions that align with their goals with hypertext environments. This chapter presents their extensive work in helping students learn from hypertext using the CoMPASS hypertext system in middle school science classes in physics. The system detects students’ self-regulated behavior with log files. The logs are used to analyze student navigation behavior and create clusters of navigation patterns. In turn, these patterns are used to inform an algorithm that provides adaptive real-time navigation prompts in order to scaffold metacognition and SRL. Venkatesh and colleagues’ chapter explores learner metacognition and self-­regulation in information retrieval environments equipped with a powerful indexing technology called Topic Maps. Their mixed-method studies describe academic self-regulatory processes associated with graduate learners’ understandings of ill-structured academic writing tasks and attempt to relate them to learners’ metacognitive ability to judge their own performance on iterations of these writing tasks. Their findings are critical in highlighting the novel intra-­sample statistical analyses used to uncover relationships between academic performance, metacognition, and task understanding. The last chapter in this section is by Winne and Hadwin and focuses on reviewing their model of SRL and identifying three obstacles learners face when they strive to effectively self-regulate learning autonomously. As such, they provide an overview of the nStudy software system, a Web application that offers learners a wide array of tools for identifying and operating on information they study. The system is designed to be a “laboratory” for learners and researchers alike to explore learning skills, metacognition, and SRL as researchers collect rich logs of fine-grained, time-stamped trace data that reflect the cognitive and metacognitive events in SRL.

ITSs and dialogue systems are the focus of the fourth section of the handbook. Whereas hypertext and hypermedia systems (featured in the previous section) focus primarily on helping learners study and understand a complex set of interrelated concepts, ITSs typically focus on “learning by doing” or problem-solving practice. Dialogue systems are systems that interact with learners in natural language (e.g., English), in ways strongly reminiscent of human tutors. Typically, these dialogues revolve around a task to be solved that requires strong conceptual knowledge. Learning with ITSs and dialogue systems tends to involve a different range of self-regulatory and metacognitive processes, than those reported in the chapters in the previous section, although there is substantial overlap. The type of scaffolding offered also differs. The seven chapters presented in this fourth section present a variety of intelligent systems designed to measure, foster, and support various processes related to metacognition and SRL across several school domains, such as math and science and age groups. In addition, a couple of chapters also focus on specific metacognitive and SRL processes (help seeking and self-explanations), learning processes (e.g., use of multiple representations), and system features (e.g., open learner models) that can foster the development of metacognition and SRL. As in other sections, there is great variety in the systems studied and the theoretical perspectives taken. A trend that can be ­discerned is that these types of systems tend to focus on particular metacognitive strategies within larger theoretical frameworks.

The first chapter by Aleven focuses on help-seeking behavior of students during tutored problem solving with an ITS, the Geometry Cognitive Tutor. As is typical of ITSs, this system provides step-by-step guidance with complex problems, including on-demand help (as well as step-by-step feedback). Help-seeking behavior is a key metacognitive process that can be initiated by learners and ITSs in order to foster and support problem solving. He discusses several key theories, including the ACT-R theory of cognition and learning, the Knowledge-Learning-Instruction theoretical framework focused on learning from instruction, SRL theories, and educational psychology theories of help seeking. As a first step toward theoretical integration, he reviews his work and that of his colleagues on rule-based modeling of help seeking, which integrates cognitive and metacognitive aspects within a single modeling framework. The rule-based model has been used to provide students with feedback on their help-seeking behavior. Beal’s chapter describes and provides evidence of how AnimalWatch, an ITS, provides students with instruction in algebra readiness problem solving, including basic computation, fractions, variables and expressions, basic statistics, and simple geometry. Students solve word problems that include authentic environmental science content. As they do so, they can access a range of multimedia resources that provide instructional scaffolding, such as video lessons and worked examples. The system enhances students’ motivation by providing learners with choices about what science topic they would like to learn about, and when they would like to navigate between different modules in the system. She summarizes several classroom evaluation studies, which have found positive effects on study-specific measures of problem solving. The chapter by Bull and Kay emphasizes the role of open learner models (OLMs), which allow systems to maintain a model of the learner’s understanding as he or she interacts with an e-learning environment, which allows adaptation to the learner’s educational needs. An OLM makes the machine’s representation of the learner available to him or her. Typically, the state of the learner’s knowledge (as inferred by the system based on the learner’s performance over a series of problems) is presented in some form, ranging from a simple overall mastery score to a detailed display of how much and what the learner appears to know, his or her misconceptions, and progress through a course. This means that an OLM provides a suitable interface onto the learner model for use by the learner and in some cases for others who support his or her learning, including peers, parents, and teachers. As such, their chapter considers some of the similarities between the goals of supporting and encouraging metacognition in ITSs and learning in general, and the benefits of opening the learner model to the user. Conati’s chapter describes her team’s research on providing computer-based support for the metacognitive skill of self-explanation. The distinguishing element of their work is that they aim to provide support for self-explanation that is student adaptive (i.e., tailored to the specific needs and traits of each individual). She demonstrates her approach by illustrating how they built such models for two different ITSs: one that helps college students self-explain worked-out solutions of physics problems, and one that supports self-explanation during interaction with an interactive simulation for mathematical functions. Interestingly, they were able to design a method (not unlike Baker and colleagues’ detectors) that automatically detects spontaneous, internal self-explanations, which are not expressed by the learner by means of overt, observable actions in the tutor’s user interface. The chapter by Litman and Forbes-Riley focuses on ITSpoke, a dialogue system for qualitative physics, which engages students in a spoken natural language dialogue about challenging physics concepts. Specifically, their work focused on the hypothesis that automatically responding to student uncertainty (as detected in the student’s speech) over and above correctness is one method for increasing both student learning and self-monitoring abilities. They tested this hypothesis using spoken data from both wizarded and fully automated versions of their tutorial dialogue system, where tutor responses to uncertain and/or incorrect student answers were manipulated. They present data on several metacognitive metrics that are significantly correlated with student learning. These results suggest that monitoring and responding to student uncertainty have the potential to improve students’ cognitive and metacognitive abilities. Renkl and colleagues’ chapter focuses on the use of multiple representations when using learning technologies. In fact, modern learning technologies (e.g., hypermedia systems, ITSs) usually provide information in multiple forms, such as text, “realistic” pictures, formal graphs of various kinds, or algebraic equations in order to foster learning. They argue that learners usually make ­suboptimal use of such multiple external representations. In this chapter, they present results from a series of experiments with older students (senior high school and up) that analyzed the effects of two metacognitive intervention procedures (i.e., self-explanation prompts and “instruction for use”—information on how to use multiple representations) that have shown to foster conceptual understanding and procedural skills. The last chapter in this section by Stevens and colleagues focuses on how learning trajectories have been developed for thousands of students who solved a series of online chemistry problem-solving simulations using quantitative measures of the efficiency and the effectiveness of their problem-solving approaches. Their analyses showed that the poorer problem solvers, as determined by item response theory analysis, were modifying their strategic efficiency as rapidly as the better students, but did not converge on effective outcomes. This trend was also observed at the classroom level with the more successful classes simultaneously improving both their problem-solving efficiency and effectiveness. They present evidence that placing students in collaborative groups increased both the efficiency and effectiveness of the problem-solving process, while providing pedagogical text messages increased problem-solving effectiveness, but at the expense of problem solving efficiency.

The four chapters found in the fifth section of the handbook focus on multi-agent systems to measure and foster metacognition and SRL. Animated pedagogical agents have a relatively long history in CBLEs and learning sciences research, but have only recently been applied to the modeling and scaffolding of self-regulatory and metacognitive processes. These agents are arguably a way of imbuing systems with ­personality, or multiple personalities, in an effort to make the interactions with the system take on a slightly more social nature, and make them more memorable, motivating, and engaging. Typically, the agent is visible in the interface (sometimes as a “talking head,” sometimes displayed “head to toe”) and produces speech output. Typically, the agent takes on the role of a tutor, sometimes a tutor specialized in particular aspects of learning (e.g., monitoring learners’ metacognitive judgements, assessing learners’ use of learning strategies, modeling key metacognitive and regulatory skills). Sometimes, the pedagogical agent takes on the role of a learning companion or of a student to be tutored (teachable agents). The social aspects of pedagogical agents may make them particularly well suited for supporting metacognition and self-regulation, as the social processes involved with these agents are a way of externalizing covert metacognitive and SRL skills for learners. The four chapters in this section represent contemporary cutting-edge work on the use of animated pedagogical agents embedded in hybrid intelligent systems (e.g., ITS, games, hypermedia) to detect, track, model, and foster middle school, high school, and college students’ metacognition and SRL.

The first chapter by Azevedo and colleagues emphasizes the importance of using multichannel trace data to examine the complex roles of cognitive, affective, and metacognitive (CAM) self-regulatory processes deployed by students during learning with multi-agent systems, such as MetaTutor. In MetaTutor, four different pedagogical agents are responsible for modeling, tracking, and scaffolding key metacognitive and regulatory processes and skills used by students while they learn about challenging biology topics. They argue and provide extensive evidence that tracing these processes as they unfold in real time is key to understanding how they contribute both individually and together to learning and problem solving. By treating SRL as an event, they provide empirical evidence from five different kinds of trace data, including concurrent think-alouds, eye tracking, note taking and drawing, log files, and facial recognition, to exemplify how these diverse sources of data help understand the complexity of CAM processes and their relation to learning. Kinnebrew and colleagues’ chapter on Betty’s Brain, a CBLE that helps ­students learn science by constructing causal concept map models, is based on the Learning by Teaching paradigm, where the system has ­students take on the role and responsibilities of being the teacher to a virtual student named Betty. They provide evidence of classroom studies conducted with elementary school children and discuss the generation of hidden Markov models (HMMs) that capture students’ aggregated behavior patterns, which form the basis for analyzing students’ metacognitive strategies in the system. They also provide ample evidence on the use of sophisticated computational methods to analyze SRL behaviors. These methods stand to contribute to our existing conceptions and framework of metacognition and SRL, and are related to the work presented in Section 2, on assessing and modeling metacognitive knowledge and skills. Indeed, the kinds of assessment methods discussed in Section 2 can (and increasingly, do) form the foundation for the pedagogical agents discussed in the current section, who in order to interact effectively must assess student metacognition. The chapter by Lester and colleagues presents their extensive evidence on narrative-centered learning environments (e.g., Crystal Island) that provide engaging, story-centric virtual spaces that afford opportunities for discreetly embedding pedagogical guidance for content knowledge and problem-solving skill acquisition. Students’ abilities to self-regulate learning significantly impacts performance in these environments and are critical for academic achievement and lifelong learning. Their chapter explores the relationship between narrative-centered learning environments and self-regulation for science learning. Empirical support from a series of studies with hundreds of middle school students provides evidence that narrative-centered learning environments are particularly well suited for simultaneously promoting learning, engagement, and self-regulation. The last chapter by Oppezzo and Schwartz emphasizes that producing lasting changes to metacognition, or the more encompassing construct of SRL, has strong parallels to producing behavior change. As such, they discuss and illustrate how techniques and theories of behavior change can inform the design of instruction intended to support the development and transfer of SRL. They present a four-stage model of behavior change and use it to critique their own work on Teachable Agents. They also discuss the successes of the Teachable Agents in achieving SRL goals and improving learning for each stage of the model.

Individual and collaborative learning in classroom settings is the theme of the nine chapters in Section 6 of the handbook. Again, the number of chapters in this section reflects the interest and empirical work in the area of individual and group learning with various learning technologies by research from various fields. As in other sections, the work is often firmly grounded in SRL theory, as well as other theoretical frameworks from the learning and educational sciences, reading comprehension, literacy, science education, and complex systems. In addition, the work in the current section pays careful attention to practical and theoretical issues that come up as technology-based scaffolds for SRL are embedded in classroom contexts. Interestingly, we see a variety of technologies ­represented, ranging from electronic portfolios to systems that support scientific inquiry and discovery learning to a toolkit for modeling biological processes, each with its own needs for metacognitive scaffolding. Many of these systems have been used in actual classrooms, underlining the relevance, to real educational settings and contexts, of the work featured in the current handbook. This theme runs throughout the handbook: Many chapters in other sections of the handbook also feature work carried out in real educational contexts.

The first chapter by Abrami and colleagues describes how they have developed, tested, and disseminated to schools an Electronic Portfolio Encouraging Active and Reflective Learning (ePEARL). ePEARL is designed to be faithful to predominant models of self-regulation, as it scaffolds and supports learners and their educators from grade one through grade 12 and beyond. The system encourages learners to engage in the cyclical phases and subphases of forethought, performance, and self-reflection. In a series of studies, they have explored the positive impacts of ePEARL on the enhancement of students’ SRL skills, their literacy skills, and changes in teaching while simultaneously researching classroom implementation fidelity and teacher professional development. Chiu and colleagues view metacognition and cognition as interacting ­processes that together promote coherent understanding. As such, their chapter proposes that the use of the knowledge integration pattern to design instructional scaffolding encourages the interplay between these two processes. They present and discuss several findings that indicate that instructional activities designed using the knowledge integration pattern promote student learning from dynamic visualizations by helping to overcome deceptive clarity. The chapter by Dalton and Palincsar describes the empirical and theoretical roots of the Reading to Learn program of research, which was designed to investigate the metacognition and learning of upper elementary students in supportive e-text environments. They present their findings, using various instructional manipulations (e.g., static, interactive, interactive diagram/coaching) designed to provide both procedural and conceptual support. Their chapter includes a critique on the methods used in the intervention studies and a proposal for future research. Goel and colleagues’ chapter describes the Aquarium Construction Toolkit (ACT) project which is an ongoing collaboration among learning, cognitive, computing, and biological scientists focusing on learning functional models of ecosystems in middle school science. The system is an interactive learning environment for stimulating and scaffolding construction of Structure–Behavior–Function (SBF) models to reason about classroom aquaria. The authors summarize the results from the deployment of ACT in several middle school science classrooms with several hundred middle school students. They found significant improvements in students’ ability to identify the structure, behaviors, and functions of classroom aquaria, as well as their appropriation of SBF modeling by some middle school teachers for modeling other natural systems. Lastly, they describe SRL in ACT while looking ahead and outlining the design of a ­metacognitive ACT. The chapter by Molenaar and colleagues describes a new method for the computerized scaffolding of SRL in CBLEs with avatars. The system works with an attention management system that registers the attentional focus of learners with the intention to adjust scaffolding to students’ current activities. They provide evidence that their scaffolding system enhances group performance and students’ metacognitive knowledge and that differential effects are most likely explained by a combination of quantitative and qualitative differences in the metacognitive activities triggered by problematizing scaffolds compared with structuring scaffolds. Thillmann and colleagues’ chapter presents new assessment methods for different aspects of metacognition and SRL. They argue that metacognitive knowledge about strategies and metacognitive regulation of strategies are two distinct components of metacognition that make different demands on their respective assessment method. Also, they contend that metacognitive knowledge about and metacognitive regulation of strategy use should be assessed with regard to the same strategies, in order to be able to relate both measures and to localize specific deficiencies. They exemplify their arguments using two CBLEs for scientific discovery learning by illustrating two kinds of assessment methods, including a test format that intends to assess metacognitive knowledge about scientific discovery strategies and log files to assess metacognitive regulation of the use of these strategies during SRL with the CBLEs. Their results reveal that the relationship between metacognitive knowledge and metacognitive regulation of the actual use of the same strategy is moderated by current motivation. The chapter by van Joolingen and de Jong discusses the use of models of inquiry processes, such as the Scientific Discovery and Dual Search (SDDS) model and the inquiry cycle for the generation of support on the regulation of these processes. Based on their extensive research, they argue that such scaffolding must be adaptive as too much scaffolding can actually hinder learning. Further, in order to make scaffolding adaptive, the system needs to gather information about the learners’ task progress. They discuss a few ways of using less obtrusive methods for obtaining learner information, and present an example of how such information can be used to support learners in monitoring their progress. Carneiro and Steffens’ chapter focuses on the challenges of using digital technologies since these technologies offer an almost unlimited access to information and a wide variety of tools for information processing and communication. It has also become clear that managing these resources requires a new kind of literacy, digital literacy, and that part of this digital literacy is the capacity to regulate one’s own learning. As such, their chapter examines recent theoretical approaches to SRL with digital technologies. They also expand on research and implementation policies for technology-enhanced learning in Europe and present two examples of research on SRL: Taconet, a community of European researchers that grew out of a project on this topic, and the New Opportunities Initiative (NOI), a large-scale program implemented by the Portuguese Government to empower low-skilled workers in which the use of digital technologies and SRL play a vital role. Lastly, the chapter by Dettori and Lupi describes the use of audio technology and metacognition to improve pronunciation in the learning of a second language (L2). They describe a methodological approach to guide L2 learners to observe their utterances and become aware of their ­pronunciation errors, with the support of peer collaboration and metacognitive prompts. Identifying pronunciation errors is not easy because it requires good self-observation, evaluation, and reflection skills.

The last section of the handbook is on motivation and affect as key processes in metacognition and SRL. While motivation has long been emphasized in theories and models of SRL, the ­inclusion of affect and its interrelations with cognitive, metacognitive, and motivational aspects of self-regulation is more novel. (We do not mean to say that affect has been entirely ignored by SRL researchers; only that it has been less emphasized than cognitive, metacognitive, and motivational realms.) The six chapters presented in this last section of the handbook represent some of the best research in the areas of motivation and emotions that has already had an impact in several fields from ­educational psychology to affective computing. These chapters represent a growing awareness that motivation and emotions are a key and integral part of understanding cognitive and metacognitive self-regulation. Recent advances in affective computing and CBLEs make this work particularly timely. This work is yielding automated methods for detecting learners’ ­affective state as they interact with learning technologies. One theme in the chapters reviewed here is on how these detectors can lead to useful scaffolds for self-regulation in CBLEs, making these environments more adaptive to individual learners and ultimately more effective. As before, we can point to interesting cross-connections with other sections of the handbook, for instance, the work on automated assessment of metacognitive and self-regulatory skills featured in Section 2. We as editors feel it is imperative that we ­continue to conduct interdisciplinary research on the role of cognitive, metacognitive, motivational, and affective processes prior to, ­during, and following learning and problem ­solving with CBLEs. The work featured in this handbook ­section points the way.

The first chapter is by Bernacki and colleagues on overcoming the weaknesses of using self-report questionnaires to measure motivation by proposing to capture motivational states during learning and problem solving. They hypothesize and illustrate that motivation can change during an activity or curricular unit. Therefore, without temporally fine-grained assessment (i.e., without frequent sampling), dynamic relations between motivation, cognitive, and metacognitive processes cannot be observed and studied. They describe a method for collecting fine-grained assessments of motivational variables during learning mathematics with ITS and examine their association with cognitive and metacognitive behaviors. The utility of their method for assessing motivation and use of these assessments to test hypotheses of SRL and motivation are discussed. Burleson’s chapter emphasizes the importance of understanding the affective state of a learner in determining when and how best to provide appropriate support. He describes an Affective Learning Companion built upon an Affective Agent Research Platform with the goal of discovering when, at various points in the problem-solving process, a student encounters optimal flow experiences or nonoptimal Stuck experiences. Using theories from metacognition and motivation, the goal is to help students become aware of their emotional states, and to develop metacognitive strategies to use this awareness to persevere in the face of frustration. The findings focus on gender differences in meta-affective skills, experiences of several affective states, goal orientations, and intrinsic-motivation. The chapter by Carr and colleagues describes the team’s extensive research with the Ecolab software, an interactive learning environment for 10–11-year-old learners designed to help children learn about food chains and food webs. Their chapter discusses the results of their recent work on achievement goal orientation and help seeking within the Ecolab environment. They situate the results within the broader landscape of previous studies and discuss the evolutionary approach they have adopted to design metacognitive learning tools. This methodology has been built up over a series of empirical studies with the Ecolab software that have demonstrated that children who achieved above-average learning gains use a high level of system help. Focusing specifically on the relationships between young learners’ metacognition (e.g., help-seeking behavior) and their achievement goal orientations, they extend their research on metacognitive software scaffolding and the influence of goal orientation on children’s learning. D’Mello and colleagues’ chapter argues that complex learning of difficult subject matter with educational technologies involves a coordination of cognitive, metacognitive, and affective processes. Their chapter describes several key theories of affect, meta-affect, and affect regulation during learning followed by a summary of their empirical research that focuses on identifying the affective states that spontaneously emerge during learning with educational technologies, how affect relates to learning outcomes, and how affect can be regulated. They provide extensive evidence across a large number of studies using a variety of educational technologies, different learning contexts, a number of student populations, and diverse ­methodologies to track affect. Lastly, they describe and evaluate an affect-sensitive version of AutoTutor, a fully automated ITS that detects and helps learners regulate their negative affective states (frustration, boredom, confusion) in order to increase engagement, task persistence, and learning gains. They conclude by discussing future directions of research on affect, meta-affect, and affect regulation during learning with educational technologies. The chapter by Moos and Stewart emphasizes the need to extend research on SRL and hypermedia to extend beyond the use of cognitively based theoretical models of SRL. As such, they argue that future contributions to this theory to the field of hypermedia learning need additional empirical research that systematically considers theoretically grounded constructs of motivation within SRL. The premise of their chapter is that motivation offers a potential explanation of individual differences in how students respond to negative feedback loops during hypermedia learning. They also highlight methodological and theoretical challenges, including the identification of specific motivation constructs (e.g., outcome expectations, incentives, efficacy expectations, attributions, and utility) that align with existing SRL theoretical frameworks. The last chapter, by Vollmeyer and Rheinberg, focuses on their paradigm using microworlds with their biology-lab task. They introduce their cognitive-motivational process model which specifies variables that help to describe SRL. For example, initial motivation (probability of success, interest, anxiety, and challenge) affects performance through mediating variables, such as strategies and motivation during learning, while metacognition, especially planning, could be included as a further mediating variable. They present their findings and discuss which aspects of metacognition could be integrated into the model without risking an overlap with the construct of motivation.

We are deeply impressed with the conceptual, theoretical, empirical, and educational work presented here, including its relevance to educational practice, and the promise it holds for future developments both in research and practice. The seven sections found in this handbook represent the most impressive cutting-edge work conducted by colleagues around the world. The work is innovative and inspirational: Not only do we see areas that traditionally have dominated research on metacognition and learning technologies, such as the extensive work on hypertext and hypermedia environments and ITSs and dialogue systems (see Sections 3 and 4), but we also see the huge promise from other emerging areas represented by the chapters on multi-agent systems, and motivation and emotions found in sections five and seven of the handbook. As seen in the first two sections of the handbook, the conceptual and theoretical work on models, components, assessment, and modeling of metacognition and SRL remains strong. This line of work is very much needed, because the emergence of novel learning technologies continues to challenge our ability to understand how they can potentially impact learners. Lastly, the section on individual and collaborative learning in classroom settings ­represents a burgeoning area of research across various disciplines using a plethora of theoretical frameworks and models. It deals successfully with the individual and collaborative nature of metacognition and SRL in authentic classroom contexts.

Future Directions

As editors, we are extraordinarily pleased to have captured a collection of the most impressive interdisciplinary work in the area of metacognition and learning technologies. Despite the efforts represented in this handbook, there is still more work to be done. We conclude this introductory chapter by highlighting a few specific issues that, we believe, necessitate further work in the areas of conceptual, theoretical, empirical, methodological, analytical, and educational issues.

First, there is a great need for theoretical clarity, including better definitions and descriptions of the components of metacognition and SRL. The challenge lies in the widespread proliferation of terms, constructs, mechanisms, and processes that are found in the literature. In addition, more theoretical work needs to be conducted so that current theoretical frameworks, models, and conceptualizations of metacognition and SRL can deal with important issues such as level of ­granularity, comprehensiveness, descriptiveness, dynamic processes and feedback loops, and the role of context. For example, some models are too abstract or provide high-level descriptions of a few key metacognitive processes without specifying how the recursive nature of dynamic metacognitive and SRL processes may impact how a learner self-regulates and ultimately learns with the learning technology. Such a specification should include a learners’ cognitive architecture, learning technology, and other contextual factors.

Second, more research is needed to examine the complex interactions between cognitive, metacognitive, motivational, and emotional processes. The complex interactions amongst these key processes are critical in determining their role, influence, and impact on one’s ability to monitor and regulate during learning with CBLEs. These issues are associated with a third issue—one of learning and instruction—namely, the fact that most models of learning and instruction provide very abstract, macro-level descriptions of learning, which make it difficult for researchers and designers to build systems that adequately scaffold and foster metacognition and SRL. For example, imagine a learner using a hypermedia system to develop a deep conceptual understanding of a complex physical system. During learning, the learner indicates that he or she is not interested in the topic, does not value the need to learn about it, and has demonstrated low self-efficacy in using effective learning strategies. In addition, he or she demonstrates an abundance of prolonged negative affective states during learning associated with confusion and frustration, rarely showing any enjoyment during learning. While the motivational and affective processes clearly indicate a lack of engagement in the task, he or she also has low prior knowledge of the domain, cannot seem to set relevant goals for the tasks, and repeatedly demonstrates that he or she is not capable of assessing his or her emerging understanding of the most appropriate content to use. These learner characteristics (whether transient or more stable) are inferred in real time from data collected with various sensors, so the question becomes—“When and how does the system intervene and offer scaffolding and feedback?” We do not know the answer to this question yet, because we lack theories and models of instruction that provide instructional prescriptions to handle the complex nature of cognitive, metacognitive, motivational, and ­affective processes during learning. The scenario provided raises the following questions: When does the system intervene, how does the system intervene, who or what should intervene (e.g., a peer, a teacher, a pedagogical agent, etc.), should the system intervene (at all), and what should the system offer (e.g., feedback, prompting, modeling, scaffolding)? If we are to design effective systems, general principles and guidelines need to be developed that help instructional designers (and systems) address these challenging questions.

Another key area that needs further attention is the measurement of metacognitive and self-regulatory processes. As seen in several chapters and sections throughout the handbook, researchers are making strides in the measurement of key cognitive, metacognitive, motivational, and affective processes. Measurement of a wide range of these processes is crucial as we strive to understand the nature of these processes prior to, during, and following learning with CBLEs. We are beginning to see the emergence of multi-method, multichannel approaches to capture the complex nature, deployment, and use of these processes during learning. Analytical techniques from educational mining and machine learning are currently being used and can contribute in many important ways. For example, patterns emerging from thousands of data points can be used to challenge current conceptions of metacognition and SRL. Further, they can provide descriptive accounts of adaptive and maladaptive SRL behavior, which are interesting from a theoretical ­perspective, but can also be used by the system to foster metacognition and learning (e.g., by recognizing maladaptive behaviors in real time and providing an adequate response). In addition, we need to expand our methodologies by using ­longitudinal studies to capture and understand the qualitative and ­quantitative changes in the acquisition, internalization, and use of metacognitive and regulatory processes over extended periods of time. We also need to continue to build and use our learning technologies as both research and learning tools. Naturally, the adoption of new methods and research designs will necessitate new analytical and statistical techniques, since current techniques are constantly challenged by the nature of the data collected in the area of metacognition and learning. For example, use of concurrent think-aloud data may be limited when data is non-normally distributed. For example, based on a learner’s verbosity, such data may yield few coded SRL processes and therefore violates assumptions of normalcy, thus limiting the use of inferential statistics. By contrast, log-file data is excellent in collecting fine-grained temporal data at the millisecond data in an ­unobtrusive manner. However, this data needs to be augmented with other data since ­making ­inferences about the presence of metacognitive processes is challenging. Researchers continually face the challenge to temporally collect and align multichannel theoretically derived data. This data needs to be captured, coded, scored, and interpreted in real time and post hoc, so we can advance the field by contributing to our theories and models, and so we can ultimately improve metacognition and self-regulation with learning technologies. These are just a few of the critical issues that need to be addressed in order to continue to make progress in our challenging interdisciplinary area of research.

In sum, we are encouraged by the advancement we (researchers working in the field of metacognition and learning technologies) have made thus far and are excited about the work that lies ahead of us!