Skip to main content
SpringerLink
Log in

Using multiple, contextualized data sources to measure learners’ perceptions of their self-regulated learning

  • Published:
Metacognition and Learning Aims and scope Submit manuscript

Abstract

As theory and research in self-regulated learning (SRL) advance, debate continues about how to measure SRL as strategic, fine-grained, dynamic adaptations learners make during and between study sessions. Recognizing learners’ perceptions are critical to the strategic adaptations they make during studying, this research examined the unique contributions of self-report data for understanding regulation as it develops over time. Data included (a) scores on the Regulation of Learning Questionnaire (RLQ) completed in the first and last few weeks of a 13-week course and (b) diary-like Weekly Reflections completed over 11 weeks. Participants were 263 undergraduate students in a course about SRL. First, exploratory factor analysis resulted in a five-factor model of the RLQ with factors labeled Task Understanding, Goal Setting, Monitoring, Evaluating, and Adapting. Second, latent class analysis of Time 1 and 2 RLQ scores revealed four classes: emergent regulators, moderate regulators, high regulators with emergent adapting, and high regulators. Finally, in-depth qualitative analysis of Weekly Reflections resulted in group SRL profiles based on a sub-sample of participants from each RLQ class. Qualitatively, these groups were labeled: unengaged regulators, active regulators, struggling regulators, and emergent regulators. Quantitative and qualitative SRL profiles were juxtaposed and similarities and differences discussed. This paper explicates and discusses the critical importance of sampling self-reports of SRL over time and tasks particularly in contexts where regulation is developing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Additional CFA and invariance models were run including only the items whose factor loadings did not change by more than .049 from Time 1 to Time 2 (based on original strong FI models). Thus, in these models items from TU (B4, B8, B9), Goal setting (B12, B15), Monitoring (A4, A5), and Evaluating (A10, A11, A12, A13, A17, A18) were not included. CFA of the RLQ as a whole had weak model fit and strong factorial invariance was considered acceptable for the modified subscales (FI models for Monitoring were unidentified with only two remaining items). Factorial invariance models had drastic improvement in fit relative to the models that included our final set of items. See discussion for further consideration of these items.

References

  • Alexander, P. A., Dinsmore, D. A., Parkinson, M. M., & Winters, F. I. (2011). Self-regulated learning in academic domains. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 393–407). New York: Routledge.

  • Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educational Psychologist, 40(4), 199–209. doi:10.1207/s15326985ep4004_2.

    Article  Google Scholar 

  • Azevedo, R., & Hadwin, A. F. (2005). Introduction to special issue: scaffolding self-regulated learning and metacognition: implications for the design of computer-based scaffolds. Instructional Science, 331, 367–379.

    Article  Google Scholar 

  • Azevedo, R., Harley, J., Trevors, G., Duffy, M., Feyzi-Behnagh, R., Bouchet, F., & Landis, R. (2013). Using trace data to examine the complex roles of cognitive, metacognitive, and emotional self-regulatory processes during learning with multi-agent systems. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 427–449). Springer: New York.

    Chapter  Google Scholar 

  • Azevedo, R., Moos, D., Johnson, A. M., & Chaucey, A. D. (2010). Measuring cognitive and metacognitive regulatory processes during hypermedia learning: Issues and challenges. Educational Psychologist, 45, 210–223. doi:10.1080/00461520.2010.515934.

  • Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417–444.

    Article  Google Scholar 

  • Bliss, L. B., & Mueller, R. J. (1993). An instrument for the assessment of study behaviors of college students. Reading Research and Instruction, 32, 46–52.

    Article  Google Scholar 

  • Boekaerts, M. (1995). Self-regulated learning: Bridging the gap between metacognitive and metamotivation theories. Education, 30(4), 195–200.

  • Boekaerts, M. (1996). Self-regulated learning at the junction of cognition and motivation. European Psychologist, 1(2), 100–112.

    Article  Google Scholar 

  • Boekaerts, M. (2006). Self-regulation and effort investment. In E. Sigel & K. A. Renninger (Eds.), Handbook of child psychology (Child psychology in practice, Vol. 4, pp. 345–377). Hoboken: Wiley.

    Google Scholar 

  • Boekaerts, M., & Corno, L. (2005). Self-regulation in the classroom: a perspective on assessment and intervention. Applied Psychology: An International Review, 54(2), 199–231.

    Article  Google Scholar 

  • Boekaerts, M., & Niemivirta, M. (2000). Self-regulated learning: finding a balance between learning goals and ego-protective goals. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of Self-Regulation (pp. 417–451). San Diego: Academic.

    Chapter  Google Scholar 

  • Butler, D. L. (2002). Qualitative approaches to investigating self-regulated learning: contributions and challenges. Educational Psychologist, 37(1), 59–63. doi:10.1207/00461520252828564.

    Article  Google Scholar 

  • Byrne, B. M. (2001). Structural equation modeling with Amos: Basic concepts, applications, and programming. Mahwah, NJ: Erlbaum.

  • Cleary, T. (2011). Emergence of self-regulated learning microanalysis. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 329–345). New York: Routledge.

    Google Scholar 

  • Cleary, T. J., & Chen, P. P. (2009). Self-regulation, motivation, and math achievement in middle school: variations across grade level and math context. Journal of School Psychology, 47(5), 291–314. doi:10.1016/j.jsp.2009.04.002

  • Dunn, K. M., Jordan, K. P., & Croft, P. R. (2010). Recall of medication use, self-care activities and pain intensity: a comparison of daily diaries and self-report questionnaires among low back pain patients. Primary Health Care Research and Development, 11, 93–102.

    Article  Google Scholar 

  • Fisher, R. J. (1993). Social desirability bias and the validity of indirect questioning. Journal of Consumer Research, 20, 303–315.

    Article  Google Scholar 

  • Greene, J. A., & Azevedo, R. (2007). A theoretical review of Winne and Hadwin’s model of self-regulated learning: new perspectives and directions. Review of Educational Research, 77(3), 334–372. doi:10.3102/003465430303953.

    Article  Google Scholar 

  • Greene, J. A., Robertson, J., & Costa, L.-J. C. (2011). Assessing self-regulated learning using think-aloud methods. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 313–328). New York: Routledge.

    Google Scholar 

  • Hadwin, A. F. (2009). Strategic Learning Questionnaire. Unpublished instrument.

  • Hadwin, A. F., Winne, P. H., Stockley, D. B., Nesbit, J. C., & Woszczyna, C. (2001). Context moderates students’ self-reports about how they study. Journal of Educational Psychology, 93(3), 477–487.

    Article  Google Scholar 

  • Hadwin, A. F., Boutara, L., Knoetzke, T., & Thompson, S. (2004). Cross-case study of self-regulated learning as a series of events. Educational Research and Evaluation, 10, 365–418.

    Article  Google Scholar 

  • Hadwin, A. F., Nesbit, J. C., Jamieson-Noel, D., Code, J., & Winne, P. H. (2007). Examining trace data to explore self-regulated learning. Metacognition and Learning, 2(2–3), 107–124. doi:10.1007/s11409-007-9016-7.

    Article  Google Scholar 

  • Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York: Guilford.

  • Koning, I. M., Harakeh, Z., Engels, R. C. M. E., & Vollebergh, W. A. M. (2010). A comparison of self-reported alcohol use measures by early adolescents: Questionnaires versus diary. Journal of Substance Use, 15, 166–173.

    Article  Google Scholar 

  • McCardle, L., Webster, E. A., Hadwin, A. F. (2012). Supporting students in setting effective goals for self-regulated learning: Does a tool for weekly self-monitoring help? Paper presented at the International Conference on Motivation, Frankfurt, Germany.

  • Meredith, W. (1993). Measurement invariance, factor analysis, and factorial invariance. Psychometrika, 58, 525–543.

  • Muthén, L. K., & Muthén, B. O. (2010). Mplus (version 6) [computer software]. Los Angeles: Muthén & Muthén.

    Google Scholar 

  • Nelson, T. O. (1996). Consciousness and metacognition. American Psychologist, 51, 102–116.

    Article  Google Scholar 

  • Nelson, T. O., & Dunlosky, J. (1991). When people’s judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: the “delayed-JOL effect”. Psychological Science, 4, 267–270.

    Article  Google Scholar 

  • Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.

    Google Scholar 

  • Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: a Monte Carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 14(4), 535–569. doi:10.1080/10705510701575396.

    Article  Google Scholar 

  • Patrick, H., & Middleton, M. J. (2002). Turning the kaleidoscope: what we see when self-regulated learning is viewed with a qualitative lens. Educational Psychologist, 37(1), 27–39.

    Article  Google Scholar 

  • Perry, N. E. (2002). Introduction: using qualitative methods to enrich understandings of self-regulated learning. Educational Psychologist, 37(1), 1–3. doi:10.1207/00461520252828500.

    Article  Google Scholar 

  • Pintrich, P. (2000). An achievement goal theory perspective on issues in motivation: Terminology, theory, and research. Contemporary Educational Psychology, 25, 92–104. doi:10.1006/ceps.1999.1017.

  • Pintrich, P. R. (2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16, 385–407. doi:10.1007/s10648-004-0006-x.

    Article  Google Scholar 

  • Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33–40.

  • Pintrich, P. R., Smith, D. A., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the motivated strategies for learning questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801–813.

    Article  Google Scholar 

  • Pintrich, P. R., Wolters, C. A., & Baxter, G. P. (2000). Assessing metacognition and self-regulated learning. In G. Schraw & J. C. Impara (Eds.), Issues in the measurement of metacognition (pp. 43–97). Lincoln: The University of Nebraska Press.

    Google Scholar 

  • Puustinen, M., & Pulkkinen, L. (2001). Models of self-regulated learning: a review. Scandinavian Journal of Educational Research, 45, 270–286.

    Article  Google Scholar 

  • Rotgans, J. I.,  & Schmidt, H. G. (2010). The Motivated Strategies for Learning Questionnaire: A measure for students’ general motivational beliefs and learning strategies? The Asia-Pacific Education Researcher, 19, 357–369.

  • Schmitz, B., Klug, J., & Schmidt, M. (2011). Assessing self-regulated learning using diary measures with university students. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 251–266). New York: Routledge.

    Google Scholar 

  • Schunk, D. H. (2003). Self-efficacy for reading and writing: Influence of modeling, goal setting, and self-evaluation. Reading & Writing Quarterly, 19, 159–172.

  • Veenman, M. V. J., & Spaans, M. A. (2005). Relation between intellectual and metacognitive skills: Age and task differences. Learning and Individual Differences, 15, 159–176.

  • Weinstein, C. E., Schulte, A., & Palmer, D. R. (1987). The learning and study strategies inventory. Clearwater: H & H Publishing.

    Google Scholar 

  • Williams, R. H., & Zimmerman, D. W. (1996). Are simple gain scores obsolete? Applied Psychological Measurement, 20, 59–69. doi:10.1177/014662169602000106

  • Winne, P. H. (1997). Experimenting to bootstrap self-regulated learning. Journal of Educational Psychology, 89, 397–410.

    Article  Google Scholar 

  • Winne, P. H. (2001). Self-regulated learning viewed from models of information processing. In B. J. Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (2nd ed., pp. 153–189). Mahwah: Lawrence Erlbaum.

    Google Scholar 

  • Winne, P. H. (2010). Improving measurements of self-regulated learning. Educational Psychologist, 45, 267–276. doi:10.1080/00461520.2010.517150.

    Article  Google Scholar 

  • Winne, P. H. (2011). A cognitive and metacognitive analysis of self-regulated learning. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of self-Regulation of learning and performance (pp. 15–32). New York: Routledge.

    Google Scholar 

  • Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277–304). Mahwah: Lawrence Erlbaum.

    Google Scholar 

  • Winne, P. H., & Hadwin, A. F. (2008). The weave of motivation and self-regulated learning. In D. H. Schunk & B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 297–314). Mahwah: Lawrence Erlbaum.

    Google Scholar 

  • Winne, P. H., & Jamieson-Noel, D. (2002). Exploring students’ calibration of self reports about study tactics and achievement. Contemporary Educational Psychology, 27, 551–572.

    Article  Google Scholar 

  • Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 531–566). Orlando: Academic.

    Chapter  Google Scholar 

  • Winne, P. H., Jamieson-Noel, D., & Muis, K. R. (2002). In P. R. Pintrich & M. L. Maehr (Eds.), Advances in motivation and achievement: New directions in measures and methods (Vol. 12, pp. 121–155). Amsterdam: Elsevier Science.

    Google Scholar 

  • Winne, P. H., Zhou, M., & Egan, R. (2011). Designing assessments of self-regulated learning. In G. Shraw & D. R. Robinson (Eds.), Assessment of higher order thinking skills (pp. 89–118). Charlotte: Information Age.

    Google Scholar 

  • Yang, Y., & Bliss, L. B. (2014). A Q factor analysis of college undergraduate students’ study behaviours. Educational Research and Evaluation: An International Journal on Theory and Practice, 20, 433–453.

    Article  Google Scholar 

  • Zeidner, M., Boekaerts, M., & Pintrich, P. R. (2000). Self-regulation: Directions and challenges for future research. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 749–768). San Diego: Academic.

    Chapter  Google Scholar 

  • Zimmerman, B. J. (1986). Becoming a self-regulated learner: which are the key subprocesses? Contemporary Educational Psychology, 11, 307–313.

    Article  Google Scholar 

  • Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81(3), 329–339. doi:10.1037//0022-0663.81.3.329.

    Article  Google Scholar 

  • Zimmerman, B. J. (2000). Attaining self-regulation: a social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 13–39). San Diego: Academic.

    Chapter  Google Scholar 

  • Zimmerman, B. J. (2008). Investigating self-regulation and motivation: historical background, methodological developments, and future prospects. American Educational Research Journal, 45, 166–183.

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by the Social Sciences and Humanities Research Council of Canada, Standard Research Grant 410-2008-0700 (PI: Hadwin) and Joseph-Armand Bombardier Canada Graduate Scholarship. We would like to acknowledge (a) invaluable consultation and assistance from Drs. Scott Hofer and Philip Winne, (b) qualitative coding assistance by Adrianna Haffey, and (c) thorough feedback from the special issue editors and anonymous reviewers on drafts of this manuscript.

Author information

Authors and Affiliations

  1. Educational Psychology and Leadership Studies, University of Victoria, Victoria, BC, V8W 3N4, Canada

    Lindsay McCardle & Allyson F. Hadwin

Authors
  1. Lindsay McCardle
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Allyson F. Hadwin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lindsay McCardle.

Additional information

This research was supported by the Social Sciences and Humanities Research Council of Canada, Standard Research Grant 410-2008-0700 (PI: Hadwin).

Appendices

Appendix A

Table 8 Weekly reflection items by semester
Full size table

Appendix B

Table 9 RLQ item loadings for step 1 EFA with 4- and 5-factor solutions
Full size table

Appendix C

Table 10 Items dropped from RLQ
Full size table

Appendix D

Table 11 RLQ item loadings for step 2 EFA with 4- and 5-factor solutions
Full size table

Appendix E

Table 12 Individual qualitative profiles for unengaged regulators
Full size table
Table 13 Individual qualitative profiles for active regulators
Full size table
Table 14 Individual qualitative profiles for struggling regulators
Full size table
Table 15 Individual qualitative profiles for emergent regulators
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

McCardle, L., Hadwin, A.F. Using multiple, contextualized data sources to measure learners’ perceptions of their self-regulated learning. Metacognition Learning 10, 43–75 (2015). https://doi.org/10.1007/s11409-014-9132-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11409-014-9132-0

Keywords

Search

Navigation