Abstract
The feelings of difficulty and familiarity (FOD and FOF) are two types of metacognitive experiences. Both may influence student engagement and the application of metacognitive strategies, but these relationships are not well understood, in part because many studies have relied on self-report measures of behaviors that may not accurately reflect students’ actual behaviors. In this study, FOD and FOF were related to objective measures of off-task behaviors and metacognitive strategies. These measures were extracted from 88 sixth graders’ action logs within a computer-based learning environment known as Betty’s Brain. Pre- and post-tests were administered to assess learning. Results reveal that high-FOD students showed more off-task behaviors and fewer strategic behaviors than low-FOD students, particularly when this difference was measured in terms of the frequency (as opposed to proportion) of strategic behaviors. FOF was not associated with off-task behaviors and metacognitive strategies but emerged as a moderator in the relationship between FOD and learning gains. Low-FOD students learned more than high-FOD students in the low-FOF group, but such a difference was not found in the high-FOF group.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Readers unfamiliar with Betty’s Brain may think the question is asking the difficulty of the concept of thermoregulation rather than the difficulty of learning thermoregulation. However, a Betty’s Brain unit mainly refers to the task of building a causal map for the unit rather than the unit’s resource book. We believe that students considered the Betty’s Brain’s unit on thermoregulation in terms of this perspective because they had received training on Betty’s Brain and used the system for several days.
We did not use the WRS2 package because it did not offer the three-way repeated ANOVA.
References
Alexander, P. A., Kulikowich, J. M., & Jetton, T. L. (1994). The role of subject-matter knowledge and interest in the processing of linear and nonlinear texts. Review of Educational Research, 64(2), 201–252. https://doi.org/10.3102/00346543064002201
Alter, A. L., Oppenheimer, D. M., Epley, N., & Eyre, R. N. (2007). Overcoming intuition: Metacognitive difficulty activates analytic reasoning. Journal of Experimental Psychology: General, 136(4), 569–576. https://doi.org/10.1037/0096-3445.136.4.569
Ayres, P. (2006). Using subjective measures to detect variations of intrinsic cognitive load within problems. Learning and Instruction, 16(5), 389–400. https://doi.org/10.1016/j.learninstruc.2006.09.001
Azevedo, R. (2015). Defining and measuring engagement and learning in science: Conceptual, theoretical, methodological, and analytical issues. Educational Psychologist, 50(1), 84–94. https://doi.org/10.1080/00461520.2015.1004069
Azevedo R., Aleven V. (2013) Metacognition and learning technologies: An overview of current interdisciplinary research. In Azevedo R. & Aleven V. (Eds), International handbook of metacognition and learning technologies (pp. 1–16). New York, NY: Springer. https://doi.org/10.1007/978-1-4419-5546-3_1
Azevedo, R., & Cromley, J. G. (2004). Does training on self-regulated learning facilitate students’ learning with hypermedia? Journal of Educational Psychology, 96(3), 523–535. https://doi.org/10.1037/0022-0663.96.3.523
Azevedo, R., Harley, J., Trevors, G., Duffy, M., Feyzi-Behnagh, R., Bouchet, F., & Landis, R. (2013). Using trace data to examine the complex roles of cognitive, metacognitive, and emotional self-regulatory processes during learning with multi-agent systems. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 427–449). Springer New York. https://doi.org/10.1007/978-1-4419-5546-3_28
Beckmann, J. F., & Goode, N. (2014). The benefit of being naïve and knowing it: The unfavourable impact of perceived context familiarity on learning in complex problem solving tasks. Instructional Science, 42(2), 271–290. https://doi.org/10.1007/s11251-013-9280-7
Biswas, G., Segedy, J. R., & Bunchongchit, K. (2016). From design to implementation to practice a learning by teaching system: Betty’s brain. International Journal of Artificial Intelligence in Education, 26(1), 350–364. https://doi.org/10.1007/s40593-015-0057-9
Borgers, N., Sikkel, D., & Hox, J. (2004). Response effects in surveys on children and adolescents: The effect of number of response options, negative wording, and neutral mid-point. Quality & Quantity, 38(1), 17–33. https://doi.org/10.1023/B:QUQU.0000013236.29205.a6
Brunner, E., & Puri, M. L. (2001). Non-parametric methods in factorial designs. Statistical Papers, 42(1), 1–52. https://doi.org/10.1007/s003620000039
Ching-En, C. (2018). Metacognitive experience modeling using eye-tracking. In Penstein Rosé C. et al. (Eds), Artificial Intelligence in Education. AIED 2018. Lecture Notes in Computer Science, Vol 10948 (pp. 503–507). Cham, Germany: Springer. https://doi.org/10.1007/978-3-319-93846-2_94
Craig, K., Hale, D., Grainger, C., & Stewart, M. E. (2020). Evaluating metacognitive self-reports: Systematic reviews of the value of self-report in metacognitive research. Metacognition and Learning, 15, 155–213. https://doi.org/10.1007/s11409-020-09222-y
D'Angelo, C., Rutstein, D., Harris, C., Bernard, R., Borokhovski, E., ... Haertel, G. (2014). Simulations for STEM learning: Systematic review and meta-analysis. Menlo Park, CA: SRI International.
Dent, A. L., & Koenka, A. C. (2016). The relation between self-regulated learning and academic achievement across childhood and adolescence: A meta-analysis. Educational Psychology Review, 28(3), 425–474. https://doi.org/10.1007/s10648-015-9320-8
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed mode surveys: The tailored design method (4th ed.). John Wiley & Sons Inc.
Dindar, M., Järvelä, S., & Järvenoja, H. (2020). Interplay of metacognitive experiences and performance in collaborative problem solving. Computers and Education, 154, Article 103922. https://doi.org/10.1016/j.compedu.2020.103922
Efklides, A. (2002). Feelings and judgments as subjective evaluations of cognitive processing: How reliable are they? Psychology: The Journal of the Hellenic Psychological Society, 9(2), 163–184.
Efklides, A. (2006). Metacognition and affect: What can metacognitive experiences tell us about the learning process? Educational Research Review, 1(1), 3–14. https://doi.org/10.1016/j.edurev.2005.11.001
Efklides, A. (2008). Metacognition: Defining its facets and levels of functioning in relation to self-regulation and co-regulation. European Psychologist, 13(4), 277–287. https://doi.org/10.1027/1016-9040.13.4.277
Efklides, A. (2009). The role of metacognitive experiences in the learning process. Psicothema, 21(1), 76–82.
Efklides, A. (2011). Interactions of metacognition with motivation and affect in self-regulated learning: The MASRL model. Educational Psychologist, 46(1), 6–25. https://doi.org/10.1080/00461520.2011.538645
Efklides, A. (2016). Metamemory and affect. In J. Dunlosky & S. U. K. Tauber (Eds.), The Oxford handbook of metamemory. (pp. 245–267). New York, NY: Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199336746.013.1
Efklides, A., Kourkoulou, A., Mitsiou, F., & Ziliaskopoulou, D. (2006). Metacognitive knowledge of effort, personality factors, and mood state: Their relationships with effort-related metacognitive experiences. Metacognition and Learning, 1(1), 33–49. https://doi.org/10.1007/s11409-006-6581-0
Efklides, A., Papadaki, M., Papantoniou, G., & Kiosseoglou, G. (1998). Individual differences in feelings of difficulty: The case of school mathematics. European Journal of Psychology of Education, 13(2), 207–226. https://doi.org/10.1007/BF03173090
Efklides, A., Samara, A., & Petropoulou, M. (1999). Feeling of difficulty: An aspect of monitoring that influences control. European Journal of Psychology of Education, 14(4), 461–476. https://doi.org/10.1007/BF03172973
Efklides, A., & Tsiora, A. (2002). Metacognitive experiences, self-concept, and self-regulation. Psychologia, 45(4), 222–236. https://doi.org/10.2117/psysoc.2002.222
Evans, J. S. B. T., & Stanovich, K. E. (2013). Dual-process theories of higher cognition. Perspectives on Psychological Science, 8(3), 223–241. https://doi.org/10.1177/1745691612460685
Feldon, D. F., Callan, G., Juth, S., & Jeong, S. (2019). Cognitive load as motivational cost. Educational Psychology Review, 31(2), 319–337. https://doi.org/10.1007/s10648-019-09464-6
Field, A. (2013). Discovering statistics using IBM SPSS statistics (4th ed.). SAGE.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906–911. https://doi.org/10.1037/0003-066X.34.10.906
Fulmer, S. M., & Tulis, M. (2013). Changes in interest and affect during a difficult reading task: Relationships with perceived difficulty and reading fluency. Learning and Instruction, 27, 11–20. https://doi.org/10.1016/j.learninstruc.2013.02.001
Garcia-Marques, T., Mackie, D. M., Maitner, A. T., & Claypool, H. M. (2016). Moderation of the familiarity-stereotyping effect: The role of stereotype fit. Social Cognition, 34(2), 81–96. https://doi.org/10.1521/soco.2016.34.2.81
Garcia-Marques, T., & Mackie, D. M. (2001). The feeling of familiarity as a regulator of persuasive processing. Social Cognition, 19(1), 9–34. https://doi.org/10.1521/soco.19.1.9.18959
KE Godwin H Seltman M Almeda M Davis Skerbetz S Kai RS Baker AV Fisher 2021 The elusive relationship between time on-task and learning: Not simply an issue of measurement Educational Psychology 1–18 https://doi.org/10.1080/01443410.2021.1894324
Gunthert, K. C., & Wenze, S. J. (2012). Daily diary methods. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 144–159). The Guilford Press.
Guthrie, J. T., Klauda, S. L., & Ho, A. N. (2013). Modeling the relationships among reading instruction, motivation, engagement, and achievement for adolescents. Reading Research Quarterly, 48(1), 9–26. https://doi.org/10.1002/rrq.035
Hansen, J., & Wänke, M. (2009). Liking what’s familiar: The importance of unconscious familiarity in the mere-exposure effect. Social Cognition, 27(2), 161–182. https://doi.org/10.1521/soco.2009.27.2.161
Henrie, C. R., Halverson, L. R., & Graham, C. R. (2015). Measuring student engagement in technology-mediated learning: A review. Computers and Education, 90, 36–53. https://doi.org/10.1016/j.compedu.2015.09.005
Kinoshita, S. (1997). Masked target priming effects on feeling-of-knowing and feeling-of-familiarity judgments. Acta Psychologica, 97(2), 183–199. https://doi.org/10.1016/S0001-6918(97)00018-8
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, Article 863. https://doi.org/10.3389/fpsyg.2013.00863
Liu, C., Liu, J., Cole, M., Belkin, N. J., & Zhang, X. (2012). Task difficulty and domain knowledge effects on information search behaviors. Proceedings of the American Society for Information Science and Technology, 49(1), 1–10. https://doi.org/10.1002/meet.14504901142
Lupo, S. M., Strong, J. Z., & Conradi Smith, K. (2019). Struggle is not a bad word: Misconceptions and recommendations about readers struggling with difficult texts. Journal of Adolescent and Adult Literacy, 62(5), 551–560. https://doi.org/10.1002/jaal.926
Mair, P., & Wilcox, R. (2020). Robust statistical methods in R using the WRS2 package. Behavior Research Methods, 52(2), 464–488. https://doi.org/10.3758/s13428-019-01246-w
Malmberg, L., Lim, W. H., Tolvanen, A., & Nurmi, J. (2016). Within-students variability in learning experiences, and teachers’ perceptions of students’ task-focus. Frontline Learning Research, 4(5), 62–82. https://doi.org/10.14786/flr.v4i5.227
Malmberg, L., Walls, T. A., Martin, A. J., Little, T. D., & Lim, W. H. (2013). Primary school students’ learning experiences of, and self-beliefs about competence, effort, and difficulty: Random effects models. Learning and Individual Differences, 28, 54–65. https://doi.org/10.1016/j.lindif.2013.09.007
Meany Daboul, M. G., Roscoe, E. M., Bourret, J. C., & Ahearn, W. H. (2007). A comparison of momentary time sampling and partial-interval recording for evaluating functional relations. Journal of Applied Behavior Analysis, 40(3), 501–514. https://doi.org/10.1901/jaba.2007.40-501
Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43–59. https://doi.org/10.1007/BF02505024
Metcalfe, J. (2009). Metacognitive judgments and control of study. Current Directions in Psychological Science, 18(3), 159–163. https://doi.org/10.1111/j.1467-8721.2009.01628.x
Molenaar, I. (2014). Advances in temporal analysis in learning and instruction. Frontline Learning Research, 2(4), 15–24. https://doi.org/10.14786/flr.v2i4.118
Nelson, T. O., & Narens, L. (1990). Metamemory: A theoretical framework and new findings. Psychology of Learning and Motivation, 26, 125–173. https://doi.org/10.1016/S0079-7421(08)60053-5
Noguchi, K., Gel, Y. R., Brunner, E., & Konietschke, F. (2012). nparLD: An R software package for the non-parametric analysis of longitudinal data in factorial experiments. Journal of Statistical Software, 50(12), 1–23. https://doi.org/10.18637/jss.v050.i12
Ohtani, K., & Hisasaka, T. (2018). Beyond intelligence: A meta-analytic review of the relationship among metacognition, intelligence, and academic performance. Metacognition and Learning, 13(2), 179–212. https://doi.org/10.1007/s11409-018-9183-8
Patall, E. A., Hooper, S., Vasquez, A. C., Pituch, K. A., & Steingut, R. R. (2018). Science class is too hard: Perceived difficulty, disengagement, and the role of teacher autonomy support from a daily diary perspective. Learning and Instruction, 58, 220–231. https://doi.org/10.1016/j.learninstruc.2018.07.004
Pintrich, P. R., Smith, D. A., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the motivated strategies for learning questionnaire (MSLQ). Educational and Psychological Measurement, 53(3), 801–813. https://doi.org/10.1177/0013164493053003024
Qiu, X., & Lo, Y. Y. (2016). Content familiarity, task repetition and Chinese EFL learners’ engagement in second language use. Language Teaching Research, 21(6), 681–698. https://doi.org/10.1177/1362168816684368
Reder, L. M., Paynter, C., Diana, R. A., Ngiam, J., & Dickison, D. (2007). Experience is a double-edged sword: A computational model of the encoding/retrieval trade-off with familiarity. In A. S. Benjamin & B. H. Ross (Eds.), The psychology of learning and motivation: Vol. 48. Skill and strategy in memory use (pp. 271–312). Elsevier Academic Press. https://doi.org/10.1016/S0079-7421(07)48007-0
Reder, L. M., & Ritter, F. E. (1992). What determines initial feeling of knowing? Familiarity with question terms, not with the answer. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(3), 435–451. https://doi.org/10.1037/0278-7393.18.3.435
Rellinger, E., Borkowski, J. G., Turner, L. A., & Hale, C. A. (1995). Perceived task difficulty and intelligence: Determinants of strategy use and recall. Intelligence, 20(2), 125–143. https://doi.org/10.1016/0160-2896(95)90029-2
Rivet, A. E., & Krajcik, J. S. (2008). Contextualizing instruction: Leveraging students’ prior knowledge and experiences to foster understanding of middle school science. Journal of Research in Science Teaching, 45(1), 79–100. https://doi.org/10.1002/tea.20203
Rovers, S. F. E., Clarebout, G., Savelberg, H. H. C. M., de Bruin, A. B. H., & van Merriënboer, J. J. G. (2019). Granularity matters: Comparing different ways of measuring self-regulated learning. Metacognition and Learning, 14(1), 1–19. https://doi.org/10.1007/s11409-019-09188-6
Schmeck, A., Opfermann, M., van Gog, T., Paas, F., & Leutner, D. (2015). Measuring cognitive load with subjective rating scales during problem solving: Differences between immediate and delayed ratings. Instructional Science, 43(1), 93–114. https://doi.org/10.1007/s11251-014-9328-3
Schwartz, B. L., & Metcalfe, J. (1992). Cue familiarity but not target retrievability enhances feeling-of-knowing judgments. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(5), 1074–1083. https://doi.org/10.1037/0278-7393.18.5.1074
Segedy, J. R. (2014). Adaptive Scaffolds in Open-Ended Computer-Based Learning Environments (Order No. 3674132). [Doctoral dissertation, Vanderbilt University]. ProQuest Dissertations & Theses Global.
Segedy, J. R., Kinnebrew, J. S., & Biswas, G. (2015). Using coherence analysis to characterize self-regulated learning behaviours in open-ended learning environments. Journal of Learning Analytics, 2(1), 13–48. https://doi.org/10.18608/jla.2015.21.3
Sockalingam, N., & Schmidt, H. G. (2013). Does the extent of problem familiarity influence students’ learning in problem-based learning? Instructional Science, 41(5), 921–932. https://doi.org/10.1007/s11251-012-9260-3
Soemer, A., Idsardi, H. M., Minnaert, A., & Schiefele, U. (2019). Mind wandering and reading comprehension in secondary school children. Learning and Individual Differences, 75, Article 101778. https://doi.org/10.1016/j.lindif.2019.101778
Soppe, M., Schmidt, H. G., & Bruysten, R. J. M. P. (2005). Influence of problem familiarity on learning in a problem-based course. Instructional Science, 33(3), 271–281. https://doi.org/10.1007/s11251-004-7688-9
Tansey, E. A., & Johnson, C. D. (2015). Recent advances in thermoregulation. Advances in Physiology Education, 39(3), 139–148. https://doi.org/10.1152/advan.00126.2014
Tornare, E., Czajkowski, N. O., & Pons, F. (2015). Children’s emotions in math problem solving situations: Contributions of self-concept, metacognitive experiences, and performance. Learning and Instruction, 39, 88–96. https://doi.org/10.1016/j.learninstruc.2015.05.011
Touroutoglou, A., & Efklides, A. (2010). Cognitive interruption as an object of metacognitive monitoring: Feeling of difficulty and surprise. In A. Efklides & P. Misailidi (Eds.), Trends and prospects in metacognition research (pp. 171–208). Boston: Springer. https://doi.org/10.1007/978-1-4419-6546-2_9
Tulis, M., & Fulmer, S. M. (2013). Students’ motivational and emotional experiences and their relationship to persistence during academic challenge in mathematics and reading. Learning and Individual Differences, 27, 35–46. https://doi.org/10.1016/j.lindif.2013.06.003
van Gog, T., & Paas, F. (2008). Instructional efficiency: Revisiting the original construct in educational research. Educational Psychologist, 43(1), 16–26. https://doi.org/10.1080/00461520701756248
Veenman, M. V. J., Elshout, J. J., & Meijer, J. (1997). The generality vs domain-specificity of metacognitive skills in novice learning across domains. Learning and Instruction, 7(2), 187–209. https://doi.org/10.1016/S0959-4752(96)00025-4
Whittlesea, B. W. A. (1993). Illusions of familiarity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19(6), 1235–1253. https://doi.org/10.1037/0278-7393.19.6.1235
Whittlesea, B. W., & Williams, L. D. (1998). Why do strangers feel familiar, but friends don’t? A discrepancy-attribution account of feelings of familiarity. Acta Psychologica, 98(2–3), 141–165. https://doi.org/10.1016/S0001-6918(97)00040-1
Wilcox, R. R. (2011). Introduction to robust estimation and hypothesis testing (3rd ed). San Diego, CA: Academic press. https://doi.org/10.1016/B978-0-12-804733-0.00014-7
Winn, W., Stahr, F., Sarason, C., Fruland, R., & Oppenheimer, P.,... Lee, Y. L. (2006). Learning oceanography from a computer simulation compared with direct experience at sea. Journal of Research in Science Teaching, 43(1), 25–42. https://doi.org/10.1002/tea.20097
Yang, Z., Barnard-Brak, L., & Lan, W. Y. (2019). Examining the association of over-claiming with mathematics achievement. Learning and Individual Differences, 70, 30–38. https://doi.org/10.1016/j.lindif.2019.01.004
Zhang, N., Biswas, G., & Hutchins, N. (2021). Measuring and analyzing students’ strategic learning behaviors in open-ended learning environments. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-021-00275-x
Zhang, Y., Paquette, L., Baker, R. S., Ocumpaugh, J., Bosch, N., Munshi, A., & Biswas, G. (2020). The relationship between confusion and metacognitive strategies in Betty’s Brain. In C. Rensing & H. Drachsler (Eds.), Proceedings of the Tenth International Conference on Learning Analytics & Knowledge (pp. 276–284). Association for Computing Machinery. https://doi.org/10.1145/3375462.3375518
Acknowledgements
We thank Anabil Munshi for his help in preprocessing the data and Stefan Slater for his helpful comments on an earlier version of this paper.
Funding
This work was supported by the National Science Foundation (grant numbers 1561676) and the China Scholarship Council (grant numbers 201806040180).
Author information
Authors and Affiliations
Contributions
Yingbin Zhang—conceptualization, methodology, formal analysis, writing (original draft preparation, review, and editing). Luc Paquette—conceptualization, methodology, writing (review and editing), supervision. Ryan S. Baker—methodology, writing (review and editing). Nigel Bosch—methodology, writing (review and editing). Jaclyn Ocumpaugh—methodology, investigation, writing (review and editing). Gautam Biswas—methodology, resources, writing (review and editing).
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Luc Paquette. Department of Curriculum & Instruction, University of Illinois at Urbana-Champaign, 1310 S Sixth Street, Champaign, IL 61820, USA. ORCID: 0000-0002-2738-3190.
Yingbin Zhang. Department of Curriculum & Instruction, University of Illinois at Urbana-Champaign, 1310 S Sixth Street, Champaign, IL 61820, USA. E-mail: yingbin2@illinois.edu. ORCID: 0000-0002-2664-3093.
Current themes of research:
Our current themes of research are students’ self-regulated learning processes during science learning within open-ended environments. Specifically, we focus on the interactions among behaviors, affect, and motivation.
Most relevant publications in the field of Psychology of Education:
Paquette, L., Grant, T., Zhang, Y., Biswas, G., & Baker, R. (2021). Using epistemic networks to analyze self-regulated learning in an open-ended problem-solving environment. In Ruis A.R., Lee S.B. (eds), Advances in Quantitative Ethnography. ICQE 2021. Communications in Computer and Information Science, vol 1312 (pp. 185–201).Cham: Springer. https://link.springer.com/chapter/10.1007/978-3-030-67788-6_13#citeas
Zhang, Y., Paquette, L., Baker, R. S., Ocumpaugh, J., Bosch, N., Munshi, A., & Biswas, G. (2020).The relationship between confusion and metacognitive strategies in Betty's Brain. In Rensing, C. & Drachsler, H. (Eds), Proceedings of the 10th International Conference on Learning Analytics & Knowledge (LAK'20) (pp. 276–284). New York, NY: ACM. https://doi.org/10.1145/3375462.3375518
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, Y., Paquette, L., Baker, R.S. et al. How are feelings of difficulty and familiarity linked to learning behaviors and gains in a complex science learning task?. Eur J Psychol Educ 38, 777–800 (2023). https://doi.org/10.1007/s10212-022-00616-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10212-022-00616-x