Skip to main content

Effects of reflection prompts on learning outcomes and learning behaviour in statistics education

Abstract

Starting from difficulties that students display when they deal with correlation analysis, an e-learning environment (‘Koralle’) was developed. The design was inspired by principles of situated and example-based learning. In order to facilitate reflective processes and thus enhance learning outcomes, reflection prompts were integrated into the learning environment. A total of 57 university students were randomly assigned to two experimental conditions: 28 students were prompted to give reasons for their decisions while working within the learning environment (EG 1); and 29 students dealt with Koralle without being prompted (EG 2). The control group consisted of 67 students who had already attended regular statistics lectures but had no access to the e-learning environment. EG 1 scored significantly higher in the posttest than EG 2, and the effect was practically relevant and sustainable. Reflection prompts did not influence time on task, task choices and motivational outcomes. Both experimental groups clearly outperformed the control group.

This is a preview of subscription content, access via your institution.

Notes

  1. 1.

    Effect sizes were classified according to the conventions specified by Cohen (1988): η² ≤ 0.05: small effect; 0.06 ≤ η² ≤ 0.13: medium effect; η² ≥ 0.14: large effect.

  2. 2.

    Adjusted degrees of freedom because of heterogeneous variance. For all comparisons between experimental and control group, p values were also calculated using a non-parametric test (Mann–Whitney U) because of differences in sample sizes. The U-test results did not differ from the results of the t-tests.

References

  1. Alexander, P. A. (1996). The past, present, and future of knowledge research: A re-examination of the role of knowledge in learning and instruction. Educational Psychologist, 3, 89–92.

    Google Scholar 

  2. Allwood, C. M. (1990). On the relation between justification of solution method and correctness of solution in statistical problem solving. Scandinavian Journal of Psychology, 31, 181–190.

    Article  Google Scholar 

  3. Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. W. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70, 181–214.

    Google Scholar 

  4. Atkinson, R. K., Renkl, A., & Merrill, M. M. (2003). Transitioning from studying examples to solving problems: Combining fading with prompting fosters learning. Journal of Educational Psychology, 95, 774–783.

    Article  Google Scholar 

  5. Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educational Psychologist (Special Issue on Computers as Metacognitive Tools for Enhancing Student Learning), 40, 199–209.

    Google Scholar 

  6. Bannert, M. (2003). Effekte metakognitiver Lernhilfen auf den Wissenserwerb in vernetzten Lernumgebungen [Effects of metacognitive learning aids on knowledge acquisition in networked learning environments]. Zeitschrift für Pädagogische Psychologie, 17, 13–25.

    Article  Google Scholar 

  7. Berthold, K., Nückles, M., & Renkl, A. (2007). Do learning protocols support learning strategies and outcomes? The role of cognitive and metacognitive prompts. Learning and Instruction, 17, 564–577.

    Article  Google Scholar 

  8. Broers, N. J. (2002). Selection and use of propositional knowledge in statistical problem solving. Learning and Instruction, 12, 323–344.

    Article  Google Scholar 

  9. Brown, A., & DeLoache, J. S. (1978). Skills, plans and self-regulation. In R. Siegler (Ed.), Children’s thinking: What develops? (pp. 3–35). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  10. Castro Sotos, A. E., Vanhoof, S., Van den Noortgate, W., & Onghena, P. (2007). Students’ misconceptions of statistical inference: A review of the empirical evidence from research on statistics education. Educational Research Review, 2, 98–113.

    Article  Google Scholar 

  11. Chi, M. T. H. (1996). Constructing self-explanations and scaffolded explanations in tutoring. Applied Cognitive Psychology, 10, S33–S49.

    Article  Google Scholar 

  12. Chi, M. T. H., Bassok, M., Lewis, M., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145–182.

    Article  Google Scholar 

  13. Chi, M. T. H., DeLeeuw, N., Chiu, M. H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439–477.

    Article  Google Scholar 

  14. Cognition and Technology Group at Vanderbilt. (1997). The Jasper project: Lessons in curriculum, instruction, assessment and professional development. Mahwah, NJ: Erlbaum.

    Google Scholar 

  15. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  16. Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997–1003.

    Article  Google Scholar 

  17. Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing and mathematics. In L. B. Resnick (Ed.), Knowing, learning, and instruction (pp. 453–494). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  18. De Jong, T., & Ferguson-Hessler, M. G. M. (1996). Types and qualities of knowledge. Educational Psychologist, 31, 105–113.

    Article  Google Scholar 

  19. Deci, E. L., & Ryan, R. M. (2000). The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11, 227–268.

    Article  Google Scholar 

  20. Dillon, A., & Gabbard, R. (1998). Hypermedia as an educational technology: A review of the quantitative research literature on learner comprehension, control, and style. Review of Educational Research, 68, 322–349.

    Google Scholar 

  21. Dochy, F. J. R. C. (1992). Assessment of prior knowledge as a determinant for future learning. The use of prior knowledge state tests and knowledge profiles. Utrecht, The Netherlands: Uitgeverij Lemma B. V.

    Google Scholar 

  22. Ericsson, K. A., & Simon, H. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.

    Google Scholar 

  23. Flavell, J. (1976). Metacognitive aspects of problem solving. In L. B. Resnick (Ed.), The nature of intelligence (pp. 231–235). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  24. Flavell, J. (1979). Metacognition and cognitive monitoring: A new area of cognitive developmental inquiry. American Psychologist, 34, 906–911.

    Article  Google Scholar 

  25. Gerjets, P., Scheiter, K., & Catrambone, R. (2004). Designing instructional examples to reduce intrinsic cognitive load: Molar versus modular presentation of solution procedures. Instructional Science, 32, 33–58.

    Article  Google Scholar 

  26. Gerjets, P., Scheiter, K., & Schuh, J. (2005). Instruktionale Unterstützung beim Fertigkeitserwerb aus Beispielen in hypertextbasierten Lernumgebungen [Instructional support for skill acquisition from examples in hypermedia-based learning environments]. Zeitschrift für Pädagogische Psychologie, 19(1/2), 23–38.

    Article  Google Scholar 

  27. Große, C. S., & Renkl, A. (2006). Effects of multiple solution methods in mathematics learning. Learning and Instruction, 16, 122–138.

    Article  Google Scholar 

  28. Hasselhorn, M. (2006). Metakognition [Metacognition]. In D. H. Rost (Ed.), Handwörterbuch Pädagogische Psychologie [Handbook of educational psychology] (pp. 480–485). Weinheim, Germany: Psychologie Verlags Union.

    Google Scholar 

  29. Hasselhorn, M., & Hager, W. (1998). Kognitive Trainings auf dem Prüfstand: Welche Komponenten charakterisieren erfolgreiche Fördermaßnahmen? [Cognitive trainings put to test: What components characterise successful facilitation methods?]. In M. Beck (Ed.), Evaluation als Maßnahme der Qualitätssicherung: Pädagogisch-psychologische Interventionen auf dem Prüfstand [Evaluation as a means of quality management: Pedagogical-psychological interventions put to test] (pp. 85–98). Tübingen, Germany: DGVT-Verlag.

    Google Scholar 

  30. Hidi, S. (1990). Interest and its contribution as a mental resource for learning. Review of Educational Research, 60(4), 549–571.

    Google Scholar 

  31. Kintsch, W. (1994). Text comprehension, memory, and learning. American Psychologist, 49, 294–303.

    Article  Google Scholar 

  32. Kopp, V., Stark, R., & Fischer, M. R. (2007). Förderung von Diagnosekompetenz in der medizinischen Ausbildung durch Implementation eines Ansatzes zum fallbasierten Lernen aus Lösungsbeispielen [Enhancing diagnostic skills in medical training by case-based learning from worked examples]. GMS Zeitschrift für Medizinische Ausbildung, 24(2): Doc 107.

    Google Scholar 

  33. Krapp, A. (2000). Individuelle Interessen als Bedingung lebenslangen Lernens [Individual interests as a condition for life-long learning]. In F. Achtenhagen & W. Lempert (Eds.), Lebenslanges Lernen im Beruf—seine Grundlegung im Kindes-und Jugendalter, Band 3: Psychologische Theorie, Empirie und Therapie [Professional lifelong learning—its foundation in childhood and adolescence, Volume 3: Psychological theory, empiricism and therapy] (pp. 54–75). Opladen, Germany: Leske + Budrich.

  34. Krapp, A. (2005). Basic needs and the development of interest and intrinsic motivational orientations. Learning and Instruction, 15, 381–395.

    Article  Google Scholar 

  35. Krause, U.-M. (2007). Feedback und kooperatives Lernen [Feedback and cooperative learning]. Münster, Germany: Waxmann.

    Google Scholar 

  36. Krause, U.-M., & Stark, R. (2006). Vorwissen aktivieren [Activating prior knowledge]. In H. Mandl & H. F. Friedrich (Eds.), Handbuch Lernstrategien [Handbook of learning strategies] (pp. 38–49). Göttingen, Germany: Hogrefe.

    Google Scholar 

  37. Krause, U.-M., & Stark, R. (2007, August). Problem-oriented learning and reflection prompts in teacher education: Enhancing knowledge about cooperative learning. Paper presented at the 12th Biennial Conference of the European Association for Research on Learning and Instruction (EARLI), Budapest, Hungary.

  38. Krause, U.-M., Stark, R., & Mandl, H. (2009). The effects of cooperative learning and feedback on e-learning in statistics. Learning and Instruction, 19, 158–170.

    Article  Google Scholar 

  39. Lan, W. Y., Bradley, L., & Parr, G. (1993). The effects of a self-monitoring process on college students’ learning in an introductory statistics course. The Journal of Experimental Education, 62, 26–40.

    Google Scholar 

  40. Langer, E. J. (1993). A mindful education. Educational Psychologist, 28, 43–50.

    Article  Google Scholar 

  41. Mandl, H., & Gräsel, C. (1997). Gestaltung konstruktivistischer Lernumgebungen in der Medizin [Designing constructivist learning environments in medical science]. In S. Höfling & H. Mandl (Eds.), Lernen für die Zukunft—Lernen in der Zukunft: Wissensmanagement in der Bildung [Learning for the future—learning in the future: Knowledge management in education] (pp. 155–165). München, Germany: Hanns-Seidel-Stiftung e.V.

    Google Scholar 

  42. Marsh, H. W., & Craven, R. G. (2006). Reciprocal effects of self-concept and performance from a multidimensional perspective. Perspectives on Psychological Science, 1, 133–163.

    Article  Google Scholar 

  43. McClelland, D. J., Atkinson, J. W., Clark, R. H., & Lowell, E. L. (1953). The achievement motive. New York: Appleton-Century-Crofts.

    Book  Google Scholar 

  44. Morris, E. (2001). The design and evaluation of link: A computer-based learning system for correlation. British Journal of Educational Technology, 32, 39–52.

    Article  Google Scholar 

  45. Paas, F. G. W. C. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84, 429–434.

    Article  Google Scholar 

  46. Paris, S. G., Lipson, M. Y., & Wixon, K. K. (1983). Becoming a strategic reader. Contemporary Educational Psychology, 8, 293–316.

    Article  Google Scholar 

  47. Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the motivated strategies for learning questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801–813.

    Article  Google Scholar 

  48. Pirolli, P. L., & Anderson, J. R. (1985). The role of practice in fact retrieval. Journal of Experimental Psychology. Learning, Memory, and Cognition, 11, 136–153.

    Article  Google Scholar 

  49. Reigeluth, C. M., & Stein, F. S. (1983). The elaboration theory of instruction. In C. M. Reigeluth (Ed.), Instructional-design theories and models: An overview of their current status (pp. 335–382). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  50. Renkl, A. (1997). Learning from worked-out examples: A study on individual differences. Cognitive Science, 21, 1–29.

    Article  Google Scholar 

  51. Renkl, A., Atkinson, R. K., & Große, C. S. (2004). How fading worked solution steps works—a cognitive load perspective. Instructional Science, 32, 59–82.

    Article  Google Scholar 

  52. Salomon, G., & Globerson, T. (1987). Skill may not be enough: The role of mindfulness in learning transfer. International Journal of Educational Research, 11, 623–637.

    Article  Google Scholar 

  53. Schiefele, U., Krapp, A., & Schreyer, I. (1993). Metaanalyse des Zusammenhangs von Interesse und schulischer Leistung [Meta-analysis of the correlation between interest and academic performance]. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 25, 120–148.

    Google Scholar 

  54. Schneider, W., & Pressley, M. (1989). Memory development between 2 and 20. New York: Springer.

    Google Scholar 

  55. Schoenfeld, A. H. (1985). Mathematical problem solving. New York: Academic Press.

    Google Scholar 

  56. Spiro, R., Feltovich, P. J., Jacobson, M. J., & Coulson, R. L. (1991). Cognitive flexibility, constructivism, and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. Educational Technology, 31(5), 24–33.

    Google Scholar 

  57. Stark, R. (1999). Lernen mit Lösungsbeispielen: Einfluss unvollständiger Lösungsbeispiele auf Beispielelaboration, Lernerfolg und Motivation [Learning with worked-out examples: The impact of incomplete examples on example elaboration, learning outcomes and motivation]. Göttingen, Germany: Hogrefe.

    Google Scholar 

  58. Stark, R. (2004). Implementing example-based learning and teaching in the context of vocational school education in business administration. Learning Environments Research, 7, 143–163.

    Article  Google Scholar 

  59. Stark, R., Gruber, H., Renkl, A., & Mandl, H. (2000). Instruktionale Effekte einer kombinierten Lernmethode: Zahlt sich die Kombination von Lösungsbeispielen und Problemlöseaufgaben aus? [Instructional effects of a combined teaching approach: Does the combination of worked-out examples and problem-solving tasks pay off?]. Zeitschrift für Pädagogische Psychologie, 14, 205–217.

    Article  Google Scholar 

  60. Stark, R., & Mandl, H. (2000). Training in empirical research methods: Analysis of problems and intervention from a motivational perspective. In J. Heckhausen (Ed.), Motivational psychology of human development (pp. 165–183). Amsterdam, The Netherlands: Elsevier.

    Chapter  Google Scholar 

  61. Stark, R., & Mandl, H. (2002). Konzeptualisierung und Evaluation einer komplexen netzbasierten Lernumgebung im Kontext der universitären Ausbildung in empirischen Forschungsmethoden [Conceptualisation and evaluation of a complex net-based learning environment in university training of empirical research methods]. Unterrichtswissenschaft, 30, 315–330.

    Google Scholar 

  62. Stark, R., & Mandl, H. (2005). Lernen mit einer netzbasierten Lernumgebung im Bereich empirischer Forschungsmethoden: Effekte zusätzlich implementierter Maßnahmen und Bedeutung von Lernvoraussetzungen [Learning within a net-based learning environment in the field of empirical research methods: Effects of additional interventions and impact of learning prerequisites]. Unterrichtswissenschaft, 33(1), 3–29.

    Google Scholar 

  63. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.

    Article  Google Scholar 

  64. Sweller, J., Van Merriёnboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–296.

    Article  Google Scholar 

  65. Tyroller, M. (2005). Effekte metakognitiver Prompts beim computerbasierten Statistiklernen [Effects of metacognitive prompts in computer-based statistics learning]. Retrieved July 21, 2009, from http://edoc.ub.uni-muenchen.de/archive/00005514/01/Michael_Tyroller.pdf.

  66. Van Merriënboer, J. J. G. (1990). Strategies for programming instruction in high school: Program completion vs. program generation. Journal of Computing Research, 6, 265–285.

    Google Scholar 

  67. Van Merriënboer, J. J. G., & de Croock, M. B. M. (1992). Strategies for computer-based programming instruction: Program completion vs. program generation. Journal of Educational Computing Research, 8, 365–394.

    Google Scholar 

  68. Veenman, M. V. J., Van Hout-Wolters, B. H. A. M., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1, 3–14.

    Article  Google Scholar 

  69. Weinstein, C. E., Palmer, D. R., & Schulte, A. C. (1987). Learning and study strategies inventory. Clearwater, FL: H & H Publishing Company.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Ulrike-Marie Krause.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Stark, R., Krause, UM. Effects of reflection prompts on learning outcomes and learning behaviour in statistics education. Learning Environ Res 12, 209–223 (2009). https://doi.org/10.1007/s10984-009-9063-x

Download citation

Keywords

  • Example-based learning
  • Learning behaviour
  • Metacognition
  • Reflection prompts
  • Situated learning
  • Statistics education