Learning Environments Research

, Volume 12, Issue 3, pp 209–223 | Cite as

Effects of reflection prompts on learning outcomes and learning behaviour in statistics education

  • Robin Stark
  • Ulrike-Marie KrauseEmail author
Original Paper


Starting from difficulties that students display when they deal with correlation analysis, an e-learning environment (‘Koralle’) was developed. The design was inspired by principles of situated and example-based learning. In order to facilitate reflective processes and thus enhance learning outcomes, reflection prompts were integrated into the learning environment. A total of 57 university students were randomly assigned to two experimental conditions: 28 students were prompted to give reasons for their decisions while working within the learning environment (EG 1); and 29 students dealt with Koralle without being prompted (EG 2). The control group consisted of 67 students who had already attended regular statistics lectures but had no access to the e-learning environment. EG 1 scored significantly higher in the posttest than EG 2, and the effect was practically relevant and sustainable. Reflection prompts did not influence time on task, task choices and motivational outcomes. Both experimental groups clearly outperformed the control group.


Example-based learning Learning behaviour Metacognition Reflection prompts Situated learning Statistics education 


  1. Alexander, P. A. (1996). The past, present, and future of knowledge research: A re-examination of the role of knowledge in learning and instruction. Educational Psychologist, 3, 89–92.Google Scholar
  2. Allwood, C. M. (1990). On the relation between justification of solution method and correctness of solution in statistical problem solving. Scandinavian Journal of Psychology, 31, 181–190.CrossRefGoogle Scholar
  3. Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. W. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70, 181–214.Google Scholar
  4. Atkinson, R. K., Renkl, A., & Merrill, M. M. (2003). Transitioning from studying examples to solving problems: Combining fading with prompting fosters learning. Journal of Educational Psychology, 95, 774–783.CrossRefGoogle Scholar
  5. Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educational Psychologist (Special Issue on Computers as Metacognitive Tools for Enhancing Student Learning), 40, 199–209.Google Scholar
  6. Bannert, M. (2003). Effekte metakognitiver Lernhilfen auf den Wissenserwerb in vernetzten Lernumgebungen [Effects of metacognitive learning aids on knowledge acquisition in networked learning environments]. Zeitschrift für Pädagogische Psychologie, 17, 13–25.CrossRefGoogle Scholar
  7. Berthold, K., Nückles, M., & Renkl, A. (2007). Do learning protocols support learning strategies and outcomes? The role of cognitive and metacognitive prompts. Learning and Instruction, 17, 564–577.CrossRefGoogle Scholar
  8. Broers, N. J. (2002). Selection and use of propositional knowledge in statistical problem solving. Learning and Instruction, 12, 323–344.CrossRefGoogle Scholar
  9. Brown, A., & DeLoache, J. S. (1978). Skills, plans and self-regulation. In R. Siegler (Ed.), Children’s thinking: What develops? (pp. 3–35). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  10. Castro Sotos, A. E., Vanhoof, S., Van den Noortgate, W., & Onghena, P. (2007). Students’ misconceptions of statistical inference: A review of the empirical evidence from research on statistics education. Educational Research Review, 2, 98–113.CrossRefGoogle Scholar
  11. Chi, M. T. H. (1996). Constructing self-explanations and scaffolded explanations in tutoring. Applied Cognitive Psychology, 10, S33–S49.CrossRefGoogle Scholar
  12. Chi, M. T. H., Bassok, M., Lewis, M., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145–182.CrossRefGoogle Scholar
  13. Chi, M. T. H., DeLeeuw, N., Chiu, M. H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439–477.CrossRefGoogle Scholar
  14. Cognition and Technology Group at Vanderbilt. (1997). The Jasper project: Lessons in curriculum, instruction, assessment and professional development. Mahwah, NJ: Erlbaum.Google Scholar
  15. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.Google Scholar
  16. Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997–1003.CrossRefGoogle Scholar
  17. Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing and mathematics. In L. B. Resnick (Ed.), Knowing, learning, and instruction (pp. 453–494). Hillsdale, NJ: Erlbaum.Google Scholar
  18. De Jong, T., & Ferguson-Hessler, M. G. M. (1996). Types and qualities of knowledge. Educational Psychologist, 31, 105–113.CrossRefGoogle Scholar
  19. Deci, E. L., & Ryan, R. M. (2000). The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11, 227–268.CrossRefGoogle Scholar
  20. Dillon, A., & Gabbard, R. (1998). Hypermedia as an educational technology: A review of the quantitative research literature on learner comprehension, control, and style. Review of Educational Research, 68, 322–349.Google Scholar
  21. Dochy, F. J. R. C. (1992). Assessment of prior knowledge as a determinant for future learning. The use of prior knowledge state tests and knowledge profiles. Utrecht, The Netherlands: Uitgeverij Lemma B. V.Google Scholar
  22. Ericsson, K. A., & Simon, H. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.Google Scholar
  23. Flavell, J. (1976). Metacognitive aspects of problem solving. In L. B. Resnick (Ed.), The nature of intelligence (pp. 231–235). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  24. Flavell, J. (1979). Metacognition and cognitive monitoring: A new area of cognitive developmental inquiry. American Psychologist, 34, 906–911.CrossRefGoogle Scholar
  25. Gerjets, P., Scheiter, K., & Catrambone, R. (2004). Designing instructional examples to reduce intrinsic cognitive load: Molar versus modular presentation of solution procedures. Instructional Science, 32, 33–58.CrossRefGoogle Scholar
  26. Gerjets, P., Scheiter, K., & Schuh, J. (2005). Instruktionale Unterstützung beim Fertigkeitserwerb aus Beispielen in hypertextbasierten Lernumgebungen [Instructional support for skill acquisition from examples in hypermedia-based learning environments]. Zeitschrift für Pädagogische Psychologie, 19(1/2), 23–38.CrossRefGoogle Scholar
  27. Große, C. S., & Renkl, A. (2006). Effects of multiple solution methods in mathematics learning. Learning and Instruction, 16, 122–138.CrossRefGoogle Scholar
  28. Hasselhorn, M. (2006). Metakognition [Metacognition]. In D. H. Rost (Ed.), Handwörterbuch Pädagogische Psychologie [Handbook of educational psychology] (pp. 480–485). Weinheim, Germany: Psychologie Verlags Union.Google Scholar
  29. Hasselhorn, M., & Hager, W. (1998). Kognitive Trainings auf dem Prüfstand: Welche Komponenten charakterisieren erfolgreiche Fördermaßnahmen? [Cognitive trainings put to test: What components characterise successful facilitation methods?]. In M. Beck (Ed.), Evaluation als Maßnahme der Qualitätssicherung: Pädagogisch-psychologische Interventionen auf dem Prüfstand [Evaluation as a means of quality management: Pedagogical-psychological interventions put to test] (pp. 85–98). Tübingen, Germany: DGVT-Verlag.Google Scholar
  30. Hidi, S. (1990). Interest and its contribution as a mental resource for learning. Review of Educational Research, 60(4), 549–571.Google Scholar
  31. Kintsch, W. (1994). Text comprehension, memory, and learning. American Psychologist, 49, 294–303.CrossRefGoogle Scholar
  32. Kopp, V., Stark, R., & Fischer, M. R. (2007). Förderung von Diagnosekompetenz in der medizinischen Ausbildung durch Implementation eines Ansatzes zum fallbasierten Lernen aus Lösungsbeispielen [Enhancing diagnostic skills in medical training by case-based learning from worked examples]. GMS Zeitschrift für Medizinische Ausbildung, 24(2): Doc 107.Google Scholar
  33. Krapp, A. (2000). Individuelle Interessen als Bedingung lebenslangen Lernens [Individual interests as a condition for life-long learning]. In F. Achtenhagen & W. Lempert (Eds.), Lebenslanges Lernen im Beruf—seine Grundlegung im Kindes-und Jugendalter, Band 3: Psychologische Theorie, Empirie und Therapie [Professional lifelong learning—its foundation in childhood and adolescence, Volume 3: Psychological theory, empiricism and therapy] (pp. 54–75). Opladen, Germany: Leske + Budrich.Google Scholar
  34. Krapp, A. (2005). Basic needs and the development of interest and intrinsic motivational orientations. Learning and Instruction, 15, 381–395.CrossRefGoogle Scholar
  35. Krause, U.-M. (2007). Feedback und kooperatives Lernen [Feedback and cooperative learning]. Münster, Germany: Waxmann.Google Scholar
  36. Krause, U.-M., & Stark, R. (2006). Vorwissen aktivieren [Activating prior knowledge]. In H. Mandl & H. F. Friedrich (Eds.), Handbuch Lernstrategien [Handbook of learning strategies] (pp. 38–49). Göttingen, Germany: Hogrefe.Google Scholar
  37. Krause, U.-M., & Stark, R. (2007, August). Problem-oriented learning and reflection prompts in teacher education: Enhancing knowledge about cooperative learning. Paper presented at the 12th Biennial Conference of the European Association for Research on Learning and Instruction (EARLI), Budapest, Hungary.Google Scholar
  38. Krause, U.-M., Stark, R., & Mandl, H. (2009). The effects of cooperative learning and feedback on e-learning in statistics. Learning and Instruction, 19, 158–170.CrossRefGoogle Scholar
  39. Lan, W. Y., Bradley, L., & Parr, G. (1993). The effects of a self-monitoring process on college students’ learning in an introductory statistics course. The Journal of Experimental Education, 62, 26–40.Google Scholar
  40. Langer, E. J. (1993). A mindful education. Educational Psychologist, 28, 43–50.CrossRefGoogle Scholar
  41. Mandl, H., & Gräsel, C. (1997). Gestaltung konstruktivistischer Lernumgebungen in der Medizin [Designing constructivist learning environments in medical science]. In S. Höfling & H. Mandl (Eds.), Lernen für die Zukunft—Lernen in der Zukunft: Wissensmanagement in der Bildung [Learning for the future—learning in the future: Knowledge management in education] (pp. 155–165). München, Germany: Hanns-Seidel-Stiftung e.V.Google Scholar
  42. Marsh, H. W., & Craven, R. G. (2006). Reciprocal effects of self-concept and performance from a multidimensional perspective. Perspectives on Psychological Science, 1, 133–163.CrossRefGoogle Scholar
  43. McClelland, D. J., Atkinson, J. W., Clark, R. H., & Lowell, E. L. (1953). The achievement motive. New York: Appleton-Century-Crofts.CrossRefGoogle Scholar
  44. Morris, E. (2001). The design and evaluation of link: A computer-based learning system for correlation. British Journal of Educational Technology, 32, 39–52.CrossRefGoogle Scholar
  45. Paas, F. G. W. C. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84, 429–434.CrossRefGoogle Scholar
  46. Paris, S. G., Lipson, M. Y., & Wixon, K. K. (1983). Becoming a strategic reader. Contemporary Educational Psychology, 8, 293–316.CrossRefGoogle Scholar
  47. Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the motivated strategies for learning questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801–813.CrossRefGoogle Scholar
  48. Pirolli, P. L., & Anderson, J. R. (1985). The role of practice in fact retrieval. Journal of Experimental Psychology. Learning, Memory, and Cognition, 11, 136–153.CrossRefGoogle Scholar
  49. Reigeluth, C. M., & Stein, F. S. (1983). The elaboration theory of instruction. In C. M. Reigeluth (Ed.), Instructional-design theories and models: An overview of their current status (pp. 335–382). Hillsdale, NJ: Erlbaum.Google Scholar
  50. Renkl, A. (1997). Learning from worked-out examples: A study on individual differences. Cognitive Science, 21, 1–29.CrossRefGoogle Scholar
  51. Renkl, A., Atkinson, R. K., & Große, C. S. (2004). How fading worked solution steps works—a cognitive load perspective. Instructional Science, 32, 59–82.CrossRefGoogle Scholar
  52. Salomon, G., & Globerson, T. (1987). Skill may not be enough: The role of mindfulness in learning transfer. International Journal of Educational Research, 11, 623–637.CrossRefGoogle Scholar
  53. Schiefele, U., Krapp, A., & Schreyer, I. (1993). Metaanalyse des Zusammenhangs von Interesse und schulischer Leistung [Meta-analysis of the correlation between interest and academic performance]. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 25, 120–148.Google Scholar
  54. Schneider, W., & Pressley, M. (1989). Memory development between 2 and 20. New York: Springer.Google Scholar
  55. Schoenfeld, A. H. (1985). Mathematical problem solving. New York: Academic Press.Google Scholar
  56. Spiro, R., Feltovich, P. J., Jacobson, M. J., & Coulson, R. L. (1991). Cognitive flexibility, constructivism, and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. Educational Technology, 31(5), 24–33.Google Scholar
  57. Stark, R. (1999). Lernen mit Lösungsbeispielen: Einfluss unvollständiger Lösungsbeispiele auf Beispielelaboration, Lernerfolg und Motivation [Learning with worked-out examples: The impact of incomplete examples on example elaboration, learning outcomes and motivation]. Göttingen, Germany: Hogrefe.Google Scholar
  58. Stark, R. (2004). Implementing example-based learning and teaching in the context of vocational school education in business administration. Learning Environments Research, 7, 143–163.CrossRefGoogle Scholar
  59. Stark, R., Gruber, H., Renkl, A., & Mandl, H. (2000). Instruktionale Effekte einer kombinierten Lernmethode: Zahlt sich die Kombination von Lösungsbeispielen und Problemlöseaufgaben aus? [Instructional effects of a combined teaching approach: Does the combination of worked-out examples and problem-solving tasks pay off?]. Zeitschrift für Pädagogische Psychologie, 14, 205–217.CrossRefGoogle Scholar
  60. Stark, R., & Mandl, H. (2000). Training in empirical research methods: Analysis of problems and intervention from a motivational perspective. In J. Heckhausen (Ed.), Motivational psychology of human development (pp. 165–183). Amsterdam, The Netherlands: Elsevier.CrossRefGoogle Scholar
  61. Stark, R., & Mandl, H. (2002). Konzeptualisierung und Evaluation einer komplexen netzbasierten Lernumgebung im Kontext der universitären Ausbildung in empirischen Forschungsmethoden [Conceptualisation and evaluation of a complex net-based learning environment in university training of empirical research methods]. Unterrichtswissenschaft, 30, 315–330.Google Scholar
  62. Stark, R., & Mandl, H. (2005). Lernen mit einer netzbasierten Lernumgebung im Bereich empirischer Forschungsmethoden: Effekte zusätzlich implementierter Maßnahmen und Bedeutung von Lernvoraussetzungen [Learning within a net-based learning environment in the field of empirical research methods: Effects of additional interventions and impact of learning prerequisites]. Unterrichtswissenschaft, 33(1), 3–29.Google Scholar
  63. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.CrossRefGoogle Scholar
  64. Sweller, J., Van Merriёnboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–296.CrossRefGoogle Scholar
  65. Tyroller, M. (2005). Effekte metakognitiver Prompts beim computerbasierten Statistiklernen [Effects of metacognitive prompts in computer-based statistics learning]. Retrieved July 21, 2009, from
  66. Van Merriënboer, J. J. G. (1990). Strategies for programming instruction in high school: Program completion vs. program generation. Journal of Computing Research, 6, 265–285.Google Scholar
  67. Van Merriënboer, J. J. G., & de Croock, M. B. M. (1992). Strategies for computer-based programming instruction: Program completion vs. program generation. Journal of Educational Computing Research, 8, 365–394.Google Scholar
  68. Veenman, M. V. J., Van Hout-Wolters, B. H. A. M., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1, 3–14.CrossRefGoogle Scholar
  69. Weinstein, C. E., Palmer, D. R., & Schulte, A. C. (1987). Learning and study strategies inventory. Clearwater, FL: H & H Publishing Company.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  1. 1.Institute of EducationSaarland UniversitySaarbrückenGermany

Personalised recommendations