Advertisement

Monitoring the Use of Learning Strategies in a Web-Based Pre-course in Mathematics

A Comparison of Quantitative and Qualitative Data Sources
  • Katja DerrEmail author
  • Reinhold Hübl
  • Mohammed Zaki Ahmed
Chapter

Abstract

With the increasing heterogeneity of first year students’ mathematics knowledge, preparatory courses are frequently used by universities to overcome large knowledge differences at the start of tertiary education. Collected at a very early stage, pre-course data could be valuable resources for learning analytics, but little is known about their informational value. This issue is explored based on quantitative and qualitative evaluations of data collected from a web-based pre-course in mathematics (demographic, test results, survey answers, log files, and interviews). The quantitative analyses revealed a dominant influence of cognitive variables, results in a diagnostic pre-test being the strongest determinant of first year mathematics achievement, which in turn was highly predictive of overall study success. Pre-course participation had a significant but relatively small moderating effect on this relation, suggesting that only students who actively participated in the course managed to improve their starting position at university. The study discusses the difficulties of collecting data from open web-based learning environments, from missing data to interactions between cognitive and metacognitive variables. It is argued that qualitative information strongly contributes to understanding the sometimes counterintuitive results of such analyses. Suggestions are made for the design of pre-courses that support “at-risk” students’ use of learning strategies.

Keywords

e-learning Evaluation Learning strategy Mathematics Predictors 

Notes

Acknowledgments

Support for this publication was provided by the German Federal Ministry of Education and Research (BMBF) in the context of the Federal “Quality pact for teaching” (ref. number 01PL17012). Responsibility for the content published in this article, including any opinions expressed therein, rests exclusively with the authors. Test items and learning materials are licensed under the Creative Commons Attribution 3.0 Unported and can be provided via www.optes.de.

References

  1. Abel, H., & Weber, B. (2014). 28 Jahre Esslinger Modell—Studienanfänger und Mathematik. In I. Bausch, R. Biehler, R. Bruder, P. R. Fischer, R. Hochmuth, W. Koepf, & T. Wassong (Eds.), Mathematische Vor- und Brückenkurse. Konzepte, Probleme und Perspektiven (pp. 9–19). Wiesbaden, Germany: Springer.Google Scholar
  2. Ackerman, P. L., Kanfer, R., & Beier, M. E. (2013). Trait complex, cognitive ability, and domain knowledge predictors of baccalaureate success, STEM persistence and gender differences. Journal of Educational Psychology, 105(3), 911–927.Google Scholar
  3. Armstrong, P. K., & Croft, A. C. (1999). Identifying the learning needs in mathematics of entrants to undergraduate engineering programmes in an English university. European Journal of Engineering Education, 24(1), 59–71.Google Scholar
  4. Artino, A. R., & Stephens, J. M. (2009). Academic motivation and self-regulation: A comparative analysis of undergraduate and graduate students learning online. Internet and Higher Education, 12, 146–151.Google Scholar
  5. Ashby, J., Sadera, W. A., & Mcnary, S. W. (2011). Comparing student success between developmental math courses offered online, blended, and face-to-face. Journal of Interactive Online Learning, 10(3), 128–140.Google Scholar
  6. Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educational Psychologist, 40(4), 199–209.Google Scholar
  7. Azevedo, R., & Cromley, J. G. (2004). Does training on self-regulated learning facilitate Students' learning with hypermedia? Journal of Educational Psychology, 96(3), 523–535.Google Scholar
  8. Ballard, C. L., & Johnson, M. (2004). Basic math skills and performance in an introductory economics class. Journal of Economic Education, 35(1), 3–23.Google Scholar
  9. Bargel, T. (2015). Studieneingangsphase und heterogene Studentenschaft—neue Angebote und ihr Nutzen: Befunde des 12. Studierendensurveys an Universitäten und Fachhochschulen (Hefte zur Bildungs- und Hochschulforschung No. 83).Google Scholar
  10. Barnard, L., Lan, W. Y., To, Y. M., Osland Paton, V., & Lai, S.-L. (2009). Measuring self-regulation in online and blended learning environments. Internet and Higher Education, 12, 1–6.Google Scholar
  11. Barnard-Brak, L., Lan, W. Y., & Paton, V. O. (2010). Profiles in self-regulated learning in the online learning environment. International review of research in open and distance. Learning, 11(1), 62–80.Google Scholar
  12. Bettinger, E. P., & Long, B. T. (2009). Addressing the needs of underprepared students in higher education: Does college remediation work? Journal of Human Resources, 44(3), 736–771.Google Scholar
  13. Biehler, R., Fischer, P. R., & Wassong, T. (2012). Designing and evaluating blended learning bridging courses in mathematics. In Congress of the European Society for Research in Mathematics Education (pp. 1971–1980).Google Scholar
  14. Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for learning: Putting it into practice. Maidenhead, England: Open University Press.Google Scholar
  15. Broadbent, J. (2017). Comparing online and blended learner's self-regulated learning strategies and academic performance. The Internet and Higher Education, 33, 24–32.Google Scholar
  16. Broadbent, J., & Poon, W. L. (2015). Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review. The Internet and Higher Education, 27, 1–13.Google Scholar
  17. Burks, R. (2010). The student mathematics portfolio: Value added to student preparation? PRIMUS, 20(5), 453–472.Google Scholar
  18. Carr, M., Bowe, B., & Ní Fhloinn, E. (2013). Core skills assessment to improve mathematical competency. European Journal of Engineering Education, 38(6), 608–619.Google Scholar
  19. Carson, A. D. (2011). Predicting student success from the Lassi for learning online (LLO). Journal of Educational Computing Research, 45(4), 399–414.Google Scholar
  20. Case, J. M. (2004). Approaches to learning: A critical examination of inventory responses from third year chemical engineering students. In A. Buffler, & R. C. Laugksch (Eds.), Proceedings of the 12th Annual Conference of the Southern African Association for Research in Mathematics, Science and Technology Education SAARMSTE (pp. 102–110). Retrieved from http://www.academia.edu
  21. Clark, M., & Lovric, M. (2009). Understanding secondary–tertiary transition in mathematics. International Journal of Mathematical Education in Science and Technology, 40(6), 755–776.Google Scholar
  22. Cook, C., Heath, F., & Thompson, R. L. (2000). A meta-analysis of response rates in web- or internet-based surveys. Educational and Psychological Measurement, 60(6), 821–836.Google Scholar
  23. Cosh cooperation schule:hochschule. (2014). Mindestanforderungskatalog Mathematik (2.0) der Hochschulen Baden-Württembergs für ein Studium von WiMINT-Fächern. Retrieved from http://www.mathematik-schule-hochschule.de/images/Aktuelles/pdf/MAKatalog_2_0.pdf
  24. Credé, M., & Phillips, L. A. (2011). A meta-analytic review of the motivated strategies for learning questionnaire. Learning and Individual Differences, 21(4), 337–346.Google Scholar
  25. Croft, A. C., Harrison, M. C., & Robinson, C. L. (2009). Recruitment and retention of students – An integrated and holistic vision of mathematics support. International Journal of Mathematical Education in Science and Technology, 40(1), 109–125.Google Scholar
  26. Derr, K., Hübl, R., & Ahmed, M. Z. (2015). Using test data for successive refinement of an online pre-course in mathematics. In 14th European Conference on e-Learning ECEL (pp. 173–180).Google Scholar
  27. Derr, K., Hübl, R., & Ahmed, M. Z. (2018). Prior knowledge in mathematics and study success in engineering: Informational value of learner data collected from a web-based pre-course. European Journal of Engineering Education, 10(3), 1–16.Google Scholar
  28. Di Pietro, G. (2012). The short-term effectiveness of a remedial mathematics course: Evidence from a UK university (Discussion Paper Series No. 6358). Bonn.Google Scholar
  29. Dreier, O. (2014). Entwicklung eines Moodle Plug-Ins zur erweiterten Auswertung von Tests: Bachelor thesis. Duale Hochschule Baden-Württemberg, Mannheim.Google Scholar
  30. Dweck, C. S. (1986). Motivational processes affecting learning. American Psychologist, 41(10), 1040–1048.Google Scholar
  31. Ecclestone, K., Biesta, G., & Hughes, M. (Eds.). (2010). Transitions and learning through the Lifecourse. London, UK: Routledge.Google Scholar
  32. Ehrenberg, R. G. (2010). Analyzing the factors that influence persistence rates in STEM field, majors: Introduction to the symposium. Economics of Education Review, 29, 888–891.Google Scholar
  33. Eley, M. G., & Meyer, J. H. F. (2004). Modelling the influences on learning outcomes of study processes in university mathematics. Higher Education, 47(4), 437–454.Google Scholar
  34. Entwistle, N. J., & McCune, V. (2004). The conceptual bases of study strategy inventories. Educational Psychology Review, 16(4), 325–345.Google Scholar
  35. Fan, W., & Yan, Z. (2010). Factors affecting response rates of the web survey: A systematic review. Computers in Human Behavior, 26, 132–139.Google Scholar
  36. Faulkner, F., Hannigan, A., & Gill, O. (2010). Trends in the mathematical competency of university entrants in Ireland by leaving certificate mathematics grade. Teaching Mathematics and Its Applications, 29(2), 76–93.Google Scholar
  37. Greefrath, G., Koepf, W., & Neugebauer, C. (2016). Is there a link between preparatory course attendance and academic success? A case study of degree Programmes in electrical engineering and computer science. International Journal of Research in Undergraduate Mathematics Education, 3(1), 143–167.Google Scholar
  38. Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Educational Technology & Society, 15(3), 42–57.Google Scholar
  39. Hadwin, A. F., Winne, P. H., & Nesbit, J. C. (2005). Roles for software technologies in advancing research and theory in educational psychology. British Journal of Educational Psychology, 75, 1–24.Google Scholar
  40. Hannafin, M. J., & Hannafin, K. M. (2010). Cognition and student-centered, web-based learning: Issues and implications for research and theory. In J. M. Spector, D. Ifenthaler, P. Isaias, Kinshuk, & D. Sampson (Eds.), Learning and instruction in the digital age (pp. 11–23). Boston, MA: Springer.Google Scholar
  41. Hattie, J. (2009). Visible learning. A synthesis of over 800 meta-analyses relating to achievement. London, UK: Routledge.Google Scholar
  42. Hell, B., Linsner, M., & Kurz, G. (2008). Prognose des Studienerfolgs. In M. Rentschler & H. P. Voss (Eds.), Studieneignung und Studierendenauswahl - Untersuchungen und Erfahrungsberichte (pp. 132–177). Aachen, Germany: Shaker.Google Scholar
  43. Heublein, U., Richter, J., Schmelzer, R., & Sommer, D. (2012). Die Entwicklung der Schwund- und Studienabbruchquoten an den deutschen Hochschulen: Statistische Berechnungen auf der Basis des Absolventenjahrgangs 2010. Projektbericht.Google Scholar
  44. Kadijevich, D. (2006). Developing trustworthy timss background measures: A case study on mathematics attitude. The Teaching of Mathematics, 17, 41–51.Google Scholar
  45. Karabenick, S. A. (2004). Perceived achievement goal structure and college student help seeking. Journal of Educational Psychology, 96(3), 569–581.Google Scholar
  46. Kift, S., Nelson, K., & Clarke, J. (2010). Transition pedagogy: A third generation approach to FYE - a case study of policy and practice for the higher education sector. The International Journal of the First Year in Higher Education, 1(1), 1–20.Google Scholar
  47. Knospe, H. (2011). Der Eingangstest Mathematik an Fachhochschulen in Nordrhein-Westfalen von 2002 bis 2010, In Proceedings des 9. Workshops Mathematik für ingenieurwissenschaftliche Studiengänge. Wismarer Frege-Reihe (Vol. 2, pp. 8–13).Google Scholar
  48. Krumke, S. O., Roegner, K., Schüler, L., Seiler, R., & Stens, R. (2012). Der Online-Mathematik Brückenkurs OMB. Eine Chance zur Lösung der Probleme an der Schnittstelle Schule/Hochschule: DMV Mitteilungen Juni 2012. Retrieved from http://page.math.tu-berlin.de/~seiler/publications/OMB-eine-Chance.pdf
  49. Lagerlöf, J. N. M., & Seltzer, A. J. (2009). The effects of remedial mathematics on the learning of economics: Evidence from a natural experiment. The Journal of Economic Education, 40(2), 115–137.Google Scholar
  50. Ledermüller, K., & Fallmann, I. (2017). Predicting learning success in online learning environments: Self-regulated learning, prior knowledge and repetition. Zeitschrift Für Hochschulentwicklung, 12(1), 79–99.Google Scholar
  51. Liston, M., & O'Donoghue, J. (2009). Factors influencing the transition to university service mathematics: Part 1 a quantitive study. Mathematics and Its Applications, 28, 77–87.Google Scholar
  52. Luk, H. S. (2005). The gap between secondary school and university mathematics. International Journal of Mathematical Education in Science and Technology, 36(2–3), 161–174.Google Scholar
  53. Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an ‘early warning system' for educators: A proof of concept. Computers & Education, 54(2), 588–599.Google Scholar
  54. Martin, P.-Y. (2012). Lernstrategien und Umgang mit ICT von Studienanfängerinnen und -anfängern. Zurich, Switzerland: Universität Zürich.Google Scholar
  55. MAXQDA. Berlin, Germany: VERBI Software Consult Sozialforschung GmbH.Google Scholar
  56. McDonald, B. (2012). Portfolio assessment: Direct from the classroom. Assessment & Evaluation in Higher Education, 37(3), 335–347.Google Scholar
  57. Meyer, J. H. F. (2000). Variation in contrasting forms of 'memorising' and associated observables. British Journal of Educational Psychology, 70(2), 163–176.Google Scholar
  58. Meyer, J. H. F., & Eley, M. G. (1999). The development of affective subscales to reflect variation in students’ experiences of studying mathematics in higher education. Higher Education, 37, 197–216.Google Scholar
  59. Michinov, N., Brunot, S., Le Bohec, O., Juhel, J., & Delaval, M. (2011). Procrastination, participation, and performance in online learning environments. Computers & Education, 56(1), 243–252.Google Scholar
  60. Morris, L. V., Finnegan, C., & Wu, S.-S. (2005). Tracking student behavior, persistence, and achievement in online courses. The Internet and Higher Education, 8(3), 221–231.  https://doi.org/10.1016/j.iheduc.2005.06.009 Google Scholar
  61. Moss, B. G., & Yeaton, W. H. (2006). Shaping policies related to developmental education: An evaluation using the regression-discontinuity design. Educational Evaluation and Policy Analysis, 28(3), 215–229.Google Scholar
  62. Mullis, I. V. S., Martin, M. O., Foy, P., & Arora, A. (2012). TIMSS 2011 International Results in Mathematics.Google Scholar
  63. Newman, R. S. (2002). How self-regulated learners cope with academic difficulty: The role of adaptive help seeking. Theory Into Practice, 41(2), 132–138.Google Scholar
  64. Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: What can be done? Assessment & Evaluation in Higher Education, 33(3), 301–314.Google Scholar
  65. Pardo, A., & Kloos, C. D. (2011). Stepping out of the box: Towards analytics outside the learning management system. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge (pp. 163–167).Google Scholar
  66. Parsons, S. J., Croft, T., & Harrison, M. (2009). Does students' confidence in their ability in mathematics matter? Teaching Mathematics and Its Applications, 28(2), 52–68.Google Scholar
  67. Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor, MI: University of Michigan.Google Scholar
  68. Plant, E. A., Ericsson, K. A., Hill, L., & Asberg, K. (2005). Why study time does not predict grade point average across college students: Implications of deliberate practice for academic performance. Contemporary Educational Psychology, 30(1), 96–116.Google Scholar
  69. Polaczek, C., & Henn, G. (2008). Vergleichende Auswertung des Mathematik-Eingangstests. Retrieved from http://www.fh-aachen.de/uploads/media/Eingangstest_WS_07_08.pdf
  70. Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university Students' academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138(2), 353–387.Google Scholar
  71. Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130(2), 261–288.  https://doi.org/10.1037/0033-2909.130.2.261 Google Scholar
  72. Robinson, C. L., & Croft, A. C. (2003). Engineering students—diagnostic testing and follow up. Teaching Mathematics and Its Applications, 22(4), 177–181.Google Scholar
  73. Robson, C. (2011). Real world research: A resource for users of social research methods in applied settings (3rd ed.). Chichester, UK: Wiley.Google Scholar
  74. Samson, P. J. (2015). Can student engagement be measured? And, if so, does it matter? In Frontiers in Education Conference Conference Proceedings.Google Scholar
  75. Schiefele, U., & Wild, K. P. (1994). Lernstrategien im Studium: Ergebnisse zur Faktorenstruktur und Reliabilität eines neuen Fragebogens. Zeitschrift Für Differentielle Und Diagnostische Psychologie, 15(4), 185–200.Google Scholar
  76. Scholes, V. (2016). The ethics of using learning analytics to categorize students on risk. Educational Technology Research and Development, 64(5), 939–955.Google Scholar
  77. Schumacher, C., & Ifenthaler, D. (2018). Features students really expect from learning analytics. Computers in Human Behavior, 47, 394–407.Google Scholar
  78. SEFI Mathematics Working Group. (2013). A framework for mathematics curricula in engineering education. Brussels, Belgium: Author.Google Scholar
  79. Smith, G. G., & Ferguson, D. (2005). Student attrition in mathematics e-learning. Australasian Journal of Educational Technology, 21(3), 323–334. Retrieved from https://ajet.org.au/index.php/AJET/article/view/1323
  80. Söderlind, J., & Geschwind, L. (2017). More students of better quality? Effects of a mathematics and physics aptitude test on student performance. European Journal of Engineering Education, 42(4), 445–457.Google Scholar
  81. Spector, J. M., Ifenthaler, D., Sampson, D. G., & Yang, L. (2016). Technology enhanced formative assessment for 21st century learning. Journal of Educational Technology & Society, 19(3), 58–71.Google Scholar
  82. SPSS Version 23. Armonk, NY: IBM Corp.Google Scholar
  83. Stake, R. E. (1994). Case Studies. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 236–247). Thousand Oaks, CA: Sage.Google Scholar
  84. Street, H. (2010). Factors influencing a learner’s decision to drop-out or persist in higher education distance learning. Online Journal of Distance Learning Administration, 13(4), 4. Retrieved from http://www.westga.edu/~distance/ojdla/winter134/street134.html
  85. Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior, 47, 157–167.Google Scholar
  86. Thiessen, V., & Blasius, J. (2008). Mathematics achievement and mathematics learning strategies: Cognitive competencies and construct differentiation. International Journal of Educational Research, 47(6), 362–371.Google Scholar
  87. Tourangeau, R., Conrad, F. G., & Couper, M. P. (2013). The science of web surveys. Oxford, UK: Oxford University Press.Google Scholar
  88. Vuik, K., Daalderop, F., Daudt, J., & van Kints, R. (2012). Evaluation MUMIE—Online math education: aerospace engineering and computer science 2011–2012. Reports of the Delft Institute of Applied Mathematics, 12–13.Google Scholar
  89. Weinstein, C. E., Zimmermann, S. A., & Palmer, D. R. (1988). Assessing learning strategies. The design and development of the LASSI. In C. E. Weinstein, E. T. Goetz, & P. A. Alexander (Eds.), Learning and study strategies. Issues in assessment, instruction, and evaluation (pp. 25–40). San Diego, CA: Academic Press.Google Scholar
  90. Winne, P. H. (2004). Students’ calibration of knowledge and learning processes: Implications for designing powerful software learning environments. International Journal of Educational Research, 41(6), 466–488.Google Scholar
  91. Winne, P. H., & Jamieson-Noel, D. (2002). Exploring students’ calibration of self reports about study tactics and achievement. Contemporary Educational Psychology, 27, 551–572.Google Scholar
  92. Yin, R. K. (2009). Case study research. Design and methods. Thousand Oaks, CA: Sage.Google Scholar
  93. Zacharis, N. Z. (2015). A multivariate approach to predicting student outcomes in web-enabled blended learning courses. Internet and Higher Education, 27, 44–53.Google Scholar
  94. Zhang, G., Anderson, T. J., Ohland, M. W., & Thorndyke, B. R. (2004). Identifying factors influencing engineering student graduation: A longitudinal and crossinstitutional study. Journal of Engineering Education, 93, 313–320.Google Scholar
  95. Zimmerman, B. J., & Moylan, A. (2009). Self-regulation: Where metacognition and motivation intersect. In D. J. Hacker (Ed.), Handbook of metacognition in education (pp. 299–316). New York, NY: Routledge.Google Scholar
  96. Zimmerman, B. J., Moylan, A., Hudesman, J., White, N., & Flugman, B. (2011). Enhancing self-reflection and mathematics achievement of at-risk urban technical college students. Psychological Test and Assessment Modeling, 53(1), 141–160.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Katja Derr
    • 1
    Email author
  • Reinhold Hübl
    • 1
  • Mohammed Zaki Ahmed
    • 2
  1. 1.DHBWMannheimGermany
  2. 2.Plymouth UniversityPlymouthUK

Personalised recommendations