Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

An exploratory quantitative case study of critical thinking development through adult distance learning


Critical thinking is a metacognitive process that, through purposeful, self-regulatory reflective judgment; skills of analysis, evaluation and inference; and a disposition towards thinking, increases the chances of producing a logical conclusion to an argument or solution to a problem. Critical thinking is vital for not only educational achievement, but also continuous professional development, as it is necessary in social and interpersonal contexts, where adequate decision-making and problem-solving are necessary on a daily basis—which is of particular relevance to adult distance learners, many of whom return to learning to further their professional development. Though a large body of extant research focuses on traditional BA student’s critical thinking performance, there is a dearth of research conducted on more mature adults’ critical thinking. ADL provides a unique opportunity to explore these abilities and the effects of critical thinking instruction through distance learning on such performance. Given the potential benefits linked with critical thinking and associated with adult distance learning, the aim of the current, retrospective, exploratory case study was to examine the effects of an adult distance learning critical thinking module, taught through a BA Training and Education programme, on critical thinking performance. A series of six paired samples t-tests were conducted in order to assess the performance of 95 ADLs from pre-to-post-testing on overall critical thinking performance and the critical thinking sub-scales of hypothesis-testing; verbal reasoning; argument analysis; judging likelihood an uncertainty; and problem-solving. Correlational analysis was also conducted. Results revealed a significant increase from pre-to-post-intervention on overall critical thinking performance, as well as all critical thinking sub-scale performances. However, there were no effects of active engagement with the module, disposition towards thinking or motivation towards learning on critical thinking performance. Results are discussed in light of theory and research on critical thinking.

This is a preview of subscription content, log in to check access.

Fig. 1


  1. 1.

    Argument mapping visually represents arguments through a ‘box-and-arrow’ style flow-chart, wherein the boxes are used to highlight propositions and the arrows are used to highlight the inferential relationships that link the propositions together (van Gelder 2003). Research has shown that argument mapping can facilitate metacognitive acts of critical thinking, both by making the structure of the argument open to deliberation and assessment; and by revealing strengths and weaknesses in the credibility, relevance, and logical soundness within argument structures (e.g. Butchart et al. 2009; Dwyer et al. 2011, 2012; van Gelder 2001). Thus, the CT intervention in the current case study utilised argument mapping to represent example arguments and facilitate the creation of student answers/arguments.

  2. 2.

    Results of the current case study indicated that the CT intervention had a beneficial effect on CT performance. However, despite the exploratory and retrospective nature of this case study, it is acknowledged that without a comparison group, these results must be discussed with added caution. In order to further elucidate the potential benefits of this CT intervention in an ADL context, access to data from a comparison and control group was obtained through randomly allocating another undergraduate cohort (i.e. students enrolled in a traditional BA programme) to either the same CT intervention (N = 42; 22 females, 20 males) or allocating them to a control condition (i.e. N = 28; 25 females, 7 males); thus, allowing for the comparison of the Adult Distance Learning CT module with a traditional cohort of students. Specifically, to ensure comparability, the experimental cohorts: consisted of undergraduate BA students, completed the same CT intervention through e-learning; and completed the same outcomes measures. Whereas the ADL group were older, full-time employed individuals who were completing the entirety of their Training and Education BA degree through distance learning, the comparison and control groups consisted of younger (M = 18.96 years) students enrolled in a full-time general BA, where classes were physically attended (with the exception of this voluntary CT module, which was also completed through e-learning, for which students received academic course credit for their participation as part of their psychology course). The only other minor difference between experimental groups was that whereas the ADL group completed a graded assignment in Week 8, traditional BAs did not, given that participation was voluntary. However, all groups would have completed multiple graded assignments as part of their respective courses. Notably, whereas traditional students significantly outperformed ADLs at baseline (d = .77), ADLs significantly outperformed both groups from time 1 to 2, as reflected in the reported effects sizes.


  1. Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., et al. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research,78(4), 1102–1134.

  2. Agarwal, R., & Day, A. E. (1998). The impact of the Internet on economic education. Journal of Economic Education,29, 99–110.

  3. Association of American Colleges & Universities. (2005). Liberal education outcomes: A preliminary report on student achievement in college. Washington, DC: AAC&U.

  4. Baddeley, A. D. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Sciences,4, 417–423.

  5. Barker, K., & Wendel, T. (2001). E-learning: Studying Canada’s virtual secondary schools. Kelowna, BC: Society for the Advancement of Excellence in Education. Retrieved April 11, 2004, from http://www.excellenceineducation.ca/pdfs/006.pdf.

  6. Brabeck, M. M. (1981). The relationship between critical thinking skills and development of reflective judgment among adolescent and adult women. Paper presented at the 89th Annual Convention of the American Psychological Association, Los Angeles, 24–26 August 1981.

  7. Bransford, J., Brown, A., & Cocking, R. (Eds.). (1999). How people learn. Washington, DC: National Academies Press.

  8. Brown, K. G. (2001). Using computers to deliver training: Why employees learn and why? Personnel Psychology,54, 271–296.

  9. Butchart, S., Bigelow, J., Oppy, G., Korb, K., & Gold, I. (2009). Improving critical thinking using web-based argument mapping exercises with automated feedback. Australasian Journal of Educational Technology,25(2), 268–291.

  10. Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment,48, 306–307.

  11. Cavanaugh, C., Gillan, K. J., Kromrey, J., Hess, M., & Blomeyer, R. (2004). The effects of distance education on K-12 student outcomes: A meta-analysis. Learning Point Associates/North Central Regional Educational Laboratory (NCREL).

  12. Chen, C. M. (2009). Personalized e-learning system with self-regulated learning assisted mechanisms for promoting learning performance. Expert Systems with Applications,36(5), 8816–8829.

  13. Clark, R. C. (2005). Multimedia learning in e-courses. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 589–615). New York: Cambridge University Press.

  14. Cowan, N. (2000). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral & Brain Sciences,24, 87–185.

  15. Dawson, T. L. (2008). Metacognition and learning in adulthood. Northhampton, MA: Developmental Testing Service LLC.

  16. Dawson-Tunik, T. L., Commons, M. L., Wilson, M., & Fischer, K. W. (2005). The shape of development. International Journal of Cognitive Development,2, 163–196.

  17. Dwyer, C. P. (2017). Critical thinking: Conceptual perspectives and practical guidelines. Cambridge, UK: Cambridge University Press.

  18. Dwyer, C. P., Hogan, M. J., Harney, O. M., & Kavanagh, C. (2016). Facilitating a student-educator conceptual model of dispositions towards critical thinking through interactive management. Educational Technology Research and Development,65, 47–73.

  19. Dwyer, C. P., Hogan, M. J., Harney, O. M., & O’Reilly, J. (2014a). Using interactive management to define and cultivate critical thinking competencies. Educational Technology Research & Development,62, 687–709.

  20. Dwyer, C. P., Hogan, M. J., & Stewart, I. (2011). The promotion of critical thinking skills through argument mapping. In C. P. Horvart & J. M. Forte (Eds.), Critical Thinking (pp. 97–122). New York: Nova Science Publishers.

  21. Dwyer, C. P., Hogan, M. J., & Stewart, I. (2012). An evaluation of argument mapping as a method of enhancing critical thinking performance in e-learning environments. Metacognition and Learning,7, 219–244.

  22. Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014b). An integrated critical thinking framework for the 21st century. Thinking Skills & Creativity,12, 43–52.

  23. Dwyer, C. P., Hogan, M. J., & Stewart, I. (2015a). An evaluation of critical thinking competencies in business settings. Journal of Education for Business,90(5), 260–269.

  24. Dwyer, C. P., Hogan, M. J., & Stewart, I. (2015b). The evaluation of argument mapping-infused critical thinking instruction as a method of enhancing reflective judgment performance. Thinking Skills & Creativity,16, 11–26.

  25. Facione, P. A. (1990). The Delphi report: Committee on pre-college philosophy. Millbrae, CA: California Academic Press.

  26. Gadzella, B. M., Ginther, D. W., & Bryant, G. W. (1996). Teaching and learning critical thinking skills. Paper presented at the 26thInternational Congress of Psychology, Montreal, August 19.

  27. Garcia, T., Pintrich, P. R., & Paul, R. (1992). Critical thinking and its relationship to motivation, learning strategies and classroom experience. Paper presented at the 100th annual meeting of the American Psychological Association, Washington, DC, 14–18 August 1992.

  28. Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and higher education,7(2), 95–105.

  29. Hake, R. (1998). Interactive-engagement vs. traditional methods: A six-thousand student survey of mechanics test data for introductory physics courses. American Journal of Physics,66(1), 64–74.

  30. Halpern, D. F. (2010). The Halpern critical thinking assessment: Manual. Vienna: Schuhfried.

  31. Halpern, D. F. (2014). Thought & knowledge: An introduction to critical thinking (5th ed.). UK: Psychology Press.

  32. Hitchcock, D. (2004). The effectiveness of computer-assisted instruction in critical thinking. Informal Logic,24(3), 183–218.

  33. Hogan, M. J., Dwyer, C. P., Noone, C., Harney, O., & Conway, R. (2014). Metacognitive skill development and applied systems science: A framework of metacognitive skills, self-regulatory functions and real-world applications. In Metacognition: Fundaments, applications, and trends, (pp. 75–106. Berlin: Springer.

  34. Huffaker, D. A., & Calvert, S. L. (2003). The new science of learning: Active learning, metacognition and transfer of knowledge in e-learning applications. Journal of Educational Computing Research,29(3), 325–334.

  35. Hugenholtz, N. I. R., de Croon, E. M., Smits, P. B., van Dijk, F. J. H., & Nieuwenhuijsen, K. (2008). Effectiveness of e-learning in continuing medical education for occupational physicians. Occupational Medicine,58, 370–372.

  36. Johnson, C. E., Hurtubise, L. C., Castrop, J., French, G., Groner, J., Ladinsky, M., et al. (2004). Learning management systems: Technology to measure the medical knowledge competency of the ACGME. Medical Education,38, 599–608.

  37. Kahneman, D. (2011). Thinking fast and slow. London: Penguin.

  38. Keegan, D. (1996). Foundations of distance education. London: Routledge.

  39. King, P. M., & Kitchener, K. S. (1994). Developing reflective judgment: Understanding and promoting intellectual growth and critical thinking in adolescents and adults. San Francisco: Jossey Bass.

  40. Kitchener, K. S., & King, P. M. (1981). Reflective judgment: Concepts of justifications and their relation to age and gender. Journal of Applied Developmental Psychology,2(2), 89–116.

  41. Knowles, M. S. S., Holton, E. F., & Swanson, R. A. (Eds.). (2005). The adult learner: The definitive classic in adult education and human resource development (6th ed.). Amsterdam: Elsevier Butterworth Heinemann.

  42. Ku, K. Y. L. (2009). Assessing students’ critical thinking performance: Urging for measurements using multi-response format. Thinking Skills and Creativity,4(1), 70–76.

  43. Kuhn, D. (1991). The skills of argument. Cambridge: Cambridge University Press.

  44. Laurillard, D. (2010). Teaching as a design science: Building pedagogical patterns for learning and technology (3rd ed.). New York: Taylor & Francis.

  45. Laws, P., Sokoloff, D., & Thornton, R. (1999). Promoting active learning using the results of physics education research. UniServe Science News,13, 14–19.

  46. MacDonald, J. (2008). Blended learning and online tutoring: Planning learner support and activity design (2nd ed.). Aldershot: Gower Publishing.

  47. Marzano, R. J. (1998). A theory-based meta-analysis of research on instruction. Aurora, CO: Mid-Continent Regional Educational Laboratory. Retrieved October 26, 2007, from http://www.mcrel.org/pdf/instruction/5982rr_instructionmeta_analysis.pdf.

  48. Mayer, R. E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist,32(1), 1–19.

  49. Mayer, R. E. (2003). The promise of multimedia learning: Using the same instructional design methods across different media. Learning and Instruction,13, 125–139.

  50. Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review,63, 814–897.

  51. Paivio, A. (1986). Mental representations: A dual-coding approach. New York: Oxford University Press.

  52. Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbour, MI: The University of Michigan, National Center for Research to Improve Post-secondary Teaching and Learning.

  53. Redish, E., Saul, J., & Steinberg, R. (1997). On the effectiveness of active-engagement microcomputer-based laboratories. American Journal of Physics,65(1), 45.

  54. Reed, J. H., & Kromrey, J. D. (2001). Teaching critical thinking in a community college history course: Empirical evidence from infusing Paul’s model. College Student Journal,35(2), 201–215.

  55. Reid-Martinez, K., Grooms, L. D., & Bocarnea, M. C. (2009). Constructivism in online distance education. Encyclopedia of Information Science and Technology,2, 701–707.

  56. Rotter, J. B. (1989). Internal versus external control of reinforcement: A case history of a variable. American Psychologist,45, 489–493.

  57. Rovai, A. P., & Jordan, H. (2004). Blended learning and sense of community: A comparative analysis with traditional and fully online graduate courses. The International Review of Research in Open and Distributed Learning. https://doi.org/10.19173/irrodl.

  58. Siemens, G., & Conole, G. (2011). Special issue—Connectivism: Design and delivery of social networked learning. The International Review of Research in Open and Distance Learning,12(3), 1–5.

  59. Solon, T. (2007). Generic critical thinking infusion and course content learning in Introductory Psychology. Journal of Instructional Psychology,34(2), 95–109.

  60. Sringam, C., & Geer, R. (2000). An investigation of an instrument for analysis of student-led electronic discussions. In Memorias del 17th annual conference of the Australasian Society for Computers in Learning in Tertiary Education (pp. 1–16).

  61. Stein, J., & Graham, C. R. (2014). Essentials for blended learning: A standards-based guide. New York: Routledge.

  62. Su, B., Bonk, C. J., Magjuka, R. J., Liu, X., & Lee, S. H. (2005). The importance of interaction in web-based education: A program-level case study of online MBA courses. Journal of Interactive Online Learning,4(1), 1–19.

  63. Tung, L. C. (2013). Improving students’ educational experience by harnessing digital technology: elgg in the ODL environment. Contemporary Educational Technology,4(4), 236–248.

  64. Twigg, C. (2002). Quality, cost and access: The case for redesign. In M. S. Pittinsky (Ed.), The wired tower (pp. 111–143). Upper Saddle River, NJ: Prentice-Hall.

  65. Vai, M., & Sosulski, K. (2015). Essentials of online course design: A standards-based guide. New York: Routledge.

  66. Valenzuela, J., Nieto, A. M., & Saiz, C. (2011). Critical Thinking Motivational Scale: A contribution to the study of relationship between critical thinking and motivation. Journal of Research in Educational Psychology,9(2), 823–848.

  67. van Gelder, T. J. (2001). How to improve critical thinking using educational technology. In G. Kennedy, M. Keppell, C. McNaught, & T. Petrovic (Eds.), Meeting at the crossroads: Proceedings of the 18th annual conference of the Australian Society for Computers in Learning in Tertiary Education (pp. 39–548). Melbourne: Biomedical Multimedia Unit.

  68. van Gelder, T. J., & Rizzo, A. (2001). Reason!Able across the curriculum. In 2001: Is IT an Odyssey in learning? Proceedings of the 2001 conference of ICT in education. Victoria, Australia.

  69. van Gelder, T. J. (2003). Enhancing deliberation through computer supported argument mapping. In P. A. Kirschner, S. Buckingham-Shum, & C. Carr (Eds.), Visualizing argumentation: Software tools for collaborative and educational sense-making (pp. 97–115). London: Springer-Verlag.

  70. van Gelder, T. J., Bissett, M., & Cumming, G. (2004). Enhancing expertise in informal reasoning. Canadian Journal of Experimental Psychology,58, 142–152.

  71. Walsh, C. M., & Hardy, R. C. (1999). Dispositional differences in critical thinking related to gender and academic major. Journal of Nursing Education,38(4), 149–155.

  72. Walsh, C. M., Seldomridge, L. A., & Badros, K. K. (2007). California critical thinking disposition inventory: Further factor analytic examination 1. Perceptual and motor skills,104(1), 141–151.

  73. Wankat, P. (2002). The effective, efficient professor: Teaching, scholarship and service. Boston: Allyn and Bacon.

  74. Wedemeyer, C. A. (1981). Learning at the back door: Reflections on non-traditional learning in the lifespan. Madison: University of Wisconsin Press.

  75. Willingham, D. T. (2008). Critical thinking: Why is it so hard to teach? Arts Education Policy Review,109(4), 21–32.

  76. Wood, D. J., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry,17, 89–100.

Download references

Author information

Correspondence to Christopher P. Dwyer.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dwyer, C.P., Walsh, A. An exploratory quantitative case study of critical thinking development through adult distance learning. Education Tech Research Dev 68, 17–35 (2020). https://doi.org/10.1007/s11423-019-09659-2

Download citation


  • Critical thinking
  • Adult distance learning
  • e-Learning
  • Disposition towards thinking