Skip to main content
SpringerLink
Log in

Does a deep learning inventory predict knowledge transfer? Linking student perceptions to transfer outcomes

  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

Abstract

Students are often encouraged to learn ‘deeply’ by abstracting generalizable principles from course content rather than memorizing details. So widespread is this perspective that Likert-style inventories are now routinely administered to students to quantify how much a given course or curriculum evokes deep learning. The predictive validity of these inventories, however, has been criticized based on sparse empirical support and ambiguity in what specific outcome measures indicate whether deep learning has occurred. Here we further tested the predictive validity of a prevalent deep learning inventory, the Revised Two-Factor Study Process Questionnaire, by selectively analyzing outcome measures that reflect a major goal of medical education—i.e., knowledge transfer. Students from two undergraduate health sciences courses completed the deep learning inventory before their course’s final exam. Shortly after, a random subset of students rated how much each final exam item aligned with three task demands associated with transfer: (1) application of general principles, (2) integration of multiple ideas or examples, and (3) contextual novelty. We then used these ratings from students to examine performance on a subset of exam items that were collectively perceived to demand transfer. Despite good reliability, the resulting transfer outcomes were not substantively predicted by the deep learning inventory. These findings challenge the validity of this tool and others like it.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. Some prior versions of the instrument included three latent constructs—deep, surface, and achieving—where the latter achieving construct reflects an intention to obtain good grades through conscientiousness, organization, and anticipation of assessment styles (e.g., Donnon & Hecker, 2008). However, large-scale studies have consistently shown that a 2-factor model with only the deep and surface constructs better fits the data. For instance, Kember and Leung (1998) analyzed inventory scores from about 5000 university students and found the two-factor model was superior to the three-factor model. This was later replicated by Zeegers (2002) with about 1000 students. Snelgrove and Slater (2003) also found considerable overlap between the deep and achieving constructs, suggesting the latter could be subsumed under the former. Such results have led prominent researchers in this area, including the original creator of the inventory (Biggs et al., 2001), to recommend use of the streamlined 2-factor model that we adopted here (Justicia et al., 2008; Kember et al., 1999; Richardson, 1994).

  2. The surface approach construct is meant to reflect rote memorization and other related superficial study strategies. However, as alluded to earlier, we were only interested in the deep approach construct here because it represents the more desirable (and challenging) goal among educators.

References

  • Abbiati, M., Baroffio, A., & Gerbase, M. W. (2016). Personal profile of medical students selected through a knowledge-based exam only: Are we missing suitable students? Medical Education Online, 21(1), 29705.

    Article  Google Scholar 

  • Agarwal, P. K. (2019). Retrieval practice & Bloom’s taxonomy: Do students need fact knowledge before higher order learning? Journal of Educational Psychology, 111(2), 189–209.

    Article  Google Scholar 

  • Alfieri, L., Nokes-Malach, T. J., & Schunn, C. D. (2013). Learning through case comparisons: A meta-analytic review. Educational Psychologist, 48(2), 87–113.

    Article  Google Scholar 

  • Baeten, M., Kyndt, E., Struyven, K., & Dochy, F. (2010). Using student-centred learning environments to stimulate deep approaches to learning: Factors encouraging or discouraging their effectiveness. Educational Research Review, 5(3), 243–260.

    Article  Google Scholar 

  • Barnett, S. M., & Ceci, S. J. (2002). When and where do we apply what we learn? A taxonomy for far transfer. Psychological Bulletin, 128(4), 612–637.

    Article  Google Scholar 

  • Biggs, J. (1993). What do inventories of students’ learning processes really measure? A theoretical review and clarification. British Journal of Educational Psychology, 63(1), 3–19.

    Article  Google Scholar 

  • Biggs, J. (1999). What the student does: Teaching for enhanced learning. Higher Education Research & Development, 18(1), 57–75.

    Article  Google Scholar 

  • Biggs, J. B. (1987). Student approaches to learning and studying. Australian Council for Educational Research Ltd.

    Google Scholar 

  • Biggs, J., Kember, D., & Leung, D. Y. (2001). The revised two-factor study process questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71(1), 133–149.

    Article  Google Scholar 

  • Bloom, B. S. (Ed.), Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D .R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. David McKay.

  • Borsboom, D., Mellenbergh, G. J., & Van Heerden, J. (2004). The concept of validity. Psychological Review, 111(4), 1061–1071.

    Article  Google Scholar 

  • Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. Review of Research in Education, 24(1), 61–100.

    Article  Google Scholar 

  • Butler, A. C. (2010). Repeated testing produces superior transfer of learning relative to repeated studying. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(5), 1118–1133.

    Google Scholar 

  • Catrambone, R., & Holyoak, K. J. (1989). Overcoming contextual limitations on problem-solving transfer. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15(6), 1147–1156.

    Google Scholar 

  • Chernobilsky, E., DaCosta, M. C., & Hmelo-Silver, C. E. (2004). Learning to talk the educational psychology talk through a problem-based course. Instructional Science, 32(4), 319–356.

    Article  Google Scholar 

  • Choy, J. L. F., O’Grady, G., & Rotgans, J. I. (2012). Is the Study Process Questionnaire (SPQ) a good predictor of academic achievement? Examining the mediating role of achievement-related classroom behaviours. Instructional Science, 40(1), 159–172.

    Article  Google Scholar 

  • Christie, S., & Gentner, D. (2010). Where hypotheses come from: Learning new relations by structural alignment. Journal of Cognition and Development, 11(3), 356–373.

    Article  Google Scholar 

  • Day, S. B., & Goldstone, R. L. (2012). The import of knowledge export: Connecting findings and theories of transfer of learning. Educational Psychologist, 47(3), 153–176.

    Article  Google Scholar 

  • DeLeeuw, K. E., & Mayer, R. E. (2008). A comparison of three measures of cognitive load: Evidence for separable measures of intrinsic, extraneous, and germane load. Journal of Educational Psychology, 100(1), 223–234.

    Article  Google Scholar 

  • DeLosh, E. L., Busemeyer, J. R., & McDaniel, M. A. (1997). Extrapolation: The sine qua non for abstraction in function learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23(4), 968–986.

    Google Scholar 

  • Dinsmore, D. L., & Alexander, P. A. (2012). A critical discussion of deep and surface processing: What it means, how it is measured, the role of context, and model specification. Educational Psychology Review, 24(4), 499–567.

    Article  Google Scholar 

  • Dolmans, D. H., Loyens, S. M., Marcq, H., & Gijbels, D. (2016). Deep and surface learning in problem-based learning: A review of the literature. Advances in Health Sciences Education, 21(5), 1087–1112.

    Article  Google Scholar 

  • Donnon, T., & Hecker, K. (2008). A model of approaches to learning and academic achievement of students from an inquiry-based bachelor of health sciences program. Canadian Journal of Higher Education, 38(1), 1–19.

    Article  Google Scholar 

  • Dunlosky, J., & Rawson, K. A. (2012). Overconfidence produces underachievement: Inaccurate self evaluations undermine students’ learning and retention. Learning and Instruction, 22(4), 271–280.

    Article  Google Scholar 

  • Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12(3), 83–87.

    Article  Google Scholar 

  • Entwistle, N., Hanley, M., & Hounsell, D. (1979). Identifying distinctive approaches to studying. Higher Education, 8(4), 365–380.

    Article  Google Scholar 

  • Eva, K. W., & Regehr, G. (2005). Self-assessment in the health professions: A reformulation and research agenda. Academic Medicine, 80(10), S46–S54.

    Article  Google Scholar 

  • Ferland, J. J., Dorval, J., & Levasseur, L. (1987). Measuring higher cognitive levels by multiple choice questions: A myth? Medical Education, 21(2), 109–113.

    Article  Google Scholar 

  • Frederiksen, N. (1984). The real test bias: Influences of testing on teaching and learning. American Psychologist, 39, 193–202.

    Article  Google Scholar 

  • Frey, R. F., Cahill, M. J., & McDaniel, M. A. (2017). Students’ concept-building approaches: A novel predictor of success in chemistry courses. Journal of Chemical Education, 94(9), 1185–1194.

    Article  Google Scholar 

  • Gentner, D., & Smith, L. (2012). Analogical reasoning. In V. S. Ramachandran (Ed.), Encyclopedia of Human Behavior (2nd ed., pp. 130–136). Elsevier.

    Chapter  Google Scholar 

  • Gick, M. L., & Holyoak, K. J. (1983). Schema induction and analogical transfer. Cognitive Psychology, 15, 1–38.

    Article  Google Scholar 

  • Gijbels, D., Van de Watering, G., Dochy, F., & Van den Bossche, P. (2005). The relationship between students’ approaches to learning and the assessment of learning outcomes. European Journal of Psychology of Education, 20(4), 327–341.

    Article  Google Scholar 

  • Goldstone, R. L., & Sakamoto, Y. (2003). The transfer of abstract principles governing complex adaptive systems. Cognitive Psychology, 46(4), 414–466.

    Article  Google Scholar 

  • Grierson, L. E. (2014). Information processing, specificity of practice, and the transfer of learning: Considerations for reconsidering fidelity. Advances in Health Sciences Education, 19(2), 281–289.

    Article  Google Scholar 

  • Groves, M. (2005). Problem-based learning and learning approach: Is there a relationship? Advances in Health Sciences Education, 10(4), 315–326.

    Article  Google Scholar 

  • Halpern, D. F., & Hakel, M. D. (2003). Applying the science of learning to the university and beyond: Teaching for long-term retention and transfer. Change: The Magazine of Higher Learning, 35(4), 36–41.

    Article  Google Scholar 

  • Hattie, J. A., & Donoghue, G. M. (2016). Learning strategies: A synthesis and conceptual model. NPJ Science of Learning, 1(1), 1–13.

    Article  Google Scholar 

  • Howie, P., & Bagnall, R. (2013). A critique of the deep and surface approaches to learning model. Teaching in Higher Education, 18(4), 389–400.

    Article  Google Scholar 

  • Justicia, F., Pichardo, M. C., Cano, F., Berbén, A. B. G., & De la Fuente, J. (2008). The revised two-factor study process questionnaire (R-SPQ-2F): Exploratory and confirmatory factor analyses at item level. European Journal of Psychology of Education, 23(3), 355–372.

    Article  Google Scholar 

  • Kek, M. Y. C. A., & Huijser, H. (2011). The power of problem-based learning in developing critical thinking skills: Preparing students for tomorrow’s digital futures in today’s classrooms. Higher Education Research & Development, 30(3), 329–341.

    Article  Google Scholar 

  • Kember, D. (1997). Evaluating the effectiveness of educational innovations: Using the study process questionnaire to show that meaningful learning occurs. Studies in Educational Evaluation, 23(2), 141–157.

    Article  Google Scholar 

  • Kember, D., & Leung, D. Y. (1998). The dimensionality of approaches to learning: An investigation with confirmatory factor analysis on the structure of the SPQ and LPQ. British Journal of Educational Psychology, 68(3), 395–407.

    Article  Google Scholar 

  • Kember, D., Wong, A., & Leung, D. Y. (1999). Reconsidering the dimensions of approaches to learning. British Journal of Educational Psychology, 69(3), 323–343.

    Article  Google Scholar 

  • Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212–218.

    Article  Google Scholar 

  • Kulasegaram, K., Manzone, J. C., Ku, C., Skye, A., Wadey, V., & Woods, N. N. (2015). Cause and effect: Testing a mechanism and method for the cognitive integration of basic science. Academic Medicine, 90(11), S63–S69.

    Article  Google Scholar 

  • Kusurkar, R. A., Ten Cate, T. J., Vos, C. M. P., Westers, P., & Croiset, G. (2013). How motivation affects academic performance: A structural equation modelling analysis. Advances in Health Sciences Education, 18(1), 57–69.

    Article  Google Scholar 

  • Laird, T. F. N., Seifert, T. A., Pascarella, E. T., Mayhew, M. J., & Blaich, C. F. (2014). Deeply affecting first-year students’ thinking: Deep approaches to learning and three dimensions of cognitive development. The Journal of Higher Education, 85(3), 402–432.

    Article  Google Scholar 

  • Laird, T. F. N., Shoup, R., Kuh, G. D., & Schwarz, M. J. (2008). The effects of discipline on deep approaches to student learning and college outcomes. Research in Higher Education, 49(6), 469–494.

    Article  Google Scholar 

  • Lindblom-Ylänne, S., & Lonka, K. (2001). Students’ perceptions of assessment practices in a traditional medical curriculum. Advances in Health Sciences Education, 6(2), 121–140.

    Article  Google Scholar 

  • Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2012). Multiple-choice tests exonerated, at least of some charges: Fostering test-induced learning and avoiding test-induced forgetting. Psychological Science, 23(11), 1337–1344.

    Article  Google Scholar 

  • McDaniel, M. A., Cahill, M. J., Frey, R. F., Rauch, M., Doele, J., Ruvolo, D., & Daschbach, M. M. (2018). Individual differences in learning exemplars versus abstracting rules: Associations with exam performance in college science. Journal of Applied Research in Memory and Cognition, 7(2), 241–251.

    Article  Google Scholar 

  • McDaniel, M. A., Cahill, M. J., Robbins, M., & Wiener, C. (2014). Individual differences in learning and transfer: Stable tendencies for learning exemplars versus abstracting rules. Journal of Experimental Psychology: General, 143(2), 668–693.

    Article  Google Scholar 

  • McManus, I. C., Richards, P., Winder, B. C., & Sproston, K. A. (1998). Clinical experience, performance in final exams, and learning style in medical students: Prospective study. British Medical Journal, 316(7128), 345–350.

    Article  Google Scholar 

  • Messick, S. (1995). Standards of validity and the validity of standards in performance assessment. Educational Measurement: Issues and Practice, 14(4), 5–8.

    Article  Google Scholar 

  • Needham, D. R., & Begg, I. M. (1991). Problem-oriented training promotes spontaneous analogical transfer: Memory-oriented training promotes memory for training. Memory & Cognition, 19(6), 543–557.

    Article  Google Scholar 

  • Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231–259.

    Article  Google Scholar 

  • Norman, G. (2009). Teaching basic science to optimize transfer. Medical Teacher, 31(9), 807–811.

    Article  Google Scholar 

  • Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38(1), 63–71.

    Article  Google Scholar 

  • Piumatti, G., Abbiati, M., Baroffio, A., & Gerbase, M. W. (2019). Associations between motivational factors for studying medicine, learning approaches and empathy among medical school candidates. Advances in Health Sciences Education, 24(2), 287–300.

    Article  Google Scholar 

  • Phan, H. P. (2011). Deep processing strategies and critical thinking: Developmental trajectories using latent growth analyses. The Journal of Educational Research, 104(4), 283–294.

    Article  Google Scholar 

  • Rawson, K. A., Thomas, R. C., & Jacoby, L. L. (2015). The power of examples: Illustrative examples enhance conceptual learning of declarative concepts. Educational Psychology Review, 27(3), 483–504.

    Article  Google Scholar 

  • Reid, W. A., Duvall, E., & Evans, P. (2005). Can we influence medical students’ approaches to learning? Medical Teacher, 27(5), 401–407.

    Article  Google Scholar 

  • Richardson, J. T. (1994). Cultural specificity of approaches to studying in higher education: A literature survey. Higher Education, 27(4), 449–468.

    Article  Google Scholar 

  • Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138(2), 353–387.

    Article  Google Scholar 

  • Rittle-Johnson, B., & Star, J. R. (2009). Compared with what? The effects of different comparisons on conceptual knowledge and procedural flexibility for equation solving. Journal of Educational Psychology, 101(3), 529–544.

    Article  Google Scholar 

  • Roediger, H. L., III. (2013). Applying cognitive psychology to education: Translational educational science. Psychological Science in the Public Interest, 14(1), 1–3.

    Article  Google Scholar 

  • Salomon, G., & Perkins, D. N. (1989). Rocky roads to transfer: Rethinking mechanism of a neglected phenomenon. Educational Psychologist, 24(2), 113–142.

    Article  Google Scholar 

  • Snelgrove, S., & Slater, J. (2003). Approaches to learning: Psychometric testing of a study process questionnaire. Journal of Advanced Nursing, 43(5), 496–505.

    Article  Google Scholar 

  • Stanger-Hall, K. F. (2012). Multiple-choice exams: An obstacle for higher-level thinking in introductory science classes. CBE: Life Sciences Education, 11(3), 294–306.

    Google Scholar 

  • Sweller, J. (2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22(2), 123–138.

    Article  Google Scholar 

  • van Merriënboer, J. J., & Sweller, J. (2010). Cognitive load theory in health professional education: Design principles and strategies. Medical Education, 44(1), 85–93.

    Article  Google Scholar 

  • Watkins, D. (2001). Correlates of approaches to learning: A cross-cultural meta-analysis. In R. Sternberg & L. Zhang (Eds.), Perspective on thinking, learning, and cognitive styles (pp. 165–195). Erlbaum.

    Google Scholar 

  • Whittlesea, B. W., Jacoby, L. L., & Girard, K. (1990). Illusions of immediate memory: Evidence of an attributional basis for feelings of familiarity and perceptual quality. Journal of Memory and Language, 29(6), 716–732.

    Article  Google Scholar 

  • Wilding, J., & Andrews, B. (2006). Life goals, approaches to study and performance in an undergraduate cohort. British Journal of Educational Psychology, 76(1), 171–182.

    Article  Google Scholar 

  • Woods, N. N. (2007). Science is fundamental: The role of biomedical knowledge in clinical reasoning. Medical Education, 41(12), 1173–1177.

    Article  Google Scholar 

  • Yonelinas, A. P. (2001). Components of episodic memory: The contribution of recollection and familiarity. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 356(1413), 1363–1374.

    Article  Google Scholar 

  • Zeegers, P. (2001). Approaches to learning in science: A longitudinal study. British Journal of Educational Psychology, 71(1), 115–132.

    Article  Google Scholar 

  • Zeegers, P. (2002). A revision of the Biggs’ study process questionnaire (R-SPQ). Higher Education Research & Development, 21(1), 73–92.

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported in part by a SSHRC Canada Graduate Scholarship (Doctoral) awarded to the first author. We also thank to two instructors at McMaster University, Dr. Russel de Souza and Ms. Laura Jin, for allowing us to conduct these studies as part their courses.

Author information

Authors and Affiliations

  1. Department of Evaluative Clinical Sciences, Sunnybrook Research Institute, Sunnybrook Health Sciences Centre, 2075 Bayview Ave, Toronto, ON, M4N 3M5, Canada

    Andrew B. LoGiudice

  2. Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Canada

    Geoffrey R. Norman, Saba Manzoor & Sandra Monteiro

  3. Faculty of Health Sciences Program in Education Research, Innovation and Theory (MERIT), McMaster University, Hamilton, Canada

    Andrew B. LoGiudice, Geoffrey R. Norman & Sandra Monteiro

Authors
  1. Andrew B. LoGiudice
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Geoffrey R. Norman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Saba Manzoor
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sandra Monteiro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrew B. LoGiudice.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A

figure a

Appendix B

figure b

Correlations for ratings of each task demand (application, integration, and novelty) for the 81 final exam items in Study 1 (top graphs) and 52 final exam items in Study 2 (bottom graphs).

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

LoGiudice, A.B., Norman, G.R., Manzoor, S. et al. Does a deep learning inventory predict knowledge transfer? Linking student perceptions to transfer outcomes. Adv in Health Sci Educ 28, 47–63 (2023). https://doi.org/10.1007/s10459-022-10141-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10459-022-10141-7

Keywords

Search

Navigation