Practice makes proficient: teaching undergraduate students to understand published research

  • Trina C. Kershaw
  • Jordan P. Lippman
  • Jennifer M. B. Fugate
Original Research

Abstract

Scientific knowledge, including the critical evaluation and comprehension of empirical articles, is a key skill valued by most undergraduate institutions for students within the sciences. Students often find it difficult to not only summarize empirical journal articles, but moreover to successfully grasp the quality and rigor of investigation behind the source. In this paper, we use instructional scaffolds (reading worksheets, RWs, with tutorials) to aid students in being able to comprehend, and ultimately transfer, the skills necessary in critically evaluating primary sources of research. We assess students’ learning of these skills on a multiple-choice assessment of Journal Article Comprehension (JAC). Students in experimental classes, who received instructional scaffolds, improved on the JAC post-test compared with students in control classes. This result shows that students are acquiring fundamental research skills such as understanding the components of research articles. We also showed that improvement on the JAC post-test for the experimental class extended to a written summary test. This result suggests that students in the experimental group are developing discipline-specific science process skills that allow them to apply JAC skills to a near-transfer task of writing a summary.

Keywords

Reading empirical articles Instructional scaffolds Assessment Learning outcomes Cognitive psychology 

Notes

Acknowledgements

The authors thank Amy Shapiro and Scott Hinze for access to their cognitive psychology courses. We thank James Bradley for his assistance in creating the summary coding scheme. Judy Sims-Knight gave invaluable advice on several statistical issues, and Susan Goldman and Micki Chi provided important suggestions for best practices in establishing inter-rater reliability. We also thank several anonymous reviewers for their feedback. Preliminary results from the pilot studies were presented at the 15th Annual European Association for Research on Learning and Instruction conference (EARLI 2013, Munich, Germany) and the 4th Biennial Conference of the International Society for the Psychology of Science and Technology (ISPST, 2012, Pittsburgh, PA).

References

  1. Lippman, J. P., Kershaw, T. C., Pellegrino, J. W., & Ohlsson, S. (2008). Beyond standard lectures: Supporting the development of critical thinking in cognitive psychology courses. In D. S. Dunn, J. S. Halonen & R. A. Smith (Eds.), Teaching critical thinking in psychology: A handbook of best practices (pp. 183–198). Boston: Blackwell Publishing.CrossRefGoogle Scholar
  2. Kershaw, T. C., Lippman, J. P., & Kolev, L. N. (in preparation). Learning to critique published psychological research.Google Scholar
  3. Anisfeld, M. (1987). A course to develop competence in critical reading of empirical research in psychology. Teaching of Psychology, 14, 224–227.  https://doi.org/10.1207/s15328023top1404_8.CrossRefGoogle Scholar
  4. Bachiochi, P., Everton, W., Evans, M., Fugere, M., Escoto, C., Letterman, M., et al. (2011). Using empirical article analysis to assess research methods courses. Teaching of Psychology, 38, 5–9.  https://doi.org/10.1177/0098628310387787.CrossRefGoogle Scholar
  5. Baram-Tsabari, A., & Yarden, A. (2005). Text genre as a factor in the formation of scientific literacy. Journal of Research in Science Teaching, 42, 403–428.  https://doi.org/10.1002/tea.20063.CrossRefGoogle Scholar
  6. Bednall, T. C., & Kehoe, E. J. (2011). Effects of self-regulatory instructional aids on self-directed study. Instructional Science, 39, 205–226.  https://doi.org/10.1007/s11251-009-9125-6.CrossRefGoogle Scholar
  7. Beilock, S. L., & Carr, T. H. (2005). When high-powered people fail: Working memory and “choking under pressure” in math. Psychological Science, 16, 101–105.  https://doi.org/10.1111/j.0956-7976.2005.00789.x.CrossRefGoogle Scholar
  8. Bretzing, B. H., & Kulhavy, R. W. (1979). Notetaking and depth of processing. Contemporary Educational Psychology, 4, 145–153.  https://doi.org/10.1016/0361-476X(79)90069-9.CrossRefGoogle Scholar
  9. Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132, 354–380.  https://doi.org/10.1037/0033-2909.132.3.354.CrossRefGoogle Scholar
  10. Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6, 271–315.  https://doi.org/10.1207/s15327809jls0603_1.CrossRefGoogle Scholar
  11. Chinn, C. A., & Brewer, W. F. (2001). Models of data: A theory of how people evaluate data. Cognition and Instruction, 19, 323–393.  https://doi.org/10.1207/S1532690XCI1903_3.CrossRefGoogle Scholar
  12. Chinn, C. A., & Malhotra, B. A. (2002). Epistemologically authentic inquiry in schools: A theoretical framework for evaluating inquiry tasks. Science Education, 86, 175–218.  https://doi.org/10.1002/sce.10001.CrossRefGoogle Scholar
  13. Christopher, A. N., & Walter, M. I. (2006). An assignment to help students learn to navigate primary sources of information. Teaching of Psychology, 33, 42–45.Google Scholar
  14. Coil, D., Wenderoth, M. P., Cunningham, M., & Dirks, C. (2010). Teaching the process of science: Faculty perceptions and an effective methodology. CBE—Life Sciences Education, 9, 524–535.  https://doi.org/10.1187/cbe.10-01-0005.Google Scholar
  15. Cooper, J. M., & Strayer, D. L. (2008). Effects of simulator practice and real-world experience on cell-phone-related driver distraction. Human Factors, 50, 893–902.CrossRefGoogle Scholar
  16. Dasgupta, A. P., Anderson, T. R., & Pelaez, N. (2014). Development and validation of a rubric for diagnosing students’ experimental design knowledge and difficulties. CBE—Life Sciences Education, 13, 265–284.  https://doi.org/10.1187/cbe.13-09-0192.Google Scholar
  17. Day, J. D. (1983). Teaching summarization skills: Influences of student ability level and strategy difficulty. Cognition and Instruction, 3, 193–210.  https://doi.org/10.1207/s1532690xci0303_3.CrossRefGoogle Scholar
  18. Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14, 4–58.  https://doi.org/10.1177/1529100612453266.CrossRefGoogle Scholar
  19. Dyer, J. W., Riley, J., & Yekovich, F. R. (1979). An analysis of three study skills: Notetaking, summarizing, and rereading. Journal of Educational Research, 73, 3–7.  https://doi.org/10.1080/00220671.1979.10885194.CrossRefGoogle Scholar
  20. Ericsson, K. A., & Charness, N. (1994). Expert performance: Its structure and acquisition. American Psychologist, 49, 725–747.  https://doi.org/10.1037/0003-066X.49.8.725.CrossRefGoogle Scholar
  21. Fleiss, J. L. (1981). Statistical methods for rates and proportions. New York: Wiley.Google Scholar
  22. Gadgil, S., & Nokes-Malach, T. J. (2012). Collaborative facilitation through error-detection: A classroom experiment. Applied Cognitive Psychology, 26, 410–420.  https://doi.org/10.1002/acp.1843.CrossRefGoogle Scholar
  23. Gillen, C. M. (2006). Criticism and interpretation: Teaching the persuasive aspects of research articles. CBE—Life Sciences Education, 5, 34–38.  https://doi.org/10.1187/cbe.05-08-0101.Google Scholar
  24. Goldman, S. R., & Bisanz, G. (2002). Toward a functional analysis of scientific genres: Implications for understanding and learning processes. In J. Otero, J. A. Leon, & A. C. Graesser (Eds.), The psychology of science text comprehension (pp. 19–50). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  25. Goldman, S. R., & Pellegrino, J. W. (2015). Research on learning and instruction: Implications for curriculum, instruction, and assessment. Policy Insights from the Behavioral and Brain Sciences, 2, 33–41.  https://doi.org/10.1177/2372732215601866.CrossRefGoogle Scholar
  26. Gottfried, G. M., Johnson, K. E., & Vosmik, J. R. (2009). Assessing student learning: A collection of evaluation tools. Office of Teaching Resources in Psychology. Retrieved from http://teachpsych.org/resources/Documents/otrp/resources/gottfried09.pdf.
  27. Gwet, K. L. (2014). Handbook of inter-rater reliability (4th ed.). Gaithersburg, MD: Advanced Analytics LLC.Google Scholar
  28. Hmelo-Silver, C., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42, 99–107.  https://doi.org/10.1080/00461520701263368.CrossRefGoogle Scholar
  29. Karcher, S. J. (2000). Student reviews of scientific literature: Opportunities to improve students’ scientific literacy and writing skills. In S. J. Karcher (Ed.), Tested studies for laboratory teaching. Proceedings of the 22nd Workshop/Conference of the Association for Biology Laboratory Education (pp. 484–487).Google Scholar
  30. Landis, J. R., & Koch, G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.CrossRefGoogle Scholar
  31. Lee, L., Frederick, S., & Ariely, D. (2006). Try it, you’ll like it: The influence of expectation, consumption, and revelation on preferences for beer. Psychological Science, 17, 1054–1058.  https://doi.org/10.1111/j.1467-9280.2006.01829.x.CrossRefGoogle Scholar
  32. Levine, E. (2001). Reading your way to scientific literacy. Journal of College Science Teaching, 31, 122–125.Google Scholar
  33. Locke, L. F., Silverman, S. J., & Spirduso, W. W. (1998). Reading and understanding research. London: Sage.Google Scholar
  34. Lorch, R. F., Jr., & Lorch, E. P. (1996). Effects of organizational signals on free recall of expository text. Journal of Educational Psychology, 88, 38–48.  https://doi.org/10.1006/ceps.1996.0022.CrossRefGoogle Scholar
  35. Macnamara, B. N., Hambrick, D. Z., & Oswald, F. L. (2014). Deliberate practice and performance in music, games, sports, education, and professions: A meta-analysis. Psychological Science, 25, 1608–1618.  https://doi.org/10.1177/0956797614535810.CrossRefGoogle Scholar
  36. Madigan, R., Johnson, S., & Linton, P. (1995). The language of psychology: APA style as epistemology. American Psychologist, 50, 428–436.  https://doi.org/10.1037/0003-066X.50.6.428.CrossRefGoogle Scholar
  37. Morris, B. J., Croker, S., Masnick, A. M., & Zimmerman, C. (2012). The emergence of scientific reasoning. In H. Kloos, B. J. Morris, & J. L. Amaral (Eds.), Current topics in children’s learning and cognition (pp. 61–82). Rijeka, Croatia: InTech.Google Scholar
  38. Newell, G. E., Beach, R., Smith, J., & VanDerHeide, J. (2011). Teaching and learning argumentative reading and writing: A review of research. Reading Research Quarterly, 46, 273–304.  https://doi.org/10.1598/RRQ.46.3.4.Google Scholar
  39. Oldenburg, C. M. (2016). Use of primary source readings in psychology courses at liberal arts colleges. Teaching of Psychology, 32, 25–29.CrossRefGoogle Scholar
  40. Reiser, B. J. (2004). Scaffolding complex learning: The mechanisms of structuring and problematizing student work. The Journal of the Learning Sciences, 13, 273–304.  https://doi.org/10.1207/s15327809jls1303_2.CrossRefGoogle Scholar
  41. Robertson, K. (2012). A journal club workshop that teaches undergraduates a systematic method for reading, interpreting, and presenting primary literature. Journal of College Science Teaching, 41, 25–31.Google Scholar
  42. Russell, J. S., Martin, L., Curtin, D., Penhale, S., & Trueblood, N. A. (2004). Non-science majors gain valuable insight studying clinical trials literature: An evidence-based medicine library assignment. Advances in Physiological Education, 28, 188–194.CrossRefGoogle Scholar
  43. Sego, S. A., & Stuart, A. E. (2016). Learning to read empirical articles in general psychology. Teaching of Psychology, 43, 38–42.  https://doi.org/10.1177/0098628315620875.CrossRefGoogle Scholar
  44. Shapiro, A. M. (2004). How including prior knowledge as a subject variable may change outcomes of learning research. American Educational Research Journal, 41, 159–189.  https://doi.org/10.3102/00028312041001159.CrossRefGoogle Scholar
  45. Son, L. K., & Simon, D. A. (2012). Distributed learning: Data, metacognition, and educational implications. Educational Psychology Review, 24, 379–399.  https://doi.org/10.1007/s10648-012-9206-y.CrossRefGoogle Scholar
  46. Suter, W. N., & Frank, P. (1986). Using scholarly journals in undergraduate experimental methodology courses. Teaching of Psychology, 13, 219–221.CrossRefGoogle Scholar
  47. Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Boston: Pearson.Google Scholar
  48. Taylor, K. K. (1983). Can college students summarize? Journal of Reading, 26, 524–528.Google Scholar
  49. van Gelder, T., Bissett, M., & Cumming, G. (2004). Cultivating expertise in informal reasoning. Canadian Journal of Experimental Psychology, 58, 142–152.  https://doi.org/10.1037/h0085794.CrossRefGoogle Scholar
  50. Van Lacum, E. B., Ossevoort, M. A., & Goedhart, M. J. (2014). A teaching strategy with a focus on argumentation to improve undergraduate students’ ability to read research articles. CBE—Life Sciences Education, 13, 253–264.  https://doi.org/10.1187/cbe.13-06-0110.Google Scholar
  51. Yarden, A. (2009). Reading scientific texts: Adapting primary literature for promoting scientific literacy. Research in Science Education, 39, 307–311.  https://doi.org/10.1007/s11165-009-9124-2.CrossRefGoogle Scholar
  52. Yarden, A., Brill, G., & Falk, H. (2001). Primary literature as a basis for a high-school biology curriculum. Journal of Biological Education, 35, 190–195.CrossRefGoogle Scholar
  53. Zieffler, A. S., & Garfield, J. B. (2009). Modeling the growth of students’ covariational reasoning during an introductory statistics course. Statistics Education Research Journal, 8, 7–31.Google Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2018

Authors and Affiliations

  • Trina C. Kershaw
    • 1
  • Jordan P. Lippman
    • 2
  • Jennifer M. B. Fugate
    • 1
  1. 1.Department of PsychologyUniversity of Massachusetts DartmouthNorth DartmouthUSA
  2. 2.The TeamBuilders GroupPittsburghUSA

Personalised recommendations