Skip to main content
SpringerLink
Log in

Anonymous versus public student feedback systems: metacognition and achievement with graduate learners

  • Research Article
  • Published:
Educational Technology Research and Development Aims and scope Submit manuscript

Abstract

The aim of this repeated measures study was to examine an anonymous student feedback system (delivered using clickers) versus a public student feedback system (delivered using flashcards) on 52 graduate learners’ metacognition and academic achievement scores. Three dimensions of metacognition were examined in a large lecture setting, including Metacognitive Learning Device Attribution, Metacognitive Knowledge in Lectures, and Metacognitive Self-Regulation. Results indicated that Metacognitive Learning Device Attribution and Metacognitive Knowledge in Lectures were significantly higher in the anonymous feedback condition as hypothesized while, contrary to our hypothesis, difference in Metacognitive Self-Regulation was not significant. Also, academic achievement differences were highly significant in favor of the anonymous feedback condition. Effect sizes for the three significant dependent variables ranged from moderate to very large with the largest effect size found for academic achievement. Findings are discussed in terms of the existing literature and the study’s internal and external validity. Recommendations for future research are made.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

*Meta-analysis

  • Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., et al. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives, abridged edition. White Plains, NY: Longman.

    Google Scholar 

  • Anthis, K. (2011). Is it the clicker, or is it the question? Untangling the effects of student feedback system use. Teaching of Psychology, 38(3), 189–193. https://doi.org/10.1177/0098628311411895.

    Article  Google Scholar 

  • Aronson, E. (2008). The social animal (pp. 2–57). New York: Worth Publishing.

    Google Scholar 

  • Artino Jr, A. R. (2005). Review of the motivated strategies for learning questionnaire. Online Submission. Retrieved from https://eric.ed.gov/?id=ED499083.

  • Asch, S. E. (1956). Studies of independence and conformity: I. A minority of one against a unanimous majority. Psychological Monographs: General and Applied, 70(9), 1.

    Article  Google Scholar 

  • Beatty, I. D., Grace, W. J., Leonard, W. J., & Dufense, R. J. (2006). Designing effective questions for classroom response systems teaching. American Journal of Physics, 74(1), 31–39. https://doi.org/10.1119/1.2121753.

    Article  Google Scholar 

  • Becker, S. A., Brown, M., Dahlstrom, E., Davis, A., DePaul, K., Diaz, V., et al. (2018). NMC horizon report: 2018 higher (Education ed.). Austin, TX: The New Media Consortium.

    Google Scholar 

  • Becker, S. A., Freeman, A., Hall, C. G., Cummins, M., & Yuhnke, B. (2016). NMC/CoSN horizon report: 2016 K-12 edition. Austin, TX: The New Media Consortium.

    Google Scholar 

  • Berns, G. S., Chappelow, J., Zink, C. F., Pagnoni, G., Martin-Skurski, M. E., & Richards, J. (2005). Neurobiological correlates of social conformity and independence during mental rotation. Biological Psychiatry, 58(3), 245–253. https://doi.org/10.1016/j.biopsych.2005.04.012.

    Article  Google Scholar 

  • Blasco-Arcas, L., Buil, I., Hernández-Ortega, B., & Sese, F. J. (2013). Using clickers in class. The role of interactivity, active collaborative learning and engagement in learning performance. Computers & Education, 62, 102–110. https://doi.org/10.1016/j.compedu.2012.10.019.

    Article  Google Scholar 

  • Brady, M., & Forest, C. (2018). Metacognition, formative assessment, and student perspective: Learning about metacognition through in-class comparison of response systems. Journal of Physician Assistant Education., 29(2), 104–108.

    Article  Google Scholar 

  • Brady, M., Rosenthal, J., & Forest, C. (2015). The Effect of Audience Response Systems on Metacognition in Graduate Students: A two-year mixed methods study. Paper presented at the international conference on Innovations in Medical Education 2015, Los Angeles, CA. Abstract retrieved from http://keck.usc.edu/medical-education/ime-conference-2018/ime-conference-past-proceedings/.

  • Brady, M. L., Seli, H., & Rosenthal, J. (2013a). “Clickers” and metacognition: A quasi-experimental comparative study about metacognitive self-regulation and use of electronic feedback devices. Computers & Education, 65(2013), 56–63. https://doi.org/10.1016/j.compedu.2013.02.001.

    Article  Google Scholar 

  • Brady, M. L., Seli, H., & Rosenthal, J. (2013b). Metacognition and the influence of polling systems: How do clickers compare with low technology systems. Educational Technology Research and Development, 62(6), 885–902. https://doi.org/10.1007/s11423-013-9318-1.

    Article  Google Scholar 

  • Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best-practice tips. Life Sciences Education, 6, 9–20. https://doi.org/10.1187/cbe.06-12-0205.

    Article  Google Scholar 

  • Campbell, D. T. (1957). Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54, 297–312.

    Article  Google Scholar 

  • Castillo-Manzano, J. I., Castro-Nuño, M., López-Valpuesta, L., Sanz-Díaz, M. T., & Yñiguez, R. (2016). Measuring the effect of ARS on academic performance: A global meta-analysis. Computers & Education, 96, 109–121. https://doi.org/10.1016/j.compedu.2016.02.007.

    Article  Google Scholar 

  • Chen, J. C., Whittinghill, D. C., & Kadlowec, J. A. (2010). Classes that click: Fast, rich feedback to enhance students’ learning and satisfaction. Journal of Engineering Education, 99(2), 158–169.

    Article  Google Scholar 

  • Chien, Y. T., Chang, Y. H., & Chang, C. Y. (2016). Do we click in the right way? A meta-analytic review of clicker-integrated instruction. Educational Research Review, 17, 1–18. https://doi.org/10.1016/j.edurev.2015.10.003.

    Article  Google Scholar 

  • Cleary, A. M. (2008). Using wireless feedback systems to replicate behavioral research findings in the classroom. Teaching of Psychology, 35, 42–44. https://doi.org/10.10180/00986280701826642.

    Article  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Laurence Erlbaum.

    Google Scholar 

  • *de Boer, H., Donker, A. S., Kostons, D. D., & van der Werf, G. P. (2018). Long-term effects of metacognitive strategy instruction on student academic performance: A meta-analysis. Educational Research Review., 24, 98–115. https://doi.org/10.1016/j.edurev.2018.03.002.

    Article  Google Scholar 

  • De Corte, E., Verschaffel, L., & Masui, C. (2004). The CLIA-model: A framework for designing powerful learning environments for thinking and problem solving. European Journal of Psychology of Education, 19(4), 365–384.

    Article  Google Scholar 

  • Duncan, D. (2006). Clickers: A new teaching aid with exceptional promise. Astronomy Education Review, 5(1), 1–19.

    Article  Google Scholar 

  • Formative Assessment for Students and Teachers (FAST) State Collaborative on Assessment and Student Standards (SCASS). (2008). Attributes of effective formative assessment. Paper prepared for the formative assessment for teachers and students state collaborative on assessment and student standards of the council of chief state school officers. Washington, DC: Council of Chief State School Officers.

  • Han, J. H. (2014). Closing the missing links and opening the relationships among the factors: A literature review on the use of clicker technology using the 3P model. Journal of Educational Technology & Society, 17(4), 150–168.

    Google Scholar 

  • *Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Abingdon: Routledge.

    Google Scholar 

  • Heaslip, G., Donovan, P., & Cullen, J. G. (2014). Student response systems and learner engagement in large classes. Active Learning in Higher Education, 15(1), 11–24.

    Article  Google Scholar 

  • Heritage, M. (2010). Formative assessment and next-generation assessment systems: Are we losing an opportunity? Council of Chief State School Officers. Retrieved from https://files.eric.ed.gov/fulltext/ED543063.pdf.

  • Hoekstra, A. (2008). Vibrant student voices: Exploring effects of the use of clickers in large college courses. Learning, Media and Technology, 33(4), 329–341.

    Article  Google Scholar 

  • Hooker, J. F., Denker, K. J., Summers, M. E., & Parker, M. (2016). The development and validation of the student response system benefit scale. Journal of Computer Assisted Learning, 32(2), 120–127.

    Article  Google Scholar 

  • Hunsu, N. J., Adesope, O., & Bayly, D. J. (2016). A meta-analysis of the effects of audience response systems (clicker-based technologies) on cognition and affect. Computers & Education, 94, 102–119. https://doi.org/10.1016/j.compedu.2015.11.013.

    Article  Google Scholar 

  • Immordino-Yang, M. H., & Christodoulou, J. C. (2014). Neuroscientific contributions to understanding and measuring emotions in educational contexts. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), Handbook of emotions in education (pp. 617–634). New York: Taylor and Francis/Routledge.

    Google Scholar 

  • Johnson, L., Becker, S. A., Cummins, M., Estrada, V., Freeman, A., & Hall, C. (2016). NMC horizon report: 2016 higher education edition (pp. 1–50). Austin, TX: The New Media Consortium.

    Google Scholar 

  • Kay, R. H., & LeSage, A. (2009). Examining the benefits and challenges of using audience response systems: A review of the literature. Computers & Education, 53(3), 819–827. https://doi.org/10.1016/j.compedu.2009.05.001.

    Article  Google Scholar 

  • Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2.

    Article  Google Scholar 

  • Lantz, M. E. (2010). The use of ‘clickers’ in the classroom: Teaching innovation or merely an amusing novelty? Computers in Human Behavior, 26(4), 556–561. https://doi.org/10.1016/j.chb.2010.02.014.

    Article  Google Scholar 

  • Mayer, R. E. (2008). Learning and instruction (2nd ed.). Upper Saddle River, NJ: Pearson Merrill Prentice Hall.

    Google Scholar 

  • Mayer, R. E. (2011). Applying the science of learning. Boston, MA: Pearson Education.

    Google Scholar 

  • Mayer, R. E., Stull, A., DeLeeuw, K., Almeroth, K., Bimber, B., Chun, D., et al. (2009). Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes. Contemporary Educational Psychology, 34, 51–57. https://doi.org/10.1016/j.cedpsych.2008.04.002.

    Article  Google Scholar 

  • Mazur, E. (1991). Peer instruction: A user’s manual. Upper Saddle River, NJ: Prentice-Hall.

    Google Scholar 

  • Meltzer, D. E., & Manivannan, K. (2002). Transforming the lecture-hall environment: The fully interactive physics lecture. American Journal of Physics, 70(6), 639–654. https://doi.org/10.1177/0092055X09353890.

    Article  Google Scholar 

  • Mokhtari, K., & Reichard, C. A. (2002). Assessing students’ metacognitive awareness of reading strategies. Journal of Educational Psychology, 94(2), 249.

    Article  Google Scholar 

  • Mollborn, S., & Hoekstra, A. (2010). “A meeting of minds” using clickers for critical thinking and discussion in large sociology classes. Teaching Sociology, 38(1), 18–27. https://doi.org/10.1177/0092055X09353890.

    Article  Google Scholar 

  • Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory into Practice, 41(4), 219–225. https://doi.org/10.1207/s15430421tip4104_3.

    Article  Google Scholar 

  • Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire. Education and Psychological Measures, 53, 801. https://doi.org/10.1177/0013164493053003024.

    Article  Google Scholar 

  • Pintrich, P. R., Wolters, C. A., & Baxter, G. P. (2000). Assessing metacognition and self-regulated learning. In G. Schraw & J. C. Impara (Eds.), Issues in the measurement of metacognition (pp. 43–97). Lincoln, NE: Buros Institute of Mental Measurements.

    Google Scholar 

  • Rocca, K. A. (2010). Student participation in the college classroom: An extended multidisciplinary literature review. Communication Education, 59(2), 185–213. https://doi.org/10.1080/03634520903505936.

    Article  Google Scholar 

  • Schell, J., Lukoff, B., & Mazur, E. (2013). Catalyzing learner engagement using cutting-edge classroom response systems in higher education. In C. Wankel & P. Blessinger (Eds.), Increasing student engagement and retention using classroom technologies: Classroom Response Systems and Mediated Discourse Technologies (pp. 233–261). Bingley: Emerald Group Publishing Limited.

    Chapter  Google Scholar 

  • Schraw, G., & Gutierrez, A. P. (2015). Metacognitive strategy instruction that highlights the role of monitoring and control processes. In Metacognition: Fundaments, applications, and trends (pp. 3–16). Cham: Springer.

  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin Company.

    Google Scholar 

  • Shunk, D. H. (2005). Self-regulated learning: The educational legacy of Paul R. Pintrich. Educational Psychologist, 40(2), 85–94. https://doi.org/10.1207/s15326985ep4002_3.

    Article  Google Scholar 

  • Stowell, J. R. (2015). Use of clickers vs. mobile devices for classroom polling. Computers & Education, 82, 329–334. https://doi.org/10.1016/j.compedu.2014.12.008.

    Article  Google Scholar 

  • Stowell, J. R., & Nelson, J. M. (2007). Benefits of electronic audience feedback systems on student participation, learning, and emotion. Teaching of Psychology, 34(4), 253–258. https://doi.org/10.1080/00986280701700391.

    Article  Google Scholar 

  • Stowell, J. R., Oldham, T., & Bennett, D. (2010). Using student feedback systems (“clickers”) to combat conformity and shyness. Teaching of Psychology, 37(2), 135–140.

    Article  Google Scholar 

  • Thorndike, E. L. (1913–1914). Educational psychology. New York: Columbia University, Teachers College.

  • Van Diik, L., Van Der Berg, G., & Van Keulen, H. (2001). Interactive lectures in engineering education. European Journal of Engineering Education, 26, 15–28. https://doi.org/10.1080/03043790123124.

    Article  Google Scholar 

  • Watkins, J., & Mazur, E. (2013). Retaining students in science, technology, engineering, and mathematics (STEM) majors. Journal of College Science Teaching, 42(5), 36–41.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Rossier School of Education, University of Southern California, 3470 Trousdale Parkway, Waites Philip Hall (WPH) 404, Los Angeles, CA, 90089-4034, USA

    Melanie Brady, Jane L. Rosenthal, Christopher P. Forest & Dennis Hocevar

Authors
  1. Melanie Brady
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jane L. Rosenthal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christopher P. Forest
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dennis Hocevar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Melanie Brady.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Brady, M., Rosenthal, J.L., Forest, C.P. et al. Anonymous versus public student feedback systems: metacognition and achievement with graduate learners. Education Tech Research Dev 68, 2853–2872 (2020). https://doi.org/10.1007/s11423-020-09800-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-020-09800-6

Keywords

Search

Navigation