Advertisement

Journal of Computing in Higher Education

, Volume 21, Issue 2, pp 95–109 | Cite as

Technology’s effect on achievement in higher education: a Stage I meta-analysis of classroom applications

  • Richard F. Schmid
  • Robert M. Bernard
  • Eugene Borokhovski
  • Rana Tamim
  • Philip C. Abrami
  • C. Anne Wade
  • Michael A. Surkes
  • Gretchen Lowerison
Article
First Online:

Abstract

This paper reports the findings of a Stage I meta-analysis exploring the achievement effects of computer-based technology use in higher education classrooms (non-distance education). An extensive literature search revealed more than 6,000 potentially relevant primary empirical studies. Analysis of a representative sample of 231 studies (k = 310) yielded a weighted average effect size of 0.28 surrounded by wide variability. A mixed effects model was adopted to explore coded moderators of effect size. Research design was found to be not significant across true, quasi- and pre-experimental designs, so the designs were combined. The variable “degree of technology use” (i.e., low, medium, and high) was found to be significant, with low and medium use performing significantly higher than high use. For the variable “type of use” (i.e., cognitive support tools, presentational tools, and multiple uses), cognitive support (g+ = 0.40) was greater than presentational and multiple uses.

Keywords

Computer technology Higher education Meta-analysis 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

An earlier version of this paper was presented at the American Educational Research Association’s Annual Meeting, San Diego, CA on April 14, 2009. This study was supported by grants from the Social Sciences and Humanities Research Council of Canada to Schmid, Abrami and Bernard. The authors express appreciation to Christina Galofre, Carol Sparkes, Larysa Lysenko, Julia Barkhouse, Anna Sokolovskaya, Katherine Hanz, Nancy Acemian, Dai Zhang and Lucie A. Ranger. Contact: Dr. Richard F. Schmid, CSLP, LB-583-3, Department of Education, Concordia University, 1455 de Maisonneuve Blvd. W., Montreal, QC, Canada H3G 1M8. E-mail: schmid@education.concordia.ca, Website: http://doe.concordia.ca/cslp.

Supplementary material

12528_2009_9021_MOESM1_ESM.pdf (78 kb)
Supplementary material 1 (PDF 78 kb)

References

Note: A list of studies included in the meta-analysis is available upon request from the authors and online as electronic supplementary material.

  1. Abrami, P. C., & Bernard, R. M. (2006). Research on distance education: In defense of field experiments. Distance Education, 27(1), 5–26.CrossRefGoogle Scholar
  2. Abrami, P.C., & Bernard, R.M. (2009). Statistical control vs. classification of study quality in meta-analysis. Manuscript submitted for publication.Google Scholar
  3. Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educational Psychologist, 40(4), 199–209.CrossRefGoogle Scholar
  4. Azevedo, R., & Bernard, R. M. (1995). A meta-analysis of the effects of feedback in computer-based instruction. Journal of Educational Computing Research, 13(2), 111–127.Google Scholar
  5. Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., et al. (2004). How does distance education compare to classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research, 74(3), 379–439.CrossRefGoogle Scholar
  6. Bernard, R.M., Abrami, P.C., Borokhovski, E., Wade, C.A., Tamim, R., Surkes, M.A., & Bethel, E.C. (2009). A meta-analysis of three types of interaction treatments in distance education. Manuscript in press, Review of Educational Research. Google Scholar
  7. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. (2005). Comprehensive meta-analysis version 2. Englewood, NJ: Biostat.Google Scholar
  8. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. (2009). Introduction to meta-analysis. Chichester, UK: Wiley & Sons.CrossRefGoogle Scholar
  9. Campbell, D., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago, IL: Rand, McNally.Google Scholar
  10. Christmann, E. P., & Badgett, J. L. (2000). The comparative effectiveness of CAI on collegiate academic performance. Journal of Computing in Higher Education, 11(2), 91–103.CrossRefGoogle Scholar
  11. Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445–459.Google Scholar
  12. Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(3), 21–29.CrossRefGoogle Scholar
  13. Clark, R.E., Yates, K., Early, S., & Moulton, K. (2009). An analysis of the failure of electronic media and discovery-based learning: Evidence for the performance benefits of guided training methods. In K. H. Silber, & R. Foshay, (Eds.). Manuscript in press: Handbook of training and improving workplace performance, Volume I: Instructional design and training delivery. Somerset, NJ: Wiley.Google Scholar
  14. Cobb, T. (1997). Cognitive efficiency: Toward a revised theory of media. Educational Technology Research and Development, 45(4), 21–35.CrossRefGoogle Scholar
  15. Cohen, P. A., & Dacanay, L. S. (1992). Computer-based instruction and health professions education: A meta-analysis of outcomes. Evaluation and the Health Professions, 15(3), 259–281.CrossRefGoogle Scholar
  16. Dabbagh, N., & Kitsantas, A. (2005). Using web-based pedagogical tools as scaffolds for self-regulated learning. Instructional Science, 33(5–6), 513–540.CrossRefGoogle Scholar
  17. Fletcher-Flinn, C. M., & Gravatt, B. (1995). The efficacy of computer assisted instruction (CAI): A meta-analysis. Journal of Educational Computing Research, 12(3), 219–242.Google Scholar
  18. Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.Google Scholar
  19. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.Google Scholar
  20. Hedges, L.V., Shymansky, J.A., & Woodworth, G. (1989). A practical guide to modern methods of meta-analysis. (ERIC Document Reproduction Service No. ED 309 952).Google Scholar
  21. Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analysis. British Medical Journal, 327(7414), 557–560.CrossRefGoogle Scholar
  22. Hsu, Y.-c. (2003). The effectiveness of computer-assisted instruction in statistics education: A meta-analysis. Unpublished doctoral dissertation, University of Arizona.Google Scholar
  23. Jonassen, D. H., & Reeves, T. C. (1996). Learning with technology: Using computers as cognitive tools. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology. New York, NY: Scholastic Press.Google Scholar
  24. Jonassen, D. H., Howland, J., Moore, J., & Marra, R. M. (2003). Learning to solve problems with technology: A constructivist perspective (2nd ed.). Upper Saddle River, NJ: Merrill Prentice Hall.Google Scholar
  25. Koufogiannakis, D., & Wiebe, N. (2006). Effective methods for teaching information literacy skills to undergraduate students: A systematic review and meta-analysis. Evidence Based Library and Information Practice, 1(3), 3–43.Google Scholar
  26. Kozma, R. (1994). Will media influence learning? Reframing the debate. Educational Technology Research and Development, 42(2), 7–19.CrossRefGoogle Scholar
  27. Lou, Y., Abrami, P. C., & D’Appollonia, S. (2001). Small group and individual learning with technology: A meta-analysis. Review of Educational Research, 71, 449–521.CrossRefGoogle Scholar
  28. Lou, Y., Bernard, R. M., & Abrami, P. C. (2006). Media and pedagogy in undergraduate distance education: A theory-based meta-analysis of empirical literature. Educational Technology Research &Development, 54(2), 141–176.CrossRefGoogle Scholar
  29. Lowerison, G., Tamim, R., Nicolaidu, I., & Schmid, R. F. (2006, April). Implications of student technology use for knowledge building. Paper presented at the annual meeting of the American Educational Research Association, San Francisco: CA.Google Scholar
  30. Michko, G. M. (2007). A meta-analysis of the effects of teaching and learning with technology on student outcomes in undergraduate engineering education. Unpublished doctoral dissertation, University of Houston, Houston, TX.Google Scholar
  31. Rosen, Y., & Salomon, G. (2007). The differential learning achievements of constructivist technology-intensive learning environments as compared with traditional ones: A meta-analysis. Journal of Educational Computing Research, 36(1), 1–14.CrossRefGoogle Scholar
  32. Schenker, J. D. (2007). The effectiveness of technology use in statistics instruction in higher education: A meta-analysis using hierarchical linear modeling. Unpublished doctoral dissertation, Kent State University, Kent, OH.Google Scholar
  33. Schmid, R. F., Borokhovski, E., Tamim, R., Bernard, R. M., Abrami, P. C., Wade, C. A., Surkes, M. A., Lowerison, G., Lysenko, L., & Galofre. (2009). Effect of computer-based technology use in postsecondary education: Preliminary findings of a meta-analysis. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.Google Scholar
  34. Tamim, R. M. (2009). Effects of technology on students’ achievement: A second-order meta-analysis. Unpublished doctoral dissertation, Concordia University, Montreal, QC, Canada.Google Scholar
  35. Timmerman, C. E., & Kruepke, K. A. (2006). Computer-assisted instruction, media richness, and college student performance. Communication Education, 55, 73–104.CrossRefGoogle Scholar
  36. Zhao, Y. (2003). Recent developments in technology and language learning: A literature review and meta-analysis. CALICO Journal, 21(10), 7–27.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Centre for the Study of Learning and Performance (CSLP)Concordia UniversityMontrealCanada

Personalised recommendations

Cite article

Buy options