Journal of Computing in Higher Education

, Volume 24, Issue 3, pp 164–181 | Cite as

Re-envisioning instructional technology research in higher education environments: a content analysis of a grant program

  • Trena M. Paulus
  • Gina Phipps
  • John Harrison
  • Mary Alice Varga


Within the field of instructional technology, scholars have long worked to define the scope and purpose of research and its role in informing practice. Increasingly, researchers outside of the instructional technology field are conducting studies to examine their use of technology in educational contexts. Few studies have been done on how researchers in other disciplines are designing such studies. We conducted a content analysis of 60 proposals submitted from 2006 to 2010 to our internal grant competition for faculty research on instructional technology to better understand the kinds of studies being proposed. Categories explored within each proposal included academic discipline, collaboration, knowledge of previous literature, context, goals of study, and research design. A majority of proposals came from outside of the education field and were submitted by individuals rather than collaborative teams. Just under half of the proposals cited previous literature to justify their study, and just over half sought to examine classroom contexts. Roughly a third proposed to study distance education contexts. Most proposals were to examine the implementation of a new instructional strategy (rather than to conduct a media comparison study) and just over half utilized a quantitative research design collecting performance or satisfaction data. We include recommendations for those who may be interested in how better to support researchers in designing effective studies to investigate instructional technology use, highlighting the use of design-based research as a viable methodology.


Instructional technology research Faculty development Content analysis Media comparison Research design Design-based research 


  1. Aspelmeier, J. E., & Pierce, T. W. (2009). SPSS: A user-friendly approach. New York, NY: Worth Publishers.Google Scholar
  2. Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1–14.CrossRefGoogle Scholar
  3. Bauer, M. (2000). Classical content analysis: A review. In M. Bauer & G. Gaskell (Eds.), Qualitative researching with text, image and sound (pp. 131–151). London: Sage.Google Scholar
  4. Bekele, T. A., & Menchaca, M. P. (2008). Research on internet-supported learning: A review. The Quarterly Review of Distance Education, 9(4), 373–405.Google Scholar
  5. Clark, R. E. (1983). Reconsidering the research on learning from media. Review of Educational Research, 53(4), 445–459.Google Scholar
  6. Clark, R. E. (1994a). Media will never influence learning. Educational Technology Research and Development, 42(2), 21–29.CrossRefGoogle Scholar
  7. Clark, R. E. (1994b). Media and method. Educational Technology Research and Development, 42(3), 7–10.CrossRefGoogle Scholar
  8. Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2–3), 87–105.Google Scholar
  9. Greenhow, C., Robelia, B., & Hughes, J. E. (2009). Web 2.0 and classroom research: What path should we take now? Educational Researcher, 38(4), 246–259.CrossRefGoogle Scholar
  10. Hew, K. F., Kale, U., & Kim, N. (2007). Past research in instructional technology: Results of a content analysis of empirical studies published in three prominent instructional technology journals from the year 2000 through 2004. Journal of Educational Computing Research, 36(3), 269–300.CrossRefGoogle Scholar
  11. Higgins, N., Sullivan, H., Harper-Marinick, M., & Lopez, C. (1989). Perspectives on educational technology research and development. Educational Technology Research & Development, 37(1), 7–17.CrossRefGoogle Scholar
  12. Hrastinski, S., & Keller, C. (2007a). An examination of research approaches that underlie research on educational technology: A review from 2000 to 2004. Journal of Educational Computing Research, 36(2), 175–190.CrossRefGoogle Scholar
  13. Hrastinski, S., & Keller, C. (2007b). Computer-mediated communication in education: A review of recent research. Educational Media International, 44(1), 61–77.CrossRefGoogle Scholar
  14. Hsieh, P.-H., Hsieh, Y.-P., Chung, W.-H., Acee, T., Thoman, G. D., Kim, H.-J., et al. (2005). Is educational intervention research on the decline? Journal of Educational Psychology, 97, 523–529.CrossRefGoogle Scholar
  15. Kirby, J. A., Hoadley, C. M., & Carr-Chellman, A. A. (2005). Instructional systems design and the learning sciences: A citation analysis. Educational Technology Research and Development, 53(1), 37–48.CrossRefGoogle Scholar
  16. Klein, D. J. (1997). ETR&D-Development: An analysis of content and survey of future direction. Educational Technology Research and Development, 45(3), 57–62.CrossRefGoogle Scholar
  17. Kozma, R. B. (1994a). Will media influence learning? Reframing the debate. Educational Technology Research and Development, 42(2), 7–19.CrossRefGoogle Scholar
  18. Kozma, R. B. (1994b). A reply: Media and methods. Educational Technology Research and Development, 42(3), 11–14.CrossRefGoogle Scholar
  19. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.CrossRefGoogle Scholar
  20. National Research Council. (2002). Scientific research in education. Washington, DC: National Academy Press.Google Scholar
  21. Oh, E., & Reeves, T. C. (2010). The implications of the differences between design research and instructional systems design for educational technology researchers and practitioners. Educational Media International, 47(4), 263–275.CrossRefGoogle Scholar
  22. Reeves, T. C. (1995). Questioning the questions of instructional technology research. In M. R. Simonson & M. Anderson (Eds.), Proceedings of the Annual Conference of the Association for Educational Communications and Technology, Research and Theory Division (pp. 459–470). Anaheim, CA.Google Scholar
  23. Reeves, T. C. (2000). Enhancing the worth of instructional technology research through “design experiments” and other development research strategies. New Orleans, LA: Paper presented at the Annual Meeting of the American Educational Research Association.Google Scholar
  24. Reeves, T. C., Herrington, J., & Oliver, R. (2005). Design research: A socially responsible approach to instructional technology research in higher education. Journal of Computing in Higher Education, 16(2), 97–116.CrossRefGoogle Scholar
  25. Ross, S. M., & Morrison, G. R. (2004). Experimental research methods. In D. J. Jonassen (Ed.), Handbook of research on educational communications and technology (2nd ed., pp. 1021–1043). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.Google Scholar
  26. Ross, S. M., Morrison, G. R., Hannafin, R. D., Young, M., van den Akker, J., Kuiper, W., et al. (2008). Research designs. In J. M. Spector, M. D. Merrill, J. van Merrienboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 715–761). New York, NY: Lawrence Erlbaum Associates Taylor & Francis Group.Google Scholar
  27. Ross, S. M., Morrison, G. R., & Lowther, D. L. (2010). Educational technology research past and present: Balancing rigor and relevance to impact school learning. Contemporary Educational Technology, 1(1), 17–35.Google Scholar
  28. Rourke, L., & Kanuka, H. (2009). Learning in communities of inquiry: A review of the literature. Journal of Distance Education, 23(1), 19–48.Google Scholar
  29. Rourke, L. & Szabo, M. (2002). A content analysis of The Journal of Distance Education, 1986–2001. The Journal of Distance Education, 17(1), 63–74. Retrieved from Scholar
  30. Russell, T. (1999). The no significant difference phenomenon: A comparative research annotated bibliography on technology for distance education. Raleigh, NC: North Carolina State University Press.Google Scholar
  31. Schmid, R. F., Bernard, R. M., Borokhovski, E., Tamim, R., Abrami, P. C., Wade, A., et al. (2009). Technology’s effect on achievement in higher education: A Stage I meta-analysis of classroom applications. Journal of Computing in Higher Education, 21, 95–109.CrossRefGoogle Scholar
  32. Shih, M., Feng, J., & Tsai, C. (2008). Research and trends in the field of e-learning from 2001 to 2005: A content analysis of cognitive studies in selected journals. Computers & Education, 51, 955–967.CrossRefGoogle Scholar
  33. Surry, D. W., & Ensminger, D. (2001). What’s wrong with media comparison studies? Educational Technology, 31(4), 32–35.Google Scholar
  34. Tamim, R., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational Research, 81, 4–28.CrossRefGoogle Scholar
  35. Warnick, B. R., & Burbules, N. C. (2007). Media comparison studies: Problems and possibilities. Teachers College Record, 109(11), 2483–2510.Google Scholar
  36. Winn, W. (2002). Current trends in educational technology research: The study of learning environments. Educational Psychology Review, 14(3), 331–351.CrossRefGoogle Scholar
  37. Wolff, W. I. (2008). “A chimera of sorts”: Rethinking educational technology grant programs, courseware innovation, and the language of educational change. Computers & Education, 51, 1184–1197.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Trena M. Paulus
    • 1
  • Gina Phipps
    • 2
  • John Harrison
    • 3
  • Mary Alice Varga
    • 4
  1. 1.Department of Educational Psychology and CounselingUniversity of TennesseeKnoxvilleUSA
  2. 2.Teaching and Technology CenterAiken Technical CollegeAikenUSA
  3. 3.Carter and Moyers School of EducationLincoln Memorial UniversityHarrogateUSA
  4. 4.Educational Psychology and CounselingUniversity of TennesseeKnoxvilleUSA

Personalised recommendations