Looking Back, Moving Forward: Impact and Measurability of the Use of Educational Technology

Living reference work entry
Part of the Springer International Handbooks of Education book series (SIHE)

Abstract

In this chapter, we carry out a critical review of the various historical analyses of the impact of technological interventions in education. The purpose is to analytically explore and learn from some of the methodological limitations and strengths of the approaches adopted to measure and capture the impact of educational technology. This retrospective examination is then used to explicate methodological design principles that can increase the use and value of research evidence regarding the impact of educational technology. Capturing and understanding the impact of digital technologies in and on learning is inherently problematic. It is exacerbated by the continually developing nature of digital technologies and their formal and informal boundary crossing. We posit methodological design principles that are sympathetic to the fact that evidence of the pedagogical application of digital technologies is both borne out of, and brought to bear in, complex and dynamic contexts that are mediated by, and impact upon, the various ways in which technologies are appropriated for educational purpose. The chapter concludes with a call for methodological perspectives that are not confined by paradigm, but are able to bridge and integrate research paradigms in order to respond to the complex sociocultural ecologies within which digital technologies are implicated.

Keywords

Educational technology Impact Methodological innovation Design principles 

References

  1. Alexander, R. (2015). Teaching and learning for all? The quality imperative revisited. International Journal of Educational Development, 40, 250–258.CrossRefGoogle Scholar
  2. Bachmair, B., & Pachler, N. (2015). Sustainability for innovative education – The case of mobile learning. Journal of Interactive Media in Education, 1(17), 1–12.  https://doi.org/10.5334/jime.ay.Google Scholar
  3. Bai, Y., Mo, D., Zhang, L., Boswell, M., & Rozelle, S. (2016). The impact of integrated ICT with teaching: Evidence from a randomized controlled trial in rural schools in China. Computers & Education, 96, 1–14.  https://doi.org/10.1016/j.compedu.2016.02.005.CrossRefGoogle Scholar
  4. Bebell, D., O’Dwyer, L., Russell, M., & Hoffmann, T. (2007) Advancing data collection in the digital age: Methodological challenges and solutions in educational technology research. In Paper presented at the annual meeting of the American Educational Research Association, Chicago. http://www.bc.edu/research/intasc/PDF/Methodological%20challenges_v2.2.pdf
  5. Beland, L., & Murphy, R. (2015). Communication: Technology, distraction & student performance. In Discussion paper no 1350. LSE: Centre for Economic Performance.Google Scholar
  6. Bruce, B., & Hogan, M. (1998). The disappearance of technology: Toward an ecological model of literacy. In D. Reinking, M. McKenna, L. Labbo, & R. Kieffer (Eds.), Handbook of literacy and technology: Transformations in a post-typographic world (pp. 269–281). Mahwah: Lawrence Erlbaum Associates.Google Scholar
  7. Chauhan, S. (2017). A meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education, 105, 14–30.CrossRefGoogle Scholar
  8. Cox, M. (2013). Formal to informal learning with IT: Research challenges and issues for e-learning. Journal of Computer Assisted Learning, 29(1), 85–105.CrossRefGoogle Scholar
  9. Cox, M. (in press). Researching IT in education: Meeting the challenges of an ever-changing environment. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), Second handbook of information technology in primary and secondary education. Cham: Springer International Publishing.Google Scholar
  10. Cox, M., & Abbott, C. (2004). ICT and attainment: A review of the research literature. Coventry: Becta.Google Scholar
  11. Cox, M., & Marshall, G. (2007). Effects of ICT: Do we know what we should know? Education and Information Technologies, 12(2), 59–70.CrossRefGoogle Scholar
  12. Cox, M., Webb, M., Abbott, C., Blakeley, B., Beauchamp, T., & Rhodes, V. (2004). ICT and pedagogy: A review of the research literature. Coventry: Becta.Google Scholar
  13. Crook, C., & Garratt, D. (2011). The positivist paradigm in contemporary social research: The interface of psychology method and socio-cultural theory. In B. Somekh & C. Lewin (Eds.), Theory and methods in social research (2nd ed., pp. 212–219). London: Sage.Google Scholar
  14. Crook, C., Harrison, C., Farrington-Flint, L., Tomás, C., & Underwood, J. (2010). The impact of technology: Value-added classroom practice. Final report. Coventry: Becta.Google Scholar
  15. Definition and Terminology Committee of the Association for Educational Communications and Technology. (2008). Definition. In A. Januszewski & M. Molena (Eds.), Educational technology: A definition with commentary (pp. 1–14). New York: Routledge.Google Scholar
  16. Ertmer, P., Addison, P., Lane, M., Ross, E., & Woods, D. (1999). Examining teachers’ beliefs about the role of technology in the elementary classroom. Journal of Research on Computing in Education, 32(1), 54–71.CrossRefGoogle Scholar
  17. Harrison, C., Comber, C., Fisher, T., Haw, K., Lewin, C., Lunzer, E., McFarlane, A., Mavers, D., Scrimshaw, P., Somekh, B., & Watling, R. (2002). ImpacCT2: The impact of information and communication technologies on pupil learning and attainment. London: DfES and Becta. http://dera.ioe.ac.uk/1572/.Google Scholar
  18. Haßler, B., Major, L., & Hennessy, S. (2015). Tablet use in schools: A critical review of the evidence for learning outcomes. Journal of Computer Assisted Learning, 32(2), 139–156.CrossRefGoogle Scholar
  19. Hattie, J. (2009). Visible learning; A synthesis of over 800 meta analyses relating to achievement. London: Routledge.Google Scholar
  20. Hew, K., & Brush, T. (2007). Integrating technology into K-12 teaching and learning: Current knowledge gaps and recommendations for future research. Educational Technology Research and Development, 55(3), 223–252.CrossRefGoogle Scholar
  21. Higgins, S., Xiao, Z., Katsipataki, M. (2012). The impact of digital technology on learning: Summary for the education endowment foundation. Full report. https://educationendowmentfoundation.org.uk/evidence/teaching-learning-toolkit/digital-technology/
  22. ICF Consulting. (2015). Literature review on the impact of technology on learning and teaching. Edinburgh: The Scottish Government. http://dera.ioe.ac.uk/id/eprint/24843
  23. Jenkinson, J. (2009). Measuring the effectiveness of educational technology: What are we attempting to measure? Electronic Journal of e-learning, 7(3), 273–280.Google Scholar
  24. Kalantzis, M., & Cope, B. (2004). Designs for learning. e-Learning, 1(1), 38–93.Google Scholar
  25. Kirkwood, A., & Price, L. (2014). Technology-enhanced learning and teaching in higher education: What is ‘enhanced’ and how do we know? A critical literature review. Learning, Media and Technology, 39(1), 6–36.CrossRefGoogle Scholar
  26. Kirschner, P. (2006). (Inter)dependent learning: Learning is interaction. Utrecht: Inaugural professorial address. http://dspace.learningnetworks.org/bitstream/1820/1619/1/OratieKirschner-LearningisInteraction.pdf.Google Scholar
  27. Kirschner, P., & Kester, L. (2016). Towards a research agenda for educational technology research. In N. Rushby & D. Surry (Eds.), The Wiley handbook of learning technology (pp. 523–541). Hoboken: Wiley.CrossRefGoogle Scholar
  28. Kjeldskov, J., & Graham, C. (2003). A review of Mobile HCI research methods. In Lecture notes in computer science: Human-computer interaction with mobile devices. 5th International Symposium, Mobile HCI 2003 (pp. 317–335). Berlin: Springer.Google Scholar
  29. Latchem, C. (2014). Opening up the educational technology research agenda. British Journal of Educational Technology, 45(1), 3–11.CrossRefGoogle Scholar
  30. Laurillard, D. (2012). Teaching as a design science: Building pedagogical patterns for learning and technology. New York/Oxon: Routledge.Google Scholar
  31. Laurillard, D. (in press). Teaching as a design science: Teachers building, testing and sharing pedagogic ideas. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), Second handbook of information technology in primary and secondary education. Cham: Springer International Publishing.Google Scholar
  32. Liao, Y.-K. C., & Lai, W.-C. (in press). Meta-analyses of large scale data sets: A tool for assessing the impact of ICT in education. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), Second handbook of information technology in primary and secondary education. Cham: Springer International Publishing.Google Scholar
  33. Livingstone, S. (2012). Critical reflections on the benefits of ICT in education. Oxford Review of Education, 38(1), 9–24.CrossRefGoogle Scholar
  34. Luckin, R., Bligh, B., Manches, A., Ainsworth, S., Crook, C., & Noss, R. (2012). Decoding learning: The proof, promise and potential of digital education. London: Nesta. http://www.nesta.org.uk/sites/default/files/decoding_learning_report.pdf.Google Scholar
  35. Mor, Y., Ferguson, R., & Wasson, B. (2015). Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology, 46(2), 221–229.CrossRefGoogle Scholar
  36. OECD. (2015). Students, computers and learning. Making the connection, PISA. Paris: OECD Publishing.CrossRefGoogle Scholar
  37. Pachler, N. (2009). Research methods in mobile and informal learning: Some issues. In G. Vavoula, N. Pachler, & A. Kukulska-Hulme (Eds.), Researching mobile learning: Frameworks, tools and research design (pp. 1–15). Oxford: Peter Lang.Google Scholar
  38. Pachler, N., & Daly, C. (2009). Narrative and learning with web 2.0 technologies: Towards a research agenda. Journal of Computer Assisted Learning, 25, 6–18.CrossRefGoogle Scholar
  39. Pachler, N., Cook, J., & Bradley, C. (2009). “I don’t really see it”: Whither case-based approaches to understanding off-site and on-campus mobile learning? In G. Vavoula, N. Pachler, & A. Kukulska-Hulme (Eds.), Researching mobile learning: Frameworks, tools and research designs (pp. 77–95). Oxford: Peter Lang Publishing.Google Scholar
  40. Pachler, N., Bachmair, B., & Cook, J. (2010). Mobile learning: Structures, agency, practices. New York: Springer.CrossRefGoogle Scholar
  41. Pérez-Sanagustín, M., Nussbaum, M., Hillinger, I., Alario-Hoyos, C., Heller, R., Twining, P., & Tsai, C.-C. (2017). Research on ICT in K-12 schools – A review of experimental and survey-based studies in computers & education 2011–2015. Computers & Education, 104, A1–A15.CrossRefGoogle Scholar
  42. Persico, D., & Pozzi, F. (2015). Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46(2), 230–248.CrossRefGoogle Scholar
  43. Reeves, T., & Oh, E. (2016). The goals and methods of educational technology research over a quarter century (1989-2014). Educational Technology Research and Development, 65(2), 325–339.  https://doi.org/10.1007/s11423-016-9474-1.CrossRefGoogle Scholar
  44. Ross, S., Morrison, G., & Lowther, D. (2010). Educational technology research past and present: Balancing rigor and relevance to impact school learning. Contemporary Educational Technology, 1(1), 17–35.Google Scholar
  45. Scardamalia, M. (2006). Technology for understanding. In K. Leithwood, P. McAdie, N. Bascia, & A. Rodrigue (Eds.), Teaching for deep understanding. What every educator needs to know (pp. 103–109). Thousand Oaks: Corwin Press.Google Scholar
  46. Sibbald, B., & Roland, M. (1998). Understanding controlled trials: Why are randomised controlled trials important? BMJ, 316, 201.  https://doi.org/10.1136/bmj.316.7126.201.CrossRefGoogle Scholar
  47. Somekh, B. (2007). Pedagogy and learning with ICT: Researching the art of innovation. London: Routledge.Google Scholar
  48. Somekh, B., Lewin, C., Mavers, D., Fisher, T., Harrison, C., Haw, K., Lunzer, E., McFarlane, A., & Scrimshaw, P. (2002). ImpaCT2: Pupils’ and teachers’ perceptions in the home, school and community. Coventry: Becta. http://dera.ioe.ac.uk/1573/.Google Scholar
  49. Tamim, R., Bernard, R., Borokhovski, E., Abrami, P., & Schmid, R. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational Research, 81(1), 4–28.CrossRefGoogle Scholar
  50. Turkle, S. (1995). Life on the screen: Identity in the age of the internet. New York: Simon & Schuster.Google Scholar
  51. Turvey, K. (2012). Constructing narrative ecologies as a site for teachers’ professional learning with new technologies and media in primary education. E-learning & Digital Media, 9(1), 113–126.CrossRefGoogle Scholar
  52. Turvey, K., & Pachler, N. (2016). Problem spaces’: A framework and questions for critical engagement with learning technologies in formal educational contexts. In N. Rushby & D. Surry (Eds.), The Wiley handbook of learning technology (pp. 113–130). New York: Wiley-Blackwell.CrossRefGoogle Scholar
  53. Twining, P., Heller, R., Nussbaum, M., & Tsai, C.-C. (2017). Some guidance on conducting and reporting qualitative studies. Computers & Education, 106, A1–A9.  https://doi.org/10.1016/j.compedu.2016.12.002.CrossRefGoogle Scholar
  54. UNESCO [United Nations Educational, Scientific and Cultural Organisation]. (2014). Teaching and learning: Achieving quality for all. In EFA global monitoring report. Paris: UNESCO. http://en.unesco.org/gem-report/report/2014/teaching-and-learning-achieving-quality-all#sthash.2hVKuzp5.dpbs.
  55. VasabØ, K., & Gudmundsdottir, G. (2014). Methodological challenges when exploring new learning sites in educational research. In G. Gudmundsdottir & K. VasabØ (Eds.), Methodological challenges when exploring digital learning spaces in education (pp. 1–12). Rotterdam: Sense Publishers.Google Scholar
  56. Voogt, J., & Knezek, G. (2008). IT in primary and secondary education: Emerging issues. In J. Voogt & G. Knezek (Eds.), International handbook of information technology in primary and secondary education (Vol. 1, pp. 117–132). New York: Springer.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute of EducationUniversity College LondonLondonUK
  2. 2.University of BrightonBrightonUK

Section editors and affiliations

  • Margaret Cox
    • 1
  • Joke Voogt
    • 2
  1. 1.King's College LondonLondonUK
  2. 2.Department of Child Development and EducationUniversity of Amsterdam/ Windesheim University of Applied SciencesAmsterdamThe Netherlands

Personalised recommendations