Education and Information Technologies

, Volume 23, Issue 3, pp 1377–1402 | Cite as

Ready for digital learning? A mixed-methods exploration of surveyed technology competencies and authentic performance activity

  • Todd J. B. BlayoneEmail author
  • Olena Mykhailenko
  • Roland vanOostveen
  • Wendy Barber


The Digital Competency Profiler (DCP) is an online application for surveying the technology preferences and abilities of students in higher education. To explore the DCP as a digital-learning-readiness tool, a mixed-methods research design was developed for relating self-reported digital competencies and online-learning activity. To this end, three authentic scenarios, comprised of six tasks mapped to self-report items, were constructed. Having submitted their survey data, each of 15 participants visited the EILAB to complete a randomly-assigned scenario with a tablet. Both the performance activity and post-activity interviews were recorded digitally using a unique activity-station setup, and task artefacts were gathered as performance outcomes. Analysis was conducted in three phases. In Phase 1, both the audio-video performance data and activity artefacts were coded, assessed and scored. Exploratory correlational analyses showed a pattern of positive relationships at the task and scenario levels for two scenario groups, suggesting some predictive value for the DCP in this context. For the third group, a positive correlation was found at the scenario level, but negative correlations were found at the task level. In Phase 2, detailed case-studies were conducted, incorporating self-report data, coded performance timelines, and post-activity interviews. Several situational influencers related to problem-solving strategy, device comfort, task difficulty and motivation, beyond the purview of the DCP, were identified. In Phase 3, the findings were interpreted to position the DCP as a tool for identifying segments of students with members who, without support, will likely struggle to engage fully in technology-rich learning environments.


Digital competence Digital skills Digital learning Online learning Readiness Digital readiness Observational study Higher education 


  1. Ala-Mutka, K. (2011). Mapping digital competence: Towards a conceptual understanding. In Seville: Institute for Prospective Technological Studies (IPTS). European Commission: Joint Research Centre Retrieved from Scholar
  2. Al-Araibi, A. A. M., Mahrin, M., & Mohd, R. C. (2016). A systematic literature review of technological factors for e-learning readiness in higher education. Journal of Theoretical and Applied Information Technology, 93(2), 500–521.Google Scholar
  3. Aparicio, M., Bacao, F., & Oliveira, T. (2016). An e-learning theoretical framework. Journal of Educational Technology & Society, 19(1), 292–307. Scholar
  4. Asselin, M., & Moayeri, M. (2010). New tools for new literacies research: An exploration of usability testing software. International Journal of Research & Method in Education, 33(1), 41–53. Scholar
  5. Aydın, C. H., & Tasci, D. (2005). Measuring readiness for e-learning: Reflections from an emerging country. Educational Technology and Society, 8(4), 244–257. Scholar
  6. Bandura, A. (1993). Perceived self-efficacy in cognitive development and functioning. Educational Psychologist, 28(2), 117–148. Scholar
  7. Beetham, H., & Sharpe, R. (2007). Rethinking pedagogy for a digital age: Designing for 21st century learning. New York: Routledge.Google Scholar
  8. Bertaux, D. (1981). From the life-history approach to the transformation of sociological practice. In D. Bertaux (Ed.), Biography and society: The life history approach in Social Sciences (pp. 29–45). London: Sage.Google Scholar
  9. Bhatt, I., & de Roock, R. (2014). Capturing the sociomateriality of digital literacy events. Research in learning technology, 21, 21281. Scholar
  10. Blayone, T., Mykhailenko, O., vanOostveen, R., Grebeshkov, O., Hrebeshkova, O., & Vostryakov, O. (2017a). Surveying digital competencies of university students and professors in Ukraine for fully online collaborative learning. Technology, Pedagogy and Education.
  11. Blayone, T., vanOostveen, R., Barber, W., DiGiuseppe, M., & Childs, E. (2017b). Democratizing digital learning: Theorizing the fully online learning community model. International Journal of Educational Technology in Higher Education, 14(1), 13. Scholar
  12. Bradlow, E. T., Hoch, S. J., & Hutchinson, J. W. (2002). An assessment of basic computer proficiency among active internet users: Test construction, calibration, antecedents and consequences. Journal of Educational and Behavioral Statistics, 27(3), 237–253. Scholar
  13. Bui, T. X., Sankaran, S., & Sebastian, I. M. (2003). A framework for measuring national e-readiness. International Journal of Electronic Business, 1(1), 3–22. Scholar
  14. Crompton, H., Burke, D., Gregory, K. H., & Gräbe, C. (2016). The use of mobile learning in science: A systematic review. Journal of Science Education and Technology, 1–12.
  15. Deci, E. L., & Ryan, R. M. (2000). The" what" and" why" of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11(4), 227–268. Scholar
  16. Demir, Ö., & Yurdugül, H. (2015). The exploration of models regarding e-learning readiness: Reference model suggestions. International Journal of Progressive Education, 11(1), 173–194.Google Scholar
  17. Desjardins, F. J. (2005). Information and communication technology in education: A competency profile of francophone secondary school teachers in Ontario. Canadian Journal of Learning and Technology/La revue canadienne de l’apprentissage et de la technologie, 31(1), 1–14. Scholar
  18. Desjardins, F. J., & Peters, M. (2007). Single-course approach versus a program approach to develop technological competencies in pre-service language teaching. In M.-A. Kassen, L. Lavine, K. Murphy-Judy, & M. Peters (Eds.), Preparing and developing technology proficient L2 teachers (pp. 3–21). Texas: Texas State University.Google Scholar
  19. Desjardins, F. J., & vanOostveen, R. (2015). Faculty and student use of digital technology in a "laptop" university. In S. Carliner, C. Fulford, & N. Ostashewski (Eds.), EdMedia: World Conference on Educational Media and Technology 2015 (pp. 990-996). Montreal: Association for the Advancement of Computing in Education (AACE).Google Scholar
  20. Desjardins, F. J., Lacasse, R., & Belair, L. M. (2001). Toward a definition of four orders of competency for the use of information and communication technology (ICT) in education. Paper presented at the computers and advanced Technology in Education. Canada: Banff Scholar
  21. Desjardins, F. J., vanOostveen, R., Bullock, S., DiGiuseppe, M., & Robertson, L. (2010). Exploring graduate student’s use of computer-based technologies for online learning. In J. Herrington & C. Montgomerie (Eds.), EdMedia: World Conference on Educational Media and Technology 2010 (pp. 440-444). Norfolk: Association for the Advancement of Computing in Education (AACE).Google Scholar
  22. DiGiuseppe, M., Partosoedarso, E., vanOostveen, R., & Desjardins, F. J. (2013). Exploring competency development with mobile devices. In M. B. Nunes & M. McPherson (Eds.), International Association for Development of the information society (IADIS) international conference on e-learning (pp. 384–388). Prague: International Association for Development of the Information Society.Google Scholar
  23. Ding, R., & Ma, F. (2013). Assessment of university student web searching competency by a task-based online test: A case study at Wuhan University, China. The Electronic Library, 31(3), 359–375. Scholar
  24. Dray, B. J., Lowenthal, P. R., Miszkiewicz, M. J., Ruiz-Primo, M. A., & Marczynski, K. (2011). Developing an instrument to assess student readiness for online learning: A validation study. Distance Education, 32(1), 29–47. Scholar
  25. Esbjörnsson, M., Brown, B., Juhlin, O., Normark, D., Östergren, M., & Laurier, E. (2006). Watching the cars go round and round: Designing for active spectating. In. In R. Grinter, T. Rodden, P. Aoki, E. Cutrell, R. Jeffries, & G. Olson (Eds.), Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1221–1224). New York: ACM.Google Scholar
  26. Eshet-Alkalai, Y., & Amichai-Hamburger, Y. (2004). Experiments in digital literacy. Cyberpsychology & Behavior, 7(4), 421–429.CrossRefGoogle Scholar
  27. Farid, A. (2014). Student online readiness assessment tools: A systematic review approach. Electronic Journal of e-Learning, 12(4), 375–382.Google Scholar
  28. Garrison, D. R., Cleveland-Innes, M., Koole, M., & Kappelman, J. (2006). Revisiting methodological issues in transcript analysis: Negotiated coding and reliability. The Internet and Higher Education, 9(1), 1–8. Scholar
  29. Gay, G. (2016). An assessment of online instructor e-learning readiness before, during, and after course delivery. Journal of Computing in Higher Education, 28(2), 199–220. Scholar
  30. Greene, J. A., Seung, B. Y., & Copeland, D. Z. (2014). Measuring critical components of digital literacy and their relationships with learning. Computers & Education, 76, 55–69. Scholar
  31. Hargittai, E. (2002). Beyond logs and surveys: In-depth measures of people's web use skills. Journal of the American Society for Information Science and Technology, 53(14), 1239–1244. Scholar
  32. Hargittai, E., & Shafer, S. (2006). Differences in actual and perceived online skills: The role of gender. Social Science Quarterly, 87(2), 432–448. Scholar
  33. Herrington, J., Reeves, T. C., & Oliver, R. (2006). Authentic tasks online: A synergy among learner, task, and technology. Distance Education, 27(2), 233–247. Scholar
  34. Horzum, M. B., Kaymak, Z. D., & Gungoren, O. C. (2015). Structural equation modeling towards online learning readiness, academic motivations, and perceived learning. Educational Sciences: Theory and Practice, 15(3), 759–770.  10.12738/estp.2015.3.2410.Google Scholar
  35. Hung, M.-L. (2016). Teacher readiness for online learning: Scale development and teacher perceptions. Computers & Education, 94, 120–133. Scholar
  36. Hung, M.-L., Chou, C., & Chen, C.-H. (2010). Learner readiness for online learning: Scale development and student perceptions. Computers & Education, 55(3), 1080–1090. Scholar
  37. IEEE. (1990). IEEE standard computer dictionary: A compilation of IEEE standard computer glossaries. In (pp. 218). New York: The Institute of Electrical and Electronics Engineers.Google Scholar
  38. Jayroe, T. J., & Wolfram, D. (2012). Internet searching, tablet technology and older adults. Proceedings of the American Society for Information Science and Technology, 49(1), 1–3. Scholar
  39. Knoblauch, H. (2012). Introduction to the special issue of qualitative research: Video-analysis and videography. Qualitative Research, 12(3), 251–254. Scholar
  40. Leigh, D., & Watkins, R. (2005). E-learner success: Validating a self-assessment of learner readiness for online training. In. In ASTD 2005 research-to-practice conference proceedings (pp. 121–131). Alexandria: ATD.Google Scholar
  41. Lin, H.-H., Lin, S., Yeh, C.-H., Wang, Y.-S., & Jansen, J. (2015). Measuring mobile learning readiness: Scale development and validation. Internet Research, 26(1), 265–287. Scholar
  42. Litt, E. (2013). Measuring users’ internet skills: A review of past assessments and a look toward the future. New Media & Society, 15(4), 612–630. Scholar
  43. Mason, M. (2010). Sample size and saturation in PhD studies using qualitative interviews. Forum qualitative Sozialforschung/Forum: Qualitative Social Research, 11(3), 1–5.MathSciNetGoogle Scholar
  44. Merriam, S. B. (1998). Qualitative research and case study applications in education. Revised and expanded from case study research in education. San Francisco: Josey-Bass Publishers.Google Scholar
  45. Mosa, A. A., Naz’ri bin Mahrin, M., & Ibrrahim, R. (2016). Technological aspects of e-learning readiness in higher education: A review of the literature. Computer and Information Science, 9(1), 113–127. Scholar
  46. Parasuraman, A. (2000). Technology readiness index (TRI) a multiple-item scale to measure readiness to embrace new technologies. Journal of Service Research, 2(4), 307–320. Scholar
  47. Park, Y. J. (2015). My whole world’s in my palm! The second-level divide of teenagers’ mobile use and skill. New Media & Society, 17(6), 977–995. Scholar
  48. Parkes, M., Stein, S., & Reading, C. (2015). Student preparedness for university e-learning environments. The Internet and Higher Education, 25, 1–10. Scholar
  49. Rourke, L., & Anderson, T. (2004). Validity in quantitative content analysis. Educational Technology Research and Development, 52(1), 5–18. Scholar
  50. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 12, 8–22.Google Scholar
  51. Savin-Baden, M. (2000). Problem-based learning in higher education: Untold stories. Philadelphia: Open University Press.Google Scholar
  52. Siemens, G., Gašević, D., & Dawson, S. (2015). Preparing for the digital university: A review of the history and current state of distance, blended, and online Learning Retrieved from
  53. Sun, X., & May, A. (2013). A comparison of field-based and lab-based experiments to evaluate user experience of personalised mobile devices. Advances in Human-Computer Interaction, 2013, 2. Scholar
  54. van Deursen, A. J. A. M. (2010). Internet skills: Vital assets in an information society. (Ph.D. Thesis), University of Twente, Enschede, the Netherlands. Retrieved from
  55. van Deursen, A. J. A. M., & van Dijk, J. A. G. M. (2010). Measuring internet skills. International Journal of Human-Computer Interaction, 26(10), 891–916. Scholar
  56. van Deursen, A. J. A. M., Helsper, E. J., & Eynon, R. (2015). Development and validation of the internet skills scale (ISS). Information, Communication & Society, 1–20.
  57. vanOostveen, R., DiGiuseppe, M., Barber, W., Blayone, T., & Childs, E. (2016). New conceptions for digital technology sandboxes: Developing a fully online learning communities (FOLC) model. In G. Veletsianos (Ed.), EdMedia 2016: World conference on educational media and technology (pp. 665–673). Vancouver: Association for the Advancement of Computing in Education (AACE).Google Scholar
  58. Watkins, R., Leigh, D., & Triner, D. (2004). Assessing readiness for e-learning. Performance Improvement Quarterly, 17(4), 66–79. Scholar
  59. Wilhelm, J. (2016). What is the minimum sample size to run Pearsons R? (Online Expert Database). Retrieved June 7, 2017, from ResearchGate:

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Todd J. B. Blayone
    • 1
    Email author
  • Olena Mykhailenko
    • 1
  • Roland vanOostveen
    • 2
  • Wendy Barber
    • 2
  1. and Educational Informatics Laboratory (EILAB)OshawaCanada
  2. 2.Faculty of Education and Educational Informatics Laboratory (EILAB), University of Ontario Institute of TechnologyOshawaCanada

Personalised recommendations