Advertisement

Education and Information Technologies

, Volume 23, Issue 3, pp 1377–1402 | Cite as

Ready for digital learning? A mixed-methods exploration of surveyed technology competencies and authentic performance activity

  • Todd J. B. Blayone
  • Olena Mykhailenko
  • Roland vanOostveen
  • Wendy Barber
Article

Abstract

The Digital Competency Profiler (DCP) is an online application for surveying the technology preferences and abilities of students in higher education. To explore the DCP as a digital-learning-readiness tool, a mixed-methods research design was developed for relating self-reported digital competencies and online-learning activity. To this end, three authentic scenarios, comprised of six tasks mapped to self-report items, were constructed. Having submitted their survey data, each of 15 participants visited the EILAB to complete a randomly-assigned scenario with a tablet. Both the performance activity and post-activity interviews were recorded digitally using a unique activity-station setup, and task artefacts were gathered as performance outcomes. Analysis was conducted in three phases. In Phase 1, both the audio-video performance data and activity artefacts were coded, assessed and scored. Exploratory correlational analyses showed a pattern of positive relationships at the task and scenario levels for two scenario groups, suggesting some predictive value for the DCP in this context. For the third group, a positive correlation was found at the scenario level, but negative correlations were found at the task level. In Phase 2, detailed case-studies were conducted, incorporating self-report data, coded performance timelines, and post-activity interviews. Several situational influencers related to problem-solving strategy, device comfort, task difficulty and motivation, beyond the purview of the DCP, were identified. In Phase 3, the findings were interpreted to position the DCP as a tool for identifying segments of students with members who, without support, will likely struggle to engage fully in technology-rich learning environments.

Keywords

Digital competence Digital skills Digital learning Online learning Readiness Digital readiness Observational study Higher education 

References

  1. Ala-Mutka, K. (2011). Mapping digital competence: Towards a conceptual understanding. In Seville: Institute for Prospective Technological Studies (IPTS). European Commission: Joint Research Centre Retrieved from http://ftp.jrc.es/EURdoc/JRC67075_TN.pdf.Google Scholar
  2. Al-Araibi, A. A. M., Mahrin, M., & Mohd, R. C. (2016). A systematic literature review of technological factors for e-learning readiness in higher education. Journal of Theoretical and Applied Information Technology, 93(2), 500–521.Google Scholar
  3. Aparicio, M., Bacao, F., & Oliveira, T. (2016). An e-learning theoretical framework. Journal of Educational Technology & Society, 19(1), 292–307. http://www.jstor.org/stable/10.2307/jeductechsoci.19.1.292Google Scholar
  4. Asselin, M., & Moayeri, M. (2010). New tools for new literacies research: An exploration of usability testing software. International Journal of Research & Method in Education, 33(1), 41–53.  https://doi.org/10.1080/17437271003597923.CrossRefGoogle Scholar
  5. Aydın, C. H., & Tasci, D. (2005). Measuring readiness for e-learning: Reflections from an emerging country. Educational Technology and Society, 8(4), 244–257. http://www.jstor.org/stable/jeductechsoci.8.4.244Google Scholar
  6. Bandura, A. (1993). Perceived self-efficacy in cognitive development and functioning. Educational Psychologist, 28(2), 117–148.  https://doi.org/10.1207/s15326985ep2802_3.CrossRefGoogle Scholar
  7. Beetham, H., & Sharpe, R. (2007). Rethinking pedagogy for a digital age: Designing for 21st century learning. New York: Routledge.Google Scholar
  8. Bertaux, D. (1981). From the life-history approach to the transformation of sociological practice. In D. Bertaux (Ed.), Biography and society: The life history approach in Social Sciences (pp. 29–45). London: Sage.Google Scholar
  9. Bhatt, I., & de Roock, R. (2014). Capturing the sociomateriality of digital literacy events. Research in learning technology, 21, 21281.  https://doi.org/10.3402/rlt.v21.21281.CrossRefGoogle Scholar
  10. Blayone, T., Mykhailenko, O., vanOostveen, R., Grebeshkov, O., Hrebeshkova, O., & Vostryakov, O. (2017a). Surveying digital competencies of university students and professors in Ukraine for fully online collaborative learning. Technology, Pedagogy and Education.  https://doi.org/10.1080/1475939X.2017.1391871.
  11. Blayone, T., vanOostveen, R., Barber, W., DiGiuseppe, M., & Childs, E. (2017b). Democratizing digital learning: Theorizing the fully online learning community model. International Journal of Educational Technology in Higher Education, 14(1), 13.  https://doi.org/10.1186/s41239-017-0051-4.CrossRefGoogle Scholar
  12. Bradlow, E. T., Hoch, S. J., & Hutchinson, J. W. (2002). An assessment of basic computer proficiency among active internet users: Test construction, calibration, antecedents and consequences. Journal of Educational and Behavioral Statistics, 27(3), 237–253.  https://doi.org/10.3102/10769986027003237.CrossRefGoogle Scholar
  13. Bui, T. X., Sankaran, S., & Sebastian, I. M. (2003). A framework for measuring national e-readiness. International Journal of Electronic Business, 1(1), 3–22.  https://doi.org/10.1504/ijeb.2003.002162.CrossRefGoogle Scholar
  14. Crompton, H., Burke, D., Gregory, K. H., & Gräbe, C. (2016). The use of mobile learning in science: A systematic review. Journal of Science Education and Technology, 1–12.  https://doi.org/10.1007/s10956-015-9597-x.
  15. Deci, E. L., & Ryan, R. M. (2000). The" what" and" why" of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11(4), 227–268.  https://doi.org/10.1207/s15327965pli1104_01.CrossRefGoogle Scholar
  16. Demir, Ö., & Yurdugül, H. (2015). The exploration of models regarding e-learning readiness: Reference model suggestions. International Journal of Progressive Education, 11(1), 173–194.Google Scholar
  17. Desjardins, F. J. (2005). Information and communication technology in education: A competency profile of francophone secondary school teachers in Ontario. Canadian Journal of Learning and Technology/La revue canadienne de l’apprentissage et de la technologie, 31(1), 1–14.  https://doi.org/10.21432/T2PG69.MathSciNetGoogle Scholar
  18. Desjardins, F. J., & Peters, M. (2007). Single-course approach versus a program approach to develop technological competencies in pre-service language teaching. In M.-A. Kassen, L. Lavine, K. Murphy-Judy, & M. Peters (Eds.), Preparing and developing technology proficient L2 teachers (pp. 3–21). Texas: Texas State University.Google Scholar
  19. Desjardins, F. J., & vanOostveen, R. (2015). Faculty and student use of digital technology in a "laptop" university. In S. Carliner, C. Fulford, & N. Ostashewski (Eds.), EdMedia: World Conference on Educational Media and Technology 2015 (pp. 990-996). Montreal: Association for the Advancement of Computing in Education (AACE).Google Scholar
  20. Desjardins, F. J., Lacasse, R., & Belair, L. M. (2001). Toward a definition of four orders of competency for the use of information and communication technology (ICT) in education. Paper presented at the computers and advanced Technology in Education. Canada: Banff http://eilab.ca/wp-content/uploads/2013/04/2001CATE.pdf.Google Scholar
  21. Desjardins, F. J., vanOostveen, R., Bullock, S., DiGiuseppe, M., & Robertson, L. (2010). Exploring graduate student’s use of computer-based technologies for online learning. In J. Herrington & C. Montgomerie (Eds.), EdMedia: World Conference on Educational Media and Technology 2010 (pp. 440-444). Norfolk: Association for the Advancement of Computing in Education (AACE).Google Scholar
  22. DiGiuseppe, M., Partosoedarso, E., vanOostveen, R., & Desjardins, F. J. (2013). Exploring competency development with mobile devices. In M. B. Nunes & M. McPherson (Eds.), International Association for Development of the information society (IADIS) international conference on e-learning (pp. 384–388). Prague: International Association for Development of the Information Society.Google Scholar
  23. Ding, R., & Ma, F. (2013). Assessment of university student web searching competency by a task-based online test: A case study at Wuhan University, China. The Electronic Library, 31(3), 359–375.  https://doi.org/10.1108/EL-03-2011-0044.CrossRefGoogle Scholar
  24. Dray, B. J., Lowenthal, P. R., Miszkiewicz, M. J., Ruiz-Primo, M. A., & Marczynski, K. (2011). Developing an instrument to assess student readiness for online learning: A validation study. Distance Education, 32(1), 29–47.  https://doi.org/10.1080/01587919.2011.565496.CrossRefGoogle Scholar
  25. Esbjörnsson, M., Brown, B., Juhlin, O., Normark, D., Östergren, M., & Laurier, E. (2006). Watching the cars go round and round: Designing for active spectating. In. In R. Grinter, T. Rodden, P. Aoki, E. Cutrell, R. Jeffries, & G. Olson (Eds.), Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1221–1224). New York: ACM.Google Scholar
  26. Eshet-Alkalai, Y., & Amichai-Hamburger, Y. (2004). Experiments in digital literacy. Cyberpsychology & Behavior, 7(4), 421–429.CrossRefGoogle Scholar
  27. Farid, A. (2014). Student online readiness assessment tools: A systematic review approach. Electronic Journal of e-Learning, 12(4), 375–382.Google Scholar
  28. Garrison, D. R., Cleveland-Innes, M., Koole, M., & Kappelman, J. (2006). Revisiting methodological issues in transcript analysis: Negotiated coding and reliability. The Internet and Higher Education, 9(1), 1–8.  https://doi.org/10.1016/j.iheduc.2005.11.001.CrossRefGoogle Scholar
  29. Gay, G. (2016). An assessment of online instructor e-learning readiness before, during, and after course delivery. Journal of Computing in Higher Education, 28(2), 199–220.  https://doi.org/10.1007/s12528-016-9115-z.CrossRefGoogle Scholar
  30. Greene, J. A., Seung, B. Y., & Copeland, D. Z. (2014). Measuring critical components of digital literacy and their relationships with learning. Computers & Education, 76, 55–69.  https://doi.org/10.1016/j.compedu.2014.03.008.CrossRefGoogle Scholar
  31. Hargittai, E. (2002). Beyond logs and surveys: In-depth measures of people's web use skills. Journal of the American Society for Information Science and Technology, 53(14), 1239–1244.  https://doi.org/10.1002/asi.10166.CrossRefGoogle Scholar
  32. Hargittai, E., & Shafer, S. (2006). Differences in actual and perceived online skills: The role of gender. Social Science Quarterly, 87(2), 432–448.  https://doi.org/10.1111/j.1540-6237.2006.00389.x.CrossRefGoogle Scholar
  33. Herrington, J., Reeves, T. C., & Oliver, R. (2006). Authentic tasks online: A synergy among learner, task, and technology. Distance Education, 27(2), 233–247.  https://doi.org/10.1080/01587910600789639.CrossRefGoogle Scholar
  34. Horzum, M. B., Kaymak, Z. D., & Gungoren, O. C. (2015). Structural equation modeling towards online learning readiness, academic motivations, and perceived learning. Educational Sciences: Theory and Practice, 15(3), 759–770.  10.12738/estp.2015.3.2410.Google Scholar
  35. Hung, M.-L. (2016). Teacher readiness for online learning: Scale development and teacher perceptions. Computers & Education, 94, 120–133.  https://doi.org/10.1016/j.compedu.2015.11.012.CrossRefGoogle Scholar
  36. Hung, M.-L., Chou, C., & Chen, C.-H. (2010). Learner readiness for online learning: Scale development and student perceptions. Computers & Education, 55(3), 1080–1090.  https://doi.org/10.1016/j.compedu.2010.05.004.CrossRefGoogle Scholar
  37. IEEE. (1990). IEEE standard computer dictionary: A compilation of IEEE standard computer glossaries. In (pp. 218). New York: The Institute of Electrical and Electronics Engineers.Google Scholar
  38. Jayroe, T. J., & Wolfram, D. (2012). Internet searching, tablet technology and older adults. Proceedings of the American Society for Information Science and Technology, 49(1), 1–3.  https://doi.org/10.1002/meet.14504901236.CrossRefGoogle Scholar
  39. Knoblauch, H. (2012). Introduction to the special issue of qualitative research: Video-analysis and videography. Qualitative Research, 12(3), 251–254.  https://doi.org/10.1177/1468794111436144.CrossRefGoogle Scholar
  40. Leigh, D., & Watkins, R. (2005). E-learner success: Validating a self-assessment of learner readiness for online training. In. In ASTD 2005 research-to-practice conference proceedings (pp. 121–131). Alexandria: ATD.Google Scholar
  41. Lin, H.-H., Lin, S., Yeh, C.-H., Wang, Y.-S., & Jansen, J. (2015). Measuring mobile learning readiness: Scale development and validation. Internet Research, 26(1), 265–287.  https://doi.org/10.1108/IntR-10-2014-0241.CrossRefGoogle Scholar
  42. Litt, E. (2013). Measuring users’ internet skills: A review of past assessments and a look toward the future. New Media & Society, 15(4), 612–630.  https://doi.org/10.1177/1461444813475424.CrossRefGoogle Scholar
  43. Mason, M. (2010). Sample size and saturation in PhD studies using qualitative interviews. Forum qualitative Sozialforschung/Forum: Qualitative Social Research, 11(3), 1–5.MathSciNetGoogle Scholar
  44. Merriam, S. B. (1998). Qualitative research and case study applications in education. Revised and expanded from case study research in education. San Francisco: Josey-Bass Publishers.Google Scholar
  45. Mosa, A. A., Naz’ri bin Mahrin, M., & Ibrrahim, R. (2016). Technological aspects of e-learning readiness in higher education: A review of the literature. Computer and Information Science, 9(1), 113–127.  https://doi.org/10.5539/cis.v9n1p113.CrossRefGoogle Scholar
  46. Parasuraman, A. (2000). Technology readiness index (TRI) a multiple-item scale to measure readiness to embrace new technologies. Journal of Service Research, 2(4), 307–320.  https://doi.org/10.1177/109467050024001.CrossRefGoogle Scholar
  47. Park, Y. J. (2015). My whole world’s in my palm! The second-level divide of teenagers’ mobile use and skill. New Media & Society, 17(6), 977–995.  https://doi.org/10.1177/1461444813520302.CrossRefGoogle Scholar
  48. Parkes, M., Stein, S., & Reading, C. (2015). Student preparedness for university e-learning environments. The Internet and Higher Education, 25, 1–10.  https://doi.org/10.1016/j.iheduc.2014.10.002.CrossRefGoogle Scholar
  49. Rourke, L., & Anderson, T. (2004). Validity in quantitative content analysis. Educational Technology Research and Development, 52(1), 5–18.  https://doi.org/10.1007/bf02504769.CrossRefGoogle Scholar
  50. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 12, 8–22.Google Scholar
  51. Savin-Baden, M. (2000). Problem-based learning in higher education: Untold stories. Philadelphia: Open University Press.Google Scholar
  52. Siemens, G., Gašević, D., & Dawson, S. (2015). Preparing for the digital university: A review of the history and current state of distance, blended, and online Learning Retrieved from http://linkresearchlab.org/PreparingDigitalUniversity.pdf.
  53. Sun, X., & May, A. (2013). A comparison of field-based and lab-based experiments to evaluate user experience of personalised mobile devices. Advances in Human-Computer Interaction, 2013, 2.  https://doi.org/10.1155/2013/619767.CrossRefGoogle Scholar
  54. van Deursen, A. J. A. M. (2010). Internet skills: Vital assets in an information society. (Ph.D. Thesis), University of Twente, Enschede, the Netherlands. Retrieved from http://doc.utwente.nl/75133/1/thesis_van_Deursen.pdf.
  55. van Deursen, A. J. A. M., & van Dijk, J. A. G. M. (2010). Measuring internet skills. International Journal of Human-Computer Interaction, 26(10), 891–916.  https://doi.org/10.1080/10447318.2010.496338.CrossRefGoogle Scholar
  56. van Deursen, A. J. A. M., Helsper, E. J., & Eynon, R. (2015). Development and validation of the internet skills scale (ISS). Information, Communication & Society, 1–20.  https://doi.org/10.1080/1369118X.2015.1078834.
  57. vanOostveen, R., DiGiuseppe, M., Barber, W., Blayone, T., & Childs, E. (2016). New conceptions for digital technology sandboxes: Developing a fully online learning communities (FOLC) model. In G. Veletsianos (Ed.), EdMedia 2016: World conference on educational media and technology (pp. 665–673). Vancouver: Association for the Advancement of Computing in Education (AACE).Google Scholar
  58. Watkins, R., Leigh, D., & Triner, D. (2004). Assessing readiness for e-learning. Performance Improvement Quarterly, 17(4), 66–79.  https://doi.org/10.1111/j.1937-8327.2004.tb00321.x.CrossRefGoogle Scholar
  59. Wilhelm, J. (2016). What is the minimum sample size to run Pearsons R? (Online Expert Database). Retrieved June 7, 2017, from ResearchGate: https://www.researchgate.net/post/What_is_the_minimum_sample_size_to_run_Pearsons_R.

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Todd J. B. Blayone
    • 1
  • Olena Mykhailenko
    • 1
  • Roland vanOostveen
    • 2
  • Wendy Barber
    • 2
  1. 1.Collaboritsi.com and Educational Informatics Laboratory (EILAB)OshawaCanada
  2. 2.Faculty of Education and Educational Informatics Laboratory (EILAB), University of Ontario Institute of TechnologyOshawaCanada

Personalised recommendations