Ready for digital learning? A mixed-methods exploration of surveyed technology competencies and authentic performance activity
The Digital Competency Profiler (DCP) is an online application for surveying the technology preferences and abilities of students in higher education. To explore the DCP as a digital-learning-readiness tool, a mixed-methods research design was developed for relating self-reported digital competencies and online-learning activity. To this end, three authentic scenarios, comprised of six tasks mapped to self-report items, were constructed. Having submitted their survey data, each of 15 participants visited the EILAB to complete a randomly-assigned scenario with a tablet. Both the performance activity and post-activity interviews were recorded digitally using a unique activity-station setup, and task artefacts were gathered as performance outcomes. Analysis was conducted in three phases. In Phase 1, both the audio-video performance data and activity artefacts were coded, assessed and scored. Exploratory correlational analyses showed a pattern of positive relationships at the task and scenario levels for two scenario groups, suggesting some predictive value for the DCP in this context. For the third group, a positive correlation was found at the scenario level, but negative correlations were found at the task level. In Phase 2, detailed case-studies were conducted, incorporating self-report data, coded performance timelines, and post-activity interviews. Several situational influencers related to problem-solving strategy, device comfort, task difficulty and motivation, beyond the purview of the DCP, were identified. In Phase 3, the findings were interpreted to position the DCP as a tool for identifying segments of students with members who, without support, will likely struggle to engage fully in technology-rich learning environments.
KeywordsDigital competence Digital skills Digital learning Online learning Readiness Digital readiness Observational study Higher education
- Al-Araibi, A. A. M., Mahrin, M., & Mohd, R. C. (2016). A systematic literature review of technological factors for e-learning readiness in higher education. Journal of Theoretical and Applied Information Technology, 93(2), 500–521.Google Scholar
- Aparicio, M., Bacao, F., & Oliveira, T. (2016). An e-learning theoretical framework. Journal of Educational Technology & Society, 19(1), 292–307. http://www.jstor.org/stable/10.2307/jeductechsoci.19.1.292Google Scholar
- Aydın, C. H., & Tasci, D. (2005). Measuring readiness for e-learning: Reflections from an emerging country. Educational Technology and Society, 8(4), 244–257. http://www.jstor.org/stable/jeductechsoci.8.4.244Google Scholar
- Beetham, H., & Sharpe, R. (2007). Rethinking pedagogy for a digital age: Designing for 21st century learning. New York: Routledge.Google Scholar
- Bertaux, D. (1981). From the life-history approach to the transformation of sociological practice. In D. Bertaux (Ed.), Biography and society: The life history approach in Social Sciences (pp. 29–45). London: Sage.Google Scholar
- Blayone, T., Mykhailenko, O., vanOostveen, R., Grebeshkov, O., Hrebeshkova, O., & Vostryakov, O. (2017a). Surveying digital competencies of university students and professors in Ukraine for fully online collaborative learning. Technology, Pedagogy and Education. https://doi.org/10.1080/1475939X.2017.1391871.
- Blayone, T., vanOostveen, R., Barber, W., DiGiuseppe, M., & Childs, E. (2017b). Democratizing digital learning: Theorizing the fully online learning community model. International Journal of Educational Technology in Higher Education, 14(1), 13. https://doi.org/10.1186/s41239-017-0051-4.CrossRefGoogle Scholar
- Bradlow, E. T., Hoch, S. J., & Hutchinson, J. W. (2002). An assessment of basic computer proficiency among active internet users: Test construction, calibration, antecedents and consequences. Journal of Educational and Behavioral Statistics, 27(3), 237–253. https://doi.org/10.3102/10769986027003237.CrossRefGoogle Scholar
- Crompton, H., Burke, D., Gregory, K. H., & Gräbe, C. (2016). The use of mobile learning in science: A systematic review. Journal of Science Education and Technology, 1–12. https://doi.org/10.1007/s10956-015-9597-x.
- Demir, Ö., & Yurdugül, H. (2015). The exploration of models regarding e-learning readiness: Reference model suggestions. International Journal of Progressive Education, 11(1), 173–194.Google Scholar
- Desjardins, F. J. (2005). Information and communication technology in education: A competency profile of francophone secondary school teachers in Ontario. Canadian Journal of Learning and Technology/La revue canadienne de l’apprentissage et de la technologie, 31(1), 1–14. https://doi.org/10.21432/T2PG69.MathSciNetGoogle Scholar
- Desjardins, F. J., & Peters, M. (2007). Single-course approach versus a program approach to develop technological competencies in pre-service language teaching. In M.-A. Kassen, L. Lavine, K. Murphy-Judy, & M. Peters (Eds.), Preparing and developing technology proficient L2 teachers (pp. 3–21). Texas: Texas State University.Google Scholar
- Desjardins, F. J., & vanOostveen, R. (2015). Faculty and student use of digital technology in a "laptop" university. In S. Carliner, C. Fulford, & N. Ostashewski (Eds.), EdMedia: World Conference on Educational Media and Technology 2015 (pp. 990-996). Montreal: Association for the Advancement of Computing in Education (AACE).Google Scholar
- Desjardins, F. J., Lacasse, R., & Belair, L. M. (2001). Toward a definition of four orders of competency for the use of information and communication technology (ICT) in education. Paper presented at the computers and advanced Technology in Education. Canada: Banff http://eilab.ca/wp-content/uploads/2013/04/2001CATE.pdf.Google Scholar
- Desjardins, F. J., vanOostveen, R., Bullock, S., DiGiuseppe, M., & Robertson, L. (2010). Exploring graduate student’s use of computer-based technologies for online learning. In J. Herrington & C. Montgomerie (Eds.), EdMedia: World Conference on Educational Media and Technology 2010 (pp. 440-444). Norfolk: Association for the Advancement of Computing in Education (AACE).Google Scholar
- DiGiuseppe, M., Partosoedarso, E., vanOostveen, R., & Desjardins, F. J. (2013). Exploring competency development with mobile devices. In M. B. Nunes & M. McPherson (Eds.), International Association for Development of the information society (IADIS) international conference on e-learning (pp. 384–388). Prague: International Association for Development of the Information Society.Google Scholar
- Esbjörnsson, M., Brown, B., Juhlin, O., Normark, D., Östergren, M., & Laurier, E. (2006). Watching the cars go round and round: Designing for active spectating. In. In R. Grinter, T. Rodden, P. Aoki, E. Cutrell, R. Jeffries, & G. Olson (Eds.), Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1221–1224). New York: ACM.Google Scholar
- Farid, A. (2014). Student online readiness assessment tools: A systematic review approach. Electronic Journal of e-Learning, 12(4), 375–382.Google Scholar
- IEEE. (1990). IEEE standard computer dictionary: A compilation of IEEE standard computer glossaries. In (pp. 218). New York: The Institute of Electrical and Electronics Engineers.Google Scholar
- Leigh, D., & Watkins, R. (2005). E-learner success: Validating a self-assessment of learner readiness for online training. In. In ASTD 2005 research-to-practice conference proceedings (pp. 121–131). Alexandria: ATD.Google Scholar
- Merriam, S. B. (1998). Qualitative research and case study applications in education. Revised and expanded from case study research in education. San Francisco: Josey-Bass Publishers.Google Scholar
- Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 12, 8–22.Google Scholar
- Savin-Baden, M. (2000). Problem-based learning in higher education: Untold stories. Philadelphia: Open University Press.Google Scholar
- Siemens, G., Gašević, D., & Dawson, S. (2015). Preparing for the digital university: A review of the history and current state of distance, blended, and online Learning Retrieved from http://linkresearchlab.org/PreparingDigitalUniversity.pdf.
- van Deursen, A. J. A. M. (2010). Internet skills: Vital assets in an information society. (Ph.D. Thesis), University of Twente, Enschede, the Netherlands. Retrieved from http://doc.utwente.nl/75133/1/thesis_van_Deursen.pdf.
- van Deursen, A. J. A. M., Helsper, E. J., & Eynon, R. (2015). Development and validation of the internet skills scale (ISS). Information, Communication & Society, 1–20. https://doi.org/10.1080/1369118X.2015.1078834.
- vanOostveen, R., DiGiuseppe, M., Barber, W., Blayone, T., & Childs, E. (2016). New conceptions for digital technology sandboxes: Developing a fully online learning communities (FOLC) model. In G. Veletsianos (Ed.), EdMedia 2016: World conference on educational media and technology (pp. 665–673). Vancouver: Association for the Advancement of Computing in Education (AACE).Google Scholar
- Watkins, R., Leigh, D., & Triner, D. (2004). Assessing readiness for e-learning. Performance Improvement Quarterly, 17(4), 66–79. https://doi.org/10.1111/j.1937-8327.2004.tb00321.x.CrossRefGoogle Scholar
- Wilhelm, J. (2016). What is the minimum sample size to run Pearsons R? (Online Expert Database). Retrieved June 7, 2017, from ResearchGate: https://www.researchgate.net/post/What_is_the_minimum_sample_size_to_run_Pearsons_R.