Advertisement

Universal Access in the Information Society

, Volume 17, Issue 2, pp 349–360 | Cite as

Performance-based testing for ICT skills assessing: a case study of students and teachers’ ICT skills in Finnish schools

  • Meri-Tuulia Kaarakainen
  • Osmo Kivinen
  • Teija Vainio
Long Paper

Abstract

Skills enabling and ensuring universal access to information have been investigated intensively during the past few years. The research results provide knowledge on the differences and digital divides. When examining ICT skill levels, the accuracy of assessment is one of the key issues to address the results and gain the applicable data for appropriate interventions to enhance digital inclusion. To date, ICT skill assessment is based mainly on self-reports and subjective evaluations. However, as previous studies have shown, people tend to overrate or underrate their own levels of competence. Thus, novel performance-based approach for assessing ICT skills is presented in this paper. The ICT skill test contains 42 tasks grouped into 17 ICT fields. The study was conducted with upper comprehensive and upper secondary level school students (n = 3159) and their teachers (n = 626) during years 2014–2016 in Finland. Using factor analysis, three ICT skills factors were created: basic digital skills, advanced technical skills, and professional ICT skills. The performance in the ICT skill test was also divided by gender, as the male students and teachers outperformed the female students and teacher. Outperformance also occurred by educational level, as both upper secondary level students and teachers were seen as possessing higher-level ICT skills than students and teachers at the comprehensive level. We thus argue that to compare the ICT skills and the validity of the assessments, we needed to ensure consistent assessment for both students and teachers. In addition, in order to diminish the ICT skill gap among students, interventions using formal education are urgently needed, and in particular, more attention should be given to both teacher training and in-service training.

Keywords

Skill assessment ICT skills Digital divide Formal education 

Notes

Acknowledgements

The authors would like to thank all the students and teachers who participated the ICT assessments. The work was supported by the University of Turku.

References

  1. 1.
    van Deursen, A.J.A.M., Courtois, C., van Dijk, J.A.G.M.: Internet skills, sources of support, and benefiting from internet use. Int. J. Hum.-Comput. Interact. 30(4), 278–290 (2014)CrossRefGoogle Scholar
  2. 2.
    Livingstone, S., Helsper, E.J.: Gradations in digital inclusion: children, young people, and the digital divide. New Media Soc. 9(4), 671–696 (2007)CrossRefGoogle Scholar
  3. 3.
    Kivinen, O., Piiroinen, T., Saikkonen, L.: Two viewpoints on the challenges of ICT in education: knowledge-building theory vs. a pragmatist conception of learning in social action. Oxf. Rev. Educ. (2016). doi: 10.1080/03054985.2016.1194263 Google Scholar
  4. 4.
    Hargittai, E., Walejko, G.: The participation divide: content creation and sharing in the digital age. Inf. Commun. Soc. 11(2), 239–256 (2008)CrossRefGoogle Scholar
  5. 5.
    Ainley, J., Fraillon, J., Schulz, W., Gebhardt, E.: Conceptualizing and measuring computer and information literacy in cross-national contexts. Appl. Meas. Educ. (2016). doi: 10.1080/08957347.2016.1209205 Google Scholar
  6. 6.
    Ferrari, A.: DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe. Publications Office of the European Union, 2013, European Union, 2013, Luxembourg (2013). http://jrc.es/EURdoc/JRC83167.pdf
  7. 7.
    Binkley, M., Erstad, E., Herman, J., Raizen, S., Ripley, M., Miller-Ricci, M., Rumble, M.: Defining 21st century skills. In: Griffin, P., McGaw, B., Care, E. (eds.) Assessment and Teaching of 21st Century Skills, pp. 17–66. Springer, Dordrecht (2012). doi: 10.1007/978-94-007-2324-5 CrossRefGoogle Scholar
  8. 8.
    Fraillon, J., Schulz, W., Ainley, J.: International Computer and Information Literacy Study: Assessment Framework. IEA, Amsterdam (2013)Google Scholar
  9. 9.
    van Deursen, A.J.A.M., Helsper, E.J.: The third level digital divide: who benefits most from being online? Commun. Inf. Technol. Annu. 10, 29–52 (2015). doi: 10.1108/S2050-206020150000010002 CrossRefGoogle Scholar
  10. 10.
    van Dijk, J.A.G.M.: The Deepening Divide: Inequality in the Information Society. Sage, London (2005)Google Scholar
  11. 11.
    Siddiq, F., Hatlevik, O., Olsen, E., Throndsen, R.V., Scherer, R.: Taking a future perspective by learning from the past—a systematic review of assessment instruments that aim to measure primary and secondary school students’ ICT literacy. Educ. Res. Rev. 19, 58–84 (2016)CrossRefGoogle Scholar
  12. 12.
    Merrit, K., Smith, D., Renzo, J.C.D.: An investigation of self-reported computer literacy: is it reliable. Issues Inf. Syst. 6(1), 289–295 (2005)Google Scholar
  13. 13.
    Umar, I., Yusoff, T.: A study of Malaysian teacher’s level of ICT skills and practices, and its impact on teaching and learning. Soc. Behav. Sci. 116, 979–984 (2014)CrossRefGoogle Scholar
  14. 14.
    van Deursen, A.J.A.M., Helsper, E.J., Eynon, R.: Development and validation of the Internet Skill Scale (IIS). Inf. Commun. Soc. 19(6), 804–823 (2016). doi: 10.1080/1369118X.2015.1078834 CrossRefGoogle Scholar
  15. 15.
    FNBE: National Core Curriculum 2004. Finnish National Board of Education, Helsinki (2004). http://www.oph.fi/english/curricula_and_qualifications/basic_education/curricula_2004
  16. 16.
    van Dijk, J.A.G.M., Hacker, K.:The digital divide as a complex and dynamic phenomenon. In: Annual Conference of the International Communication Association, Acapulco, 1–5 June 2000Google Scholar
  17. 17.
    Hargittai, E., Hsieh, Y.P.: Digital inequality. In: Dutton, W.H. (ed.) Oxford Handbook of Internet Studies, pp. 129–150. Oxford University Press, Oxford (2013)Google Scholar
  18. 18.
    van Deursen, A.J.A.M., van Dijk, J.A.G.M.: Internet skills and the digital divide. New Media Soc. 13(6), 893–911 (2011)CrossRefGoogle Scholar
  19. 19.
    Hatlevik, O.E., Guomundsdóttir, G.B., Loi, M.: Digital diversity among upper secondary students. Comput. Educ. 81(February 2015), 345–353 (2015). doi: 10.1016/j.compedu.2014.10.019 CrossRefGoogle Scholar
  20. 20.
    Goode, J.: The digital identity divide: how technology knowledge impacts college students. New Media Soc. 12(3), 497–513 (2010)CrossRefGoogle Scholar
  21. 21.
    Helsper, E., Eynon, R.: Digital natives: where is the evidence? Br. Educ. Res. J. 36(3), 503–520 (2010)CrossRefGoogle Scholar
  22. 22.
    Claro, M., Preiss, D., San Martin, E., Jara, I., Hinostroza, J.E., Valenzuela, S., Cortes, F., Nussbaum, M.: Assessment of 21st century ICT skills in Chile: test design and results from high school level students. Comput. Educ. 59, 1042–1053 (2012)CrossRefGoogle Scholar
  23. 23.
    Gui, M., Argentin, G.: Digital skills of internet natives: different forms of internet literacy in a random sample of northern Italian high school students. New Media Soc. 13(6), 963–980 (2011)CrossRefGoogle Scholar
  24. 24.
    Thompson, P.: The digital natives as learners: technology use patterns and approaches to learning. Comput. Educ. 65(1), 12–33 (2013)CrossRefGoogle Scholar
  25. 25.
    Litt, E.: Measuring users’ internet skills: a review of past assessments and a look toward the future. New Media Soc. 15(4), 612–630 (2013)CrossRefGoogle Scholar
  26. 26.
    Livingstone, S., Helsper, E.: Balancing opportunities and risks in teenagers’ use of the internet: the role of online skills and internet self-efficacy. New media Soc. 12(2), 309–329 (2010). doi: 10.1177/1461444809342697 CrossRefGoogle Scholar
  27. 27.
    Hargittai, E.: Beyond logs and surveys: in-depth measures of people’s web use skills. J. Am. Soc. Inf. Sci. Technol. 23(14), 1239–1244 (2002)CrossRefGoogle Scholar
  28. 28.
    van Deursen, A.J.A.M., van Dijk, J.A.G.M.: Improving digital skills for the use of online public information and services. Gov. Inf. Q. 26(2), 333–340 (2009)CrossRefGoogle Scholar
  29. 29.
    Eshet-Alkalai, Y., Amichai-Hamburger, Y.: Experiments in digital literacy. CyberPsychol. Behav. 7(4), 421–429 (2004)CrossRefGoogle Scholar
  30. 30.
    Hakkarainen, K., Ilomäki, L., Lipponen, L., Muukkonen, H., Rahikainen, M., Tuominen, T., Lakkala, M., Lehtinen, E.: Students’ skills and practices of using ICT: results of a national assessment in Finland. Comput. Educ. 34(2), 103–117 (2000). doi: 10.1016/S0360-1315(00)00007-5 CrossRefGoogle Scholar
  31. 31.
    Viteli, J.: Teachers and use of ICT in education: pilot study and testing of the Opeka system. In: Herrington, J., et al. (eds.) Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2013, pp. 2326–2346. AACE, Chesapeake (2013)Google Scholar
  32. 32.
    FNBE: Finnish Education in a Nutshell. Education in Finland. FNBE, Helsinki (2012). http://www.oph.fi/download/146428_Finnish_Education_in_a_Nutshell.pdf
  33. 33.
    Christmann, A., Van Aelst, S.: Robust estimation of Cronbach’s alpha. J. Multivar. Analysis 97, 1660–1674 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Calvani, A., Fini, A., Ranieri, M., Picci, P.: Are young generations in secondary school digitally competent? A study on Italian teenagers. Comput. Educ. 58, 797–807 (2012)CrossRefGoogle Scholar
  35. 35.
    OECD: Pisa 2009 Results: Students On Line. Digital Technologies and Performance, vol. VI. OECD, Paris (2011)Google Scholar
  36. 36.
    Tømte, C., Hatlevik, O.E.: Gender-differences in Self-efficacy ICT related to various ICT-user profiles in Finland and Norway. How to self-efficacy, gender and ICT-user profiles relate to findings from PISA 2006. Comput. Educ. 57(1), 1416–1424 (2011)CrossRefGoogle Scholar
  37. 37.
    Ilomäki, L.: Does gender have a role in ICT among Finnish teachers and students? Scand. J. Educ. Res. 55(3), 325–340 (2011)CrossRefGoogle Scholar
  38. 38.
    Hargittai, E.: Digital na(t)ives? Variation in internet skills and uses among members of the ‘net generation’. Sociol. Inq. 80(1), 92–113 (2010)CrossRefGoogle Scholar
  39. 39.
    OECD: The 2012 Programme for International Student Assessment (PISA), Computers, Education and Skills. OECD Publishing, Paris (2012). doi: 10.1787/9789264239555-en Google Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  • Meri-Tuulia Kaarakainen
    • 1
  • Osmo Kivinen
    • 1
  • Teija Vainio
    • 1
    • 2
  1. 1.Research Unit for the Sociology of Education (RUSE)University of Turku, Turun YliopistoTurkuFinland
  2. 2.School of ManagementUniversity of Tampere, Tampereen yliopistoTampereFinland

Personalised recommendations