Advertisement

Evaluation of ICT Competencies

  • Agustin Tristán-López
  • Miguel Angel Ylizaliturri-Salcedo
Chapter

Abstract

Information and Communication Technology (ICT) competencies comprise a subset of ­digital literacy, one of the various areas of technological competencies and skills necessary for real-life functioning in all kind of professional activities and levels of education from preschool to graduate levels. Assessment of ICT competencies involves the definition of specific target activities appropriate to the environment in which they are required. These environments range from the home to the workplace and involve such disparate activities as information-retrieval in healthcare settings or libraries; use of clerical, business, or investment applications; and interactions with government or other public services. The range of ICT competencies also includes the specific abilities needed by professionals responsible for the development of software or communication products and services. This chapter discusses several ICT assessment projects, addresses the primary technical specifications required for evaluation, and explores solutions to problems of test administration. These solutions range from electronic quizzes (similar to paper-and-pencil tests) to more “authentic” forms of assessment using e-portfolios or simulations of real software applications. A detailed analysis of the primary approaches for assessment of social, academic, commercial, or economic environments reveals that these approaches primarily focus on a basic core of skills consisting of Web navigation and the use of e-mail and office tools (text processor, spreadsheet, presentation, and database management). In the future, ICT evaluation will involve automatic scoring of natural language responses in documents, solutions of mathematical problems, and graphical or health applications, among many other real-life endeavors.

Keywords

ICT literacy assessment Digital literacy ICT competencies Assessment standards Automatic scoring 

Notes

Acknowledgements

This work was sponsored by project 2010-I03-0506 of IEIA, Mexico. Copyright of all images by Instituto de Evaluación e Ingeniería Avanzada, México (2007), reprinted with permission. Additional references can be found on the following Web page: www.ieia.com.mx.

References

  1. *    Ainley, J., & Fraillon, J. (2007). Assessing computer literacy using simulated and real world tasks. ACER. European Association for Research on Learning and Instruction. 12th Biennial Conference. Retrieved June 20, 2012, from http://earli2007live.nqcontent.net/nq/home/scientific_program/programme/proposal_view/%26;abstractid=510
  2. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. American Educational Research Association, American Psychological Association and National Council on Measurement in Education. Washington, DC: AERA.Google Scholar
  3. *Attali, Y., Bridgeman, B., & Trapani, C. (2010). Performance of a generic approach in automated essay scoring. Journal of Technology, Learning and Assessment, 10(3). Retrieved June 20, 2012, from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1603/1455
  4. Axelson, M. (2005). Online assessments of technology literacy: The perfect Petri Dish. Insight. Vol. 5. (ERIC document ED489135). Retrieved June 20, 2012, from www.eric.ed.gov/PDFS/ED489135.pdf.
  5. Campbell, L. M., Littlejohn, A. H., & Duncan, C. (2001). Share and share alike: Encouraging the re-use of academic resources through the Scottish electronic Staff Development Library, Association of Learning Technologies Journal (ALT-J), 9.2, 28–38. ISSN: 0968-7769. Retrieved June 20, 2012, from http://repository.alt.ac.uk/350/1/ALT_J_Vol9_No2_2001_Share%20and%20share%20alike_%20Encoura.pdf
  6. Cole, R. P., MacIsaac, D., & Cole, D. (2001). A comparison of paper-based and Web-based testing. AERA Meeting, Seattle, WA. (ERIC document: ED453224). Retrieved June 20, 2012, from www.eric.ed.gov/PDFS/ED453224.pdf
  7. *Dikli, S. (2006). An overview of automated scoring of essays. The Journal of Technology, Learning and Assessment, 5(1). Retrieved June 20, 2012, from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1640/1489
  8. European Computer Drivers Licence. (2010). Building a digitally literate Europe through education. European Computer Drivers Licence Foundation, ECDL. Retrieved March 7, 2012, from www.ecdl.org/media/ECDL%20Position%20Paper%20-%20Buillding%20Dig.%20Literate%20Europe%20Through%20Educ_2010.pdf
  9. European Computer Drivers Licence. (2011). Promoting inclusive economic growth and employment through enhanced ICT skills and knowledge. European Computer Drivers Licence Foundation, ECDL. Retrieved March 7, 2012, from www.ecdl.org/media/ECDL_Position_Paper-e-Productivity1.pdf
  10. *Higgins, J., Patterson, M. B., Bozman, M., & Katz, M. (2010). Examining the feasibility and effect of transitioning GED tests to computer. Journal of Technology, Learning and Assessment, 10(2). Retrieved June 20, 2012, from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1602/
  11. *International Technology Education Association. (2007). Standards for technological literacy: Content for the study of technology (3rd ed.). Reston, VA: ITEA. ISBN: 11-887101-02-0.Google Scholar
  12. *Kalu, K. N. (2007). Capacity building and IT diffusion: A comparative assessment of E-government environment in Africa. Social Science Computer Review. 25(3), 358–371.Google Scholar
  13. *Katz, I. R., Elliot, N., Attali, Scharf, D., Powers, D., Huey, H., Joshi, K., & Briller, V. (2008). The assessment of information literacy: A case study. (ETS Res. Rep. No. RR-08-33). Princeton NJ: Educational Testing Service.Google Scholar
  14. Katz, I. R., & Macklin, A. S. (2007). Information and communication technology (ICT) literacy: Integration and assessment in higher education. Systemics, Cybernetics and Informatics, 5(4), 50–55.Google Scholar
  15. *Kelly, M. (2007). E-Portfolios and Web 2.0. Retrieved June 20, 2012, from www.education.ed.ac.uk/e-learning/gallery/kelly_eportfolios.pdf
  16. Levy, F. (2010). How technology changes demands for human skills. OECD Education Working Papers, No. 45, OECD Publishing. Retrieved June 20, 2012, from http://dx.doi.org/10.1787/5kmhds6czqzq-en
  17. Leye, V. (2007). UNESCO, ICT corporations and the passion of ICT for development, modernization resurrected. Media, Culture and Society, 29(6), 972–993.CrossRefGoogle Scholar
  18. Mason, R., Pegler, C., & Weller, M. (2004). E-portfolios: An assessment tool for online courses. British Journal of Educational Technology, 35(6), 717–727.CrossRefGoogle Scholar
  19. *Masters, J. (2010). Automated scoring of an interactive geometry item: A proof-of-concept. Journal of Technology, Learning and Assessment, 8(7). Retrieved June 20, 2012, from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1626/1470
  20. Monaghan, W., & Bridgeman, B. (2005). E-rater as a quality control on human scores. R&D Connections. ETS. Retrieved June 20, 2012, from www.ets.org/Media/Research/pdf/RD_Connections2.pdf
  21. Omona, W., & Ikoja-Odongo, R. (2006). Application of information and communication technology (ICT) in health information access and dissemination in Uganda. Journal of Librarian and Information Science, 38(1), 45–55.CrossRefGoogle Scholar
  22. *Organisation for Economic Co-operation and Development. (2006). Assessing scientific, reading and mathematical literacy—A framework for PISA 2006. Paris: Organisation for Economic Co-operation and Development, OECD.Google Scholar
  23. Organisation for Economic Co-operation and Development. (2009). PIAAC problem solving in technology-rich environments: a conceptual framework. Education Working paper No. 36. Paris: Organisation for Economic Co-operation and Development, OECD.Google Scholar
  24. *Smith, D. (2007). Assessment technology: Choose one from column B. Campus Technology Magazine. Retrieved June, 20, 2012, from http://campustechnology.com/Articles/2007/03/Assessment-Technology--Choose-One-From-Column-B.aspx?Page=6
  25. Tannenbaum, R., & Katz, I. R. (2008). Setting standards on the core and advanced iSkillsTM assessments. (Educational Testing Services, Report RM-08-04). Retrieved June, 20, 2012, from www.ets.org/Media/Research/pdf/RM-08-04.pdf
  26. Tomei, L. A. (2005). Taxonomy for the technology domain (Cap. 5, pp. 89–108). Hershey, PA: Idea Group.Google Scholar
  27. *Tristán, L. A. (2007). Analysis and automated scoring of essay texts in Spanish (Project Kalt-Lexico). 2nd regional Meeting of Educational Evaluation. Campeche, Mexico: Instituto de Evaluación e Ingeniería Avanzada. ISBN: 978-970-92251-2-9.Google Scholar
  28. *Tristán, L. A., & Ylizaliturri, S. M. A. (2008). Software for ICT competencies assessment in a simulated environment. 8th Forum of Educational Evaluation, Ceneval, Mexico.Google Scholar
  29. Umrani, F., & Ghadially, R. (2003). Empowering women through ICT education: Facilitating computer adoption. Gender Technology and Development. (7), 359–377.Google Scholar
  30. *United Nations Educational, Scientific and Cultural Organization. (2008). UNESCO’s ICT competency standards for teachers: Towards ICT skills for teachers. United Nations Educational, Scientific and Cultural Organization. Retrieved June, 20, 2012, from http://cst.unesco-ci.org/sites/projects/cst/default.aspx
  31. United Nations Educational, Scientific and Cultural Organization. (2009). Guide to measuring information and communication technologies (ICT) in education. (United Nations Educational, Scientific and Cultural Organization. Technical paper No. 2). Retrieved June, 20, 2012, from www.uis.unesco.org/Library/Documents/ICT_Guide_EN_v19_reprintwc.pdf
  32. *Williamson, D. M., Mislevy, R. J., & Bejar, I. I. (Eds.). (2007). Automated scoring of complex tasks in computer-based testing. Malwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  33. Wonacott, M. E. (2001). Technological Literacy. ERIC Digest. (ERIC Document 459371). Retrieved June, 20, 2012, from www.eric.ed.gov/PDFS/ED459371.pdf
  34. Ylizaliturri, S. M. A. (2007). On-line testing, an option for E-assessment. 2nd regional meeting of educational evaluation. Campeche, Mexico: Instituto de Evaluación e Ingeniería Avanzada. ISBN: 978-970-92251-2-9.Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Agustin Tristán-López
    • 1
  • Miguel Angel Ylizaliturri-Salcedo
    • 1
  1. 1.Instituto de Evaluación e Ingeniería Avanzada, S.C.San Luis PotosiMexico

Personalised recommendations