Abstract
Digital competence is a basic skill for citizens and should be systematically assessed, taking into account characteristics such as knowledge, skills and attitudes. However, measuring and identifying the digital competence of students is still a major challenge. To this end, the indicator model (INCODIES) has been developed and validated following the structure of the DIGCOMP model, which can be used to create assessment tests that measure student digital competence when they have completed their compulsory education. This article presents the results of a process of content validation carried out by 77 experts, in both the school and the university context. The analysis of the experts’ assessments and the level of agreement among those rating them were carried out following an original method which consists of determining a quantitative index for the validity of content called the Content Validity Ratio (CVR). The results obtained allowed us to confirm that the competences included in the reference framework DIGCOMP are correctly represented in the items formulated for the INCODIES model, which obtained high content validity. The model can serve as a structure and basis for the designing of specific evaluation tests on this topic.
Article PDF
Avoid common mistakes on your manuscript.
References
Aesaert, K., & Braak, J. V. (2014). Exploring factors related to primary school pupils’ ICT self-efficacy: A multilevel approach. Computers in Human Behavior, 41, 327–341. https://doi.org/10.1016/j.chb.2014.10.006
Aesaert, K., Voogt, J., Kuiper, E., & Braak, J. V. (2017). Accuracy and bias of ICT self-efficacy: An empirical study into students’ over- and underestimation of their ICT competences. Computers in Human Behavior, 75, 92–102. https://doi.org/10.1016/j.chb.2017.05.010
Ait, E., & González, J. (2017). Instruments per a l’avaluació de la competència digital de l’alumnat de secundària. UTE. Revista de Ciències de l’Educació, 1, 21–34. https://doi.org/10.17345/ute.2017.1.1747
Ala-Mutka, K. (2011). Mapping Digital Competence: Towards a Conceptual Understanding (and others, Ed.). Luxemburgo: JRC-IPTS European Commission. Retrieved from http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=4699
Almanasreh, E., Moles, R., & Chen, T. (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214–221. https://doi.org/10.1016/j.sapharm.2018.03.066
Amaro, A. C., Oliveira, L., & Veloso, A. I. (2017). Intergenerational and collaborative use of tablets: «in-medium» and «in-room» communication, learning and interaction. Observatorio (OBS), 11(1), 83–94. https://doi.org/10.15847/obsOBS1102017995
Barroso, J., & Cabero, J. (2010). La investigación educativa en TIC. Visiones prácticas (and others, Ed.). Madrid: Síntesis.
Calvani, A., Fini, A., Ranieri, M., & Picci, P. (2012). Are young generations in secondary school digitally competent? A study on Italian teenagers. Computers & Education, 58(2), 797–807. https://doi.org/10.1016/j.compedu.2011.10.004
Cameron, L., Wise, S. L., & Lottridge, S. M. (2007). The development and validation of the information literacy test. College & Research Libraries, 68(3), 229–237. https://doi.org/10.5860/crl.68.3.229
Carrió, C., Soler, M., & Aymerich, M. (2015). Análisis de la Validez de Contenido de un Cuestionario de Evaluación del Aprendizaje Basado en Problemas: Un Enfoque Cualitativo. Formación universitaria, 8(1), 13–22. https://doi.org/10.4067/S0718-50062015000100003
Colás, P., Conde, J., & Reyes, S. (2017). Competencias digitales del alumnado no universitario. Revista Latinoamericana de Tecnología Educativa, 16(1), 7–20. https://doi.org/10.17398/1695-288X.16.1.7
Dardanou, M., & Kofoed, T. (2019). It is not only about the tools! Professional digital competence. In G. Colette & I. Palaiologou (Eds.), Early learning in the digital age (pp. 61–76). London: SAGE Publications.
Ding, C., & Hershberger, S. (2002). Assessing content validity and content equivalence using structural equation modeling. Multidisciplinary Journal, 9(2), 283–297. https://doi.org/10.1207/S15328007SEM0902_7
Dornaleteche, J., Buitrago, A., & Moreno, L. (2015). Categorización, selección de ítems y aplicación del test de alfabetización digital online como indicador de la competencia mediática. Comunicar, 22(44), 177–185. https://doi.org/10.3916/C44-2015-19
Enochsson, A. (2019). Teenage pupils’ searching for information on the Internet. Information Research, 24(1). Retrieved from http://www.informationr.net/ir/24-1/isic2018/isic1822.html
Escobar-Pérez, J., & Cuervo-Martínez, A. (2008). Validez de contenido y juicio de expertos: una aproximación a su utilización. Avances en Medición, 6, 27–36.
Esteve, F., Gisbert, G., & Lázaro, J.L. (2016). La competencia digital de los futuros docentes: ¿cómo se ven los actuales estudiantes de educación? . Perspectiva Educacional. Formación de Profesores, 55(2), 38–54. https://doi.org/10.4151/07189729
European Comission. (2007). Competencias clave para un aprendizaje a lo largo de la vida un marco de referencia europeo. Bélgica: Comunidades Europeas.
Fernández-Miravete, A. D. (2018). La competencia digital del alumnado de Educación Secundaria en el marco de un Proyecto educativo TIC (1:1). Edutec. Revista Electrónica de Tecnología Educativa, 63, 60–72. https://doi.org/10.21556/edutec.2018.63.1027
Ferrari, A. (2013). DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe. Sevilla: Joint Research Centre, Institute for Prospective Technological Studies. https://doi.org/10.2788/52966
García, R., Duarte, A., & Guerra, S. (2014). Propuesta de un instrumento de evaluación para medir el grado de competencia mediática en la etapa de la educación infantil. . Pixel-Bit: Revista de medios y educación, 44, 81–96. https://doi.org/10.12795/pixelbit.2014.i44.06
García-Valcárcel, A. (2016). Las competencias digitales en el ámbito educativo. Retrieved from http://hdl.handle.net/10366/130340
García-Valcárcel, A., & Tejedor, F. J. (2017). Percepción de los estudiantes sobre el valor de las TIC en sus estrategias de aprendizaje y su relación con el rendimiento. Educación XXI, 20(2), 137–159. https://doi.org/10.5944/educxx1.19035
Gisbert, M., Espuny, C., & González, J. (2011). INCOTIC. Una herramienta para la @utoevaluación diagnóstica de la competencia digital en la universidad. Profesorado: Revista de curriculum y formación del profesorado, 15(1), 75–90.
Gisbert, M., & Esteve, F. (2011). Digital learners: la competencia digital de los estudiantes universitarios. La cuestión universitaria, 7, 48–59.
González, J., Espuny, C., Cid, M. J. D., & Gisbert, M. (2012). INCOTIC-ESO. Cómo autoevaluar y diagnosticar la competencia digital en la Escuela 2.0. Revista de Investigación Educativa, 30(2), 287–302. https://doi.org/10.6018/rie.30.2.117941
Hazar, E. (2018). Digital competence in primary education: The case of Turkish language, mathematics and personal and social development courses. International Online Journal of Education and Teaching (IOJET), 5(2), 443–458.
Hernández, A., & Iglesias, A. (2017). La importancia de las competencias digitales e informacionales para el desarrollo de una escuela intercultural. Interacçoes, 43, 205–232. https://doi.org/10.25755/int.12038
Hyrkäs, K., Appelqvist-Schmidlechner, K., & Oksa, L. (2003). Validating an instrument for clinical supervision using an expert panel. International Journal of nursing studies, 40(6), 36–37. https://doi.org/10.1016/S0020-7489(03)00036-1
ISTE. (2016). ISTE Standards for students. Retrieved from http://www.iste.org/standards/ISTE -standards/standards-for-students
Katz, I. R. (2007). Testing information literacy in digital environments: ETS’s iSkills assessment. Information technology and Libraries, 26(3), 3–12. https://doi.org/10.6017/ital.v26i3.3271
Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
Lee, L., Chen, D.T., Li, J. Y., & Lin, T. B. (2015). Understanding new media literacy: The development of a measuring instrument. Computers & Education, 85, 84–93. https://doi.org/10.1016/j.compedu.2015.02.006
Matamala, C. (2014). Factores predictivos de las competencias TIC en alumnos chilenos de secundaria. Revista iberoamericana de educación, 67(1), 121–136. https://doi.org/10.35362/rie671267
MEC-INTEF. (2017). Marco común de la Competencia Digital Docente. Retrieved from http://aprende.educalab.es/wp-content/uploads/2017/11/2017_1020_Marco-Com%C3%BAn-de-Competencia-Digital-Docente.pdf
Oakleaf, M. (2008). Dangers and Opportunities: A Conceptual Map of Information Literacy Assessment Approaches. Portal: Libraries and the Academy, 8(3), 233–253. https://doi.org/10.1353/pla.0.0011
Pinto, M. (2009). Design of the IL-HUMASS survey on information literacy in higher education: A self-assessment approach. Journal of Information Science, 36(1), 86–103. https://doi.org/10.1177/0165551509351198
Pinto, M., & Guerrero, D. (2017). Cómo perciben las competencias informacionales los estudiantes universitarios españoles: un estudio de caso. Investigación bibliotecológica. Investigación bibliotecológica, 31(73), 213–236. https://doi.org/10.22201/iibi.24488321xe.2017.73.57854
Quiroz, J. S., Miranda, P., Gisbert, M., Morales, J., & Onetto, A. (2016). Indicadores para evaluar la competencia digital docente en la formación inicial en el contexto Chileno-Uruguayo. RELATEC, 15(3), 55–68. https://doi.org/10.17398/1695-288X.15.3.55
Rodríguez, M. J., Olmos, S., & Martínez, F. (2012). Propiedades métricas y estructura dimensional de la adaptación española de una escala de evaluación de competencia informacional autopercibida (IL-HUMASS). Revista de Investigación Educativa, 30(2), 347–365. https://doi.org/10.6018/rie.30.2.120231
Scherer, R., Rohatgi, A., & Hatlevik, O. (2017). Students’ profiles of ICT use: Identification, determinants, and relations to achievement in a computer and information literacy test. Computers in Human Behavior, 70, 486–499. https://doi.org/10.1016/j.chb.2017.01.034
Siddiq, F., Hatlevik, O. E., Olsen, R. V., Throndsen, I., & Scherer, R. (2016). Taking a future perspective by learning from the past - A systematic review of assessment instruments that aim to measure primary and secondary school students’ ICT literacy. Educational Research Review, 19, 58–84.
Stopar, K., & Bartol, T. (2019). Digital competences, computer skills and information literacy in secondary education: mapping and visualization of trends and concepts. Scientometrics, 118(2), 479–498. https://doi.org/10.1007/s11192-018-2990-5
Tristán-López, A. (2008). Modificación al modelo de Lawshe para el dictamen cuantitativo de la validez de contenido de un instrumento objetivo. Avances en Medición, 6, 37–48.
Urrutia, M., Barrios, S., Gutiérrez, M., & Mayorga, M. (2014). Optimal method for content validity. Revista cubana de educación médica superior. Revista cubana de educación médica superior, 3, 547–558.
Van Deursen, A., Helsper, E. J., & Eynon, R. (2016). Development and validation of the Internet Skills Scale (ISS). Information, Communication & Society, 19(6), 804–823. https://doi.org/10.1080/1369118X.2015.1078834
Van Deursen, A., & Van Diepen, S. (2013). Information and strategic Internet skills of secondary students: A performance test. Computers & Education, 63, 218–226. https://doi.org/10.1016/j.compedu.2012.12.007
Van Deursen, A., & Van Dijk, J. (2011). Internet skills and the digital divide. New media & Society, 13, 893–911. https://doi.org/10.1177/1461444810386774
Vuorikari, R., Punie, Y., Carretero, S., & Brande, L. V. D. (2016). DigComp 2.0: The digital competence framework for citizens Scientific and Policy Report. Seville: Joint Research Centre of the European Commission.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
García-Valcárcel Muñoz-Repiso, A., Casillas Martín, S. & Basilotta Gómez-Pablos, V. Validation of an Indicator Model (INCODIES) for Assessing Student Digital Competence in Basic Education. J. New Approaches Educ. Res. 9, 110–125 (2020). https://doi.org/10.7821/naer.2020.1.459
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.7821/naer.2020.1.459