Multimodal Interactivity in Foreign Language Testing

  • Teresa Magal-RoyoEmail author
  • Jesús García Laborda


Multimodality in interactive digital environments for second language acquisition testing has been begun to be used only very lately. Some of the multimodality concepts have been recently developed in various research projects. Research into the automation of online university entrance exams has been prompted by the need to efficiently manage online tests and handle the task of exam marking semi-automatically. At the same time, we have addressed the use of multi-platform and/or multi-browser applications to handle the technical and functional validation of accessibility and so enable universal access. The application of multi-modularity to methods of navigation during the examination is possible in mobile devices that enable simultaneous interaction when students input data. This chapter aims to demonstrate the technical feasibility of the implementation process for an examination in regard to the technological and formal variables of navigation in the exam. These variables are currently being handled in mobile devices and the result is a further advancement in the process of Computer Aided Language Learning, CALL. This paper presents the multimodal approach has been taken into account in the development of an online prototype which validates the functional and technical assumptions adapted to university entrance exams for foreign languages.


Computer assisted language learning Multimodal interaction Computer-oriented multimodal user interfaces Usability Second language acquisition 



The authors would like to thank the Ministry of Economy and Competitiveness for funding the research project (with co-financing by ERDF) within the framework of the National R + D + I (2011–2014) “Guidance, proposals and teaching for English section in the entrance examination to the University” (Reference FFI2011-22442). The researchers would also thank the Spanish Ministry of Education, Culture, and Sports because without the grant for the Senior Researchers Mobility this paper would probably have not been possible.


  1. 1.
    Chapelle, C. A., Enright, M. K., & Jamieson, J. M. (2007). Building a validity argument for the test of English as a foreign language™. New York, NY: Routledge.Google Scholar
  2. 2.
    García Laborda, J. (2007). On the net: Introducing standardized EFL/ESL exams. Language Learning and Technology, 11(2), 3–9.Google Scholar
  3. 3.
    Weir, C. (2005). Language testing and validation: An evidence-based approach. New York, NY: Palgrave Macmillan.CrossRefGoogle Scholar
  4. 4.
    Garrett, N. (1991). Technology in the service of language learning: Trends and issues. Modern Language Journal, 75, 74–101.CrossRefGoogle Scholar
  5. 5.
    Roever, C. (2001). Web-based language testing. Language Learning and Technology, 5(2), 84–94.Google Scholar
  6. 6.
    Stricker, L. J., Wilder, G. S., & Rock, D. A. (2004). Attitudes about the computer-based test of English as a foreign language. Computers in Human Behavior, 20, 37–54.CrossRefGoogle Scholar
  7. 7.
    García Laborda, J., Magal-Royo, T., & Enriquez Carrasco, E. V. (2010). Teachers’ trialing procedures for Computer Assisted Language Testing Implementation. Eurasian Journal of Educational Research, 39, 161–174.Google Scholar
  8. 8.
    García Laborda, J. (2010). Necesitan las universidades españolas una prueba de acceso informatizada? El caso de la definición del constructo y la previsión del efecto en la enseñanza para idiomas extranjeros. Revista de orientación y Psicopadagogía, 21(1), 71–80.CrossRefGoogle Scholar
  9. 9.
    Chapelle, C. A., & Douglas, D. (2006). Assessing language through computer technology. New York, NY: CUP.CrossRefGoogle Scholar
  10. 10.
    Suvorov, R., & Hegelheimer, V. (2013). Computer-assisted language testing. In A. J. Kunnan (Ed.), Companion to language assessment (pp. 593–613). Malden, MA: Wiley-Blackwell.Google Scholar
  11. 11.
    Karabulut, A., LeVelle, K., Li, J., & Suvorov, R. (2012). Technology for French learning: A mismatch between expectations and reality. CALICO Journal, 29(2), 341–366.CrossRefGoogle Scholar
  12. 12.
    Chalhoub-Deville, M. (2010). Technology in standardized language assessments. In R. Kaplan (Ed.), Oxford handbook of applied linguistics (2nd ed.). Oxford: Oxford University Press.Google Scholar
  13. 13.
    García Laborda, J., Magal Royo, T., Litzler, M. F., & Giménez López, J. L. (2014). Mobile phones for a University Entrance Examination language test in Spain. Educational Technology and Society, 17(2), 17–30.Google Scholar
  14. 14.
    Sharma, R., Pavlovi, V. I., & Huang, T. S. (1998). Toward multimodal human–computer interface. Proceedings of the IEEE, 86(5), 853–869.CrossRefGoogle Scholar
  15. 15.
    World Wide Web Consortium, W3C (2003). Device independence principles. Accessed 12 Dec 2015.
  16. 16.
    Larson, J. A. (2010). Standard languages for developing multimodal applications. Accessed 12 Dec 2015.
  17. 17.
    Zander, T. O., Kothe, C., Jatzev, S., & Gaertner, M. (2010). Enhancing human–computer interaction with input from active and passive brain–computer interfaces. Human-Computer Interaction Series, 0(3), 181–199. doi: 10.1007/978-1-84996-272-8_11. Accessed 12 Dec 2015.
  18. 18.
    World Wide Web Consortium, W3C (2009). EMMA: Extensible MultiModal Annotation markup language. Accessed 12 Dec 2015.
  19. 19.
    World Wide Web Consortium, W3C (2009). Guía Breve de Interacción Multimodal. Oficina Española del W3C. Accessed 12 Dec 2015.
  20. 20.
    World Wide Web Consortium, W3C (2009). Multimodal architecture and interfaces. Accessed 12 Dec 2015.
  21. 21.
    Oviatt, S., DeAngeli, A., & Kuhn K. (1997). Integration and synchronization of input modes during multimodal human–computer interaction. In Proceedings of Conference Human Factors in Computing Systems, CHI’97, Atlanta, pp. 415–422.Google Scholar
  22. 22.
    García Laborda, J., Magal-Royo, T., Da Rocha Siqueira, J. M., & Fernández Alvarez, M. (2010). Ergonomics factors in English as a foreign language testing: The case of PLEVALEX. Computers and Education, 54(2), 384–391.CrossRefGoogle Scholar
  23. 23.
    Giménez-López, J. L., Magal-Royo, T., Garde Calvo, F., & Prefasi Gomar, S. (2009). The adaptation of contents for the creation of foreign language learning exams for mobile devices. IMCL2009. International Journal of Interactive Mobile Technologies, 3, Special Issue ISSN: 1865–7923.Google Scholar
  24. 24.
    Giménez-López, J. L., Magal-Royo, T., García Laborda, J., & Garde Calvo, F. (2010). Methods of adapting digital content for the learning process via mobile devices. Procedia—Social and Behavioral Sciences, 1, 2673–2677.CrossRefGoogle Scholar
  25. 25.
    Fuster-Duran, A. (1996). Perception of conflicting audio-visual speech: An examination across Spanish and German. In D. G. Stork & M. E. Hennecke (Eds.), Speech reading by humans and machines: Models, systems and applications (pp. 135–143). New York, NY: Springer.CrossRefGoogle Scholar
  26. 26.
    Magal-Royo, T., García Laborda, J., Peris-Fajarnes, G., & Spachtholz, P. (2007). Visual learning through guided iconography in wireless scenarios. In Proceedings of ECEL 2007. 6th European Conference on e-Learning, pp. 415–418.Google Scholar
  27. 27.
    Magal-Royo, T., Peris-Fajarnes, G., Tortajada Montañana, I., & Defez García, B. (2007). Evaluation methods on usability of m-learning environments. International Journal of Interactive Mobile Technologies, 1(1), ISSN: 1865–7923.Google Scholar
  28. 28.
    Magal-Royo, T., Gimenez-Lopez, J. L., & Pairy, B. (2011). Multimodal applications for foreign language teaching. In 14th International Conference on Interactive Collaborative Learning, ICL, Piestany, Slovakia, pp. 145–148.Google Scholar
  29. 29.
    Magal-Royo, T., Laborda, G. J., & Gimenez-Lopez, J. L. (2011). Accessible multimodal interaction for language learning on mobile devices. In Q. Y. Zhou (Ed.) International Conference on Applied Social Science (ICASS 2011) Changsha, China, March 19–20.Google Scholar
  30. 30.
    Campillo, L., & Lanos, L. (2010). Tecnologías del habla y análisis de la voz. Aplicaciones en la enseñanza de la lengua. Revista Dialogo de la lengua. Accessed 12 Dec 2015.
  31. 31.
    Lingle, V., & Deshpande, A. (2010). Online multimodal interaction for speech interpretation. International Journal of Computer Applications, 1(19), 81–85.Google Scholar
  32. 32.
    Magal-Royo, T., & Giménez Lopez, J. L. (2012). Multimodal interactivity in the foreign language section of the Spanish university admission examination. Revista Educacion, 357, 163–176.Google Scholar
  33. 33.
    Chittaro, L. (2010). Distinctive aspects of mobile interaction and their implications for the design of multimodal interfaces. Journal on Multimodal User Interfaces, 3(3), 157–165. SpringerLink (Ed.).CrossRefGoogle Scholar
  34. 34.
    König, W. A., Rädle, R., & Reiterer, H. (2010). Interactive design of multimodal user interfaces. Reducing technical and visual complexity. Journal on Multimodal User Interfaces, 3(3), 197–213. doi: 10.1007/s12193-010-0044-2. SpringerLink Ed.CrossRefGoogle Scholar
  35. 35.
    Tan D., & Nijholt, A. (2010).Brain–computer interfaces and human–computer interaction. Brain-Computer Interfaces. Human-Computer Interaction Series, 0(1), 3–19. doi: 10.1007/978-1-84996-272-8_1. Accessed 12 Dec 2015.
  36. 36.
    Alseid, M., & Rigas, D. (2008). An empirical Investigation into the use of multimodal E-learning interfaces. In S. Pinder (Ed.), Advances in human-computer interaction (Vol. 5, pp. 85–100). USA: Hindawi Publishing Corporation.Google Scholar
  37. 37.
    MCgee-lennon, M., Nigay, L., & Gray, P. (2010). The challenges of engineering multimodal interaction. Journal on Multimodal User Interfaces, 3(3), 155–156. SpringerLink Ed. Accessed 12 Dec 2015.CrossRefGoogle Scholar
  38. 38.
    Alwan, A., Vijian, B., Black, M., Casey, L., Gerosa, M., Heritage, M., et al. (2007). A system for technology based assessment of language and literacy in young children: The role of multiple information sources. In IEEE 9th Workshop on Multimedia Signal Processing, pp. 26–30. Accessed 12 Dec 2015.
  39. 39.
    García Laborda, J., & Litzler, M. F. (in press). Students’ opinions about ubiquitous delivery of standardized English exams. Porta Linguarum.Google Scholar
  40. 40.
    García Laborda, J., Magal-Royo, T., & Bakieva, M. (2016). Looking to the future of language assessment: Tablet PCs usability in language testing. Journal of Universal Computer Science, 22(1).Google Scholar
  41. 41.
    Duarte, C. (2008). Design and evaluation of adaptive multimodal systems. Doctoral theses, Departamento de Informática, Universidad de Lisboa. Accessed 12 Dec 2015.

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  1. 1.Universitat Politécnica de ValenciaValenciaSpain
  2. 2.Universidad de Alcalá de HenaresAlcalá de Henares, MadridSpain

Personalised recommendations