Skip to main content
SpringerLink
Log in

Speech and Eye Tracking Features for L2 Acquisition: A Multimodal Experiment

  • Conference paper
  • First Online:
Book cover Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium (AIED 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13356))

Included in the following conference series:

  • 3396 Accesses

Abstract

Spoken language variation analysis is increasingly considered in multimodal settings combining knowledge from computer, human and social sciences. This work focuses on second language (L2) acquisition via the study of linguistic variation combined with eye-tracking measures. Its goal is to model L2 pronunciation, to understand and to predict through AI techniques the related metacognitive information concerning reading strategies, text comprehension and L2 level. We present an experimental protocol involving a reading aloud setup, as well as first data collection to gather L2 speech with associated eye-tracking measures.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.techlearning.com/buying-guides/best-thermal-imaging-cameras-for-schools.

References

  1. Ai, H., Lu, X.: A web-based system for automatic measurement of lexical complexity. In: ALICO-2010, pp. 8–12, June 2010

    Google Scholar 

  2. Augereau, O., Fujiyoshi, H., Kise, K.: Towards an automated estimation of English skill via TOEIC score based on reading analysis. In: ICPR 2016, pp. 1285–1290 (2016)

    Google Scholar 

  3. Azmat, G., Iriberri, N.: The importance of relative performance feedback information: evidence from a natural experiment using high school students. J. Publ. Econ. 94(7–8), 435–452 (2010)

    Article  Google Scholar 

  4. Berzak, Y., Katz, B., Levy, R.: Assessing language proficiency from eye movements in reading. In: NAACL 2018: HLT, vol. 1, pp. 1986–1996. ACL, New Orleans, June 2018

    Google Scholar 

  5. Billières, M.: Méthode verbo tonale : diagnostic des erreurs surlaxe clair/sombre.www.verbotonale-phonetique.com/methode-verbo-tonale-diagnostic-erreurs-axe-clair-sombre/

  6. Derakhshan, A., Salehi, D., Rahimzadeh, M.: Computer-assisted language learning (call): pedagogical pros and cons. Int. J. Engl. Lang. Lit. Stud. 4, 111–120 (2015)

    Google Scholar 

  7. El Baha, M., Augereau, O., Kobylyanskaya, S., Vasilescu, I., Laurence, D.: Eye got it: a system for automatic calculation of the eye-voice span. In: 15th IAPR DAS (2022)

    Google Scholar 

  8. Flege, J.: Second language speech learning: theory, findings and problems, pp. 229–273 (1995)

    Google Scholar 

  9. Frenck-Mestre, C.: Eye-movement recording as a tool for studying syntactic processing in a second language: a review of methodologies and experimental findings. Second Lang. Res. 21 (2005)

    Google Scholar 

  10. Godfroid, A., Winke, P., Conklin, K.: Exploring the depths of second language processing with eye tracking: an introduction. Second Lang. Res. 36(3), 243–255 (2020)

    Article  Google Scholar 

  11. Goulas, S., Megalokonomou, R.: Knowing who you are: the effect of feedback information on short and long term outcomes. Economic rese, Department of Economics, University of Warwick (2015)

    Google Scholar 

  12. Guryan, J., Kim, J.S., Park, K.H.: Motivation and incentives in education: evidence from a summer reading experiment. Econ. Educ. Rev. 55, 1–20 (2016)

    Article  Google Scholar 

  13. Huettig, F., Rommers, J., Meyer, A.: Using the visual world paradigm to study language processing: a review and critical evaluation. Acta Psychologica 137, 151–71 (2011)

    Google Scholar 

  14. Kang, S.: Exploring L2 English learners articulatory problems using a read-aloud task (2020)

    Google Scholar 

  15. Kumar, M.: Advanced Educational Technology. Sankalp Publication, Bilaspur (2020)

    Google Scholar 

  16. Lallé, S., Conati, C., Azevedo, R.: Prediction of student achievement goals and emotion valence during interaction with pedagogical agents. In: AAMAS 2018, pp. 1222-1231 (2018)

    Google Scholar 

  17. Lallé, S., Murali, R., Conati, C., Azevedo, R.: Predicting co-occurring emotions from eye-tracking and interaction data in MetaTutor. In: Roll, I., McNamara, D., Sosnovsky, S., Luckin, R., Dimitrova, V. (eds.) AIED 2021. LNCS (LNAI), vol. 12748, pp. 241–254. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78292-4_20

    Chapter  Google Scholar 

  18. Levitt, S.D., List, J.A., Neckermann, S., Sadoff, S.: The behavioralist goes to school: Leveraging behavioral economics to improve educational performance. Am. Econ. J. Econ. Pol. 8(4), 183–219 (2016)

    Article  Google Scholar 

  19. Boula de Mareuil, P., Vieru-Dimulescu, B., Woehrling, C., Adda-Decker, M.: Accents étrangers et régionaux en francais: Caractérisation et identification. Traitement automatique des langues 49(3), 135–163 (2008)

    Google Scholar 

  20. Novikova, J., Balagopalan, A., Shkaruta, K., Rudzicz, F.: Lexical features are more vulnerable, syntactic features have more predictive power. CoRR abs/1910.00065 (2019)

    Google Scholar 

  21. Rayner, K.: The 35th sir Frederick Bartlett lecture: eye movements and attention in reading, scene perception, and visual search. Q. J. Exp. Psychol. 62(8), 1457–1506 (2009)

    Article  Google Scholar 

  22. Reinisch, E., Jesse, A., M. McQueen, J.: Early use of phonetic information in spoken word recognition: lexical stress drives eye movements immediately. Q. J. Exp. Psychol. 63(4), 772–783 (2010)

    Google Scholar 

  23. Roberts, L., Siyanova-Chanturia, A.: Using eye-tracking to investigate topics in L2 acquisition and L2 processing. Stud. Second Lang. Acquisit. 35 (2013)

    Google Scholar 

  24. Schotter, E., Fennell, A.: Readers can identify the meanings of words without looking at them: evidence from regressive eye movements. Psychon. Bull. Rev. 26 (2019)

    Google Scholar 

  25. Sims, S.D., Conati, C.: A Neural Architecture for Detecting User Confusion in Eye-Tracking Data, pp. 15–23. Association for Computing Machinery, New York (2020)

    Google Scholar 

  26. Stolcke, A., et al.: Automatic detection of sentence boundaries and disfluencies based on recognized words, January 1998

    Google Scholar 

  27. Szaszi, B., Palinkas, A., Palfi, B., Szollosi, A., Aczel, B.: A systematic scoping review of the choice architecture movement: toward understanding when and why nudges work. J. Behav. Decis. Mak. 31(3), 355–366 (2018)

    Article  Google Scholar 

  28. Thaler, R., Sunstein, C.: Nudge: Improving Decisions About Health, Wealth, and Happiness. Yale University Press, New Haven (2008)

    Google Scholar 

  29. Vasilescu, I., Adda-Decker, M.: Language, gender, speaking style and language proficiency as factors influencing the autonomous vocalic filler production in spontaneous speech, September 2006

    Google Scholar 

  30. Weijers, R.J., de Koning, B.B., Paas, F.: Nudging in education: from theory towards guidelines for successful implementation. Eur. J. Psychol. Educ. 36(3), 883–902 (2021)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LISN-CNRS, Orsay, France

    Sofiya Kobylyanskaya

  2. Paris-Saclay University, Gif-sur-Yvette, France

    Sofiya Kobylyanskaya

Authors
  1. Sofiya Kobylyanskaya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sofiya Kobylyanskaya .

Editor information

Editors and Affiliations

  1. Ateneo De Manila University, Quezon, Philippines

    Maria Mercedes Rodrigo

  2. Department of Computer Science, North Carolina State University, Raleigh, NC, USA

    Noburu Matsuda

  3. Durham University, Durham, UK

    Alexandra I. Cristea

  4. University of Leeds, Leeds, UK

    Vania Dimitrova

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kobylyanskaya, S. (2022). Speech and Eye Tracking Features for L2 Acquisition: A Multimodal Experiment. In: Rodrigo, M.M., Matsuda, N., Cristea, A.I., Dimitrova, V. (eds) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium. AIED 2022. Lecture Notes in Computer Science, vol 13356. Springer, Cham. https://doi.org/10.1007/978-3-031-11647-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-11647-6_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-11646-9

  • Online ISBN: 978-3-031-11647-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation