Skip to main content

LextPT: A reliable and efficient vocabulary size test for L2 Portuguese proficiency

Abstract

Vocabulary size has been repeatedly shown to be a good indicator of second language (L2) proficiency. Among the many existing vocabulary tests, the LexTALE test and its equivalents are growing in popularity since they provide a rapid (within 5 minutes) and objective way to assess the L2 proficiency of several languages (English, French, Spanish, Chinese, and Italian) in experimental research. In this study, expanding on the standard procedure of test construction in previous LexTALE tests, we develop a vocabulary size test for L2 Portuguese proficiency: LextPT. The selected lexical items fall in the same frequency interval in European and Brazilian Portuguese, so that LextPT accommodates both varieties. A large-scale validation study with 452 L2 learners of Portuguese shows that LextPT is not only a sound and effective instrument to measure L2 lexical knowledge and indicate the proficiency of both European and Brazilian Portuguese, but is also appropriate for learners with different L1 backgrounds (e.g. Chinese, Germanic, Romance, Slavic). The construction of LextPT, apart from joining the effort to provide a standardised assessment of L2 proficiency across languages, shows that the LexTALE tests can be extended to cover different varieties of a language, and that they are applicable to bilinguals with different linguistic experience.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Notes

  1. The Test of English for International Communication (TOEIC) is a standard English proficiency test for non-native speakers.

  2. The percentage of subjects selected for participation under the prediction, but who did not actually obtain the minimum QPT score required.

  3. The Common European Framework of Reference for Languages (CEFR), led by the European Council and launched in 2001, is a worldwide standard for organiszing foreign language proficiency in six levels (A1, A2, BA, B2, C1, C2), currently available in 40 European and non-European languages.

  4. Portuguese is the language of over 230 million people, about 15 million of whom are speakers of European Portuguese (Segura, 2013), and more than 170 million of whom are speakers of Brazilian Portuguese (Mattos e & Silva, 2013). However, it is difficult to accurately count the number of speakers of the Angolan, Mozambican, Cape Verdean, Guinean, São Tomean, Timorese, and Galician varieties.

  5. SUBTLEX_PT is a lexical database containing 132,.710 Portuguese words, obtained from a 78-million-word corpus based on subtitles of European Portuguese film and television series screened between 1990 and 2011.

  6. SUBLEXT_PT_BR comprises 136,.147 word types obtained from 61 million words of conversational Brazilian Portuguese.

  7. The nonword items are in fact all pseudo-words. But in line with previous LexTALE tests, we refer to them here as nonwords.

  8. Although both orthographic forms, oxigénio and oxigênio, can be found in SUBTLEX-PT-BR (Tang, 2012), 14 out of 61 L1-BP speakers rejected oxigénio as a real Portuguese word.

  9. CAPLE (Centro de Avaliação de Português Língua Estrangeira or Centre for Evaluation of Portuguese as a Foreign Language) exams, developed by the University of Lisbon, aim at certifying the proficiency of European Portuguese as a foreign language, offered at six reference levels, from A1 (beginner) to C2 (near-native), conforming to the Common European Framework of Reference of Languages (CEFR). For detailed information, please consult: https://caple.letras.ulisboa.pt/pagina/1/caple

  10. CELPE-Bras (Certificado de Proficiência em Língua Portuguesa para Estrangeiros or Certificate of Proficiency in Portuguese for Foreigners), developed by the Brazilian Ministry of Education, is an official exam that certifies the proficiency of Brazilian Portuguese as a foreign language by assigning candidates to one of the four levels of proficiency: intermediate, upper intermediate, advanced or highly advanced. For detailed information, please consult: https://www.gov.br/inep/pt-br/areas-de-atuacao/avaliacao-e-exames-educacionais/celpe-bras#

  11. In line with Amenta et al. (2020), we consider that selecting all items instantiates a response strategy rather than a real performance on the test.

  12. One of the L2 learners of BP reported having the “highly advanced” certificate, but this was not sufficient for between-group comparison.

References

  • Amenta, S., Badan, L., & Brysbaert, M. (2020). LexITA: A Quick and Reliable Assessment Tool for Italian L2 Receptive Vocabulary Size. Applied Linguistics, 2020, 1-24. https://doi.org/10.1093/applin/amaa020

    Article  Google Scholar 

  • Astika, G. G. (1993). Analytical assessment of foreign students’ writing. RELC Journal, 24, 61-72. https://doi.org/10.1177/003368829302400104

    Article  Google Scholar 

  • Barreiro, A., Wittmann, L. H., & Pereira, M. J. (1996). Lexical differences between European and Brazilian Portuguese. INESC Journal of Research and Development, 5(2), 75–101.

    Google Scholar 

  • Bateman, B. E., & de Almeida Oliveira, D. (2014). Students’ Motivations for Choosing (or Not) to Study Portuguese: A Survey of Beginning-level University Classes. Hispania, 97(2), 264–280. http://www.jstor.org/stable/24368776

    Article  Google Scholar 

  • Beglar, D., & Hunt, A. (1999). Revising and Validating the 2000 Word Level and University Word Level Vocabulary Tests. Language Testing 16(2). 131-162. https://doi.org/10.1177/026553229901600202

    Article  Google Scholar 

  • Bonk, W. (2000). Second Language Lexical Knowledge and Listening Comprehension. International Journal of Listening, 14(1), 14-31. https://doi.org/10.1080/10904018.2000.10499033

    Article  Google Scholar 

  • Brysbaert, M. & New, B. (2009). Moving beyond Kucera and Francis: A Critical Evaluation of Current Word Frequency Norms and the Introduction of a New and Improved Word Frequency Measure for American English. Behavior Research Methods, 41, 997-990. https://doi.org/10.3758/BRM.41.4.977

    Article  Google Scholar 

  • Brysbaert, M. (2013). Lextale_FR a fast, free, and efficient test to measure language proficiency in French. Psychologica Belgica, 53(1), 23–37. https://doi.org/10.5334/pb-53-1-23

    Article  Google Scholar 

  • Ferré, P., & Brysbaert, M. (2017). Can Lextale-Esp discriminate between groups of highly proficient Catalan-Spanish bilinguals with different language dominances?. Behavior research methods, 49(2), 717–723. https://doi.org/10.3758/s13428-016-0728-y

    Article  PubMed  Google Scholar 

  • Chan, I. L., & Chang, C. B. (2018). LEXTALE_CH: A quick, character-based proficiency test for Mandarin Chinese. in Proceedings of the Annual Boston University Conference on Language Development (BUCLD) 42: 114 – 130.

  • Comrie, B. (1989). Language Universals and Linguistic Typology: Syntax and Morphology, Second Edition. The University of Chicago Press.

  • Diependaele, K., Lemhöfer, K., & Brysbaert, M. (2013). The word frequency effect in first- and second-language word recognition: a lexical entrenchment account. Quarterly Journal of Experimental Psychology, 66(5), 843–863. https://doi.org/10.1080/17470218.2012.720994

    Article  Google Scholar 

  • Eyckmans, J. (2004). Measuring Receptive Vocabulary Size. LOT.

  • Hulstijn, J. H. (2010). Measuring second language proficiency. In Blom E., & Unsworth, S. (eds): Experimental Methods in Language Acquisition Research (pp. 185–200). : Benjamins.

    Chapter  Google Scholar 

  • Izura, C., Cuetos, F., and Brysbaert, M. (2014). Lextale-Esp: A test to rapidly and efficiently assess the Spanish vocabulary size. Psicológica, 35(1), 49–66.

    Google Scholar 

  • Justi, F. R. R., Justi, C. N. G., & Roazzi, A. (2014). Efeitos da similaridade ortográfica das pseudopalavras no acesso lexical [Pseudowords' orthographic similarity effects on lexical access]. Arquivos Brasileiros de Psicologia 66(3). 133-147. http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1809-52672014000300011&lng=pt&tlng=pt

    Google Scholar 

  • Khare, V., Verma, A., Kar, B., Srinivasan, N., & Brysbaert, M. (2013). Bilingualism and the increased attentional blink effect: Evidence that the difference between bilinguals and monolinguals generalizes to different levels of second language proficiency. Psychological Research 77(6), 728–73. https://doi.org/10.1007/s00426-012-0466-4

    Article  PubMed  Google Scholar 

  • Laufer, B. (1992). How much lexis is necessary for reading comprehension. In Arnaud P. J. L., & Béjoint, H. (eds.): Vocabulary and applied linguistics (pp. 126-132). : Macmillan.

    Chapter  Google Scholar 

  • Laufer, B. (1998). The development of passive and active vocabulary in a second language: Same or different. Applied Linguistics, 19, 255-271. https://doi.org/10.1093/applin/19.2.255

    Article  Google Scholar 

  • LeBlanc, R. & Painchaud, G. (1985). Self-assessment as a second language placement instrument. TESOL Quarterly 19(4), 673–687. https://doi.org/10.2307/3586670

    Article  Google Scholar 

  • Leclercq, P. & Edmonds, A. (2014). How to assess l2 proficiency? An overview of proficiency assessment research. In Leclercq, P., Edmonds, A., & Hilton, H. (eds): Measuring L2 Proficiency: Perspectives from SLA (pp.3–23). https://doi.org/10.21832/9781783092291-004

  • Lemhöfer, K., & Broersma, M. (2012). Introducing LexTALE: a quick and valid Lexical Test for Advanced Learners of English. Behavior research methods, 44(2), 325–343. https://doi.org/10.3758/s13428-011-0146-0

    Article  PubMed  Google Scholar 

  • Marian, V., Blumenfeld, H. K., & Kaushanskaya, M. (2007). The language experience and proficiency questionnaire (LEAP-Q). Journal of Speech, Language and Hearing Research 50(4), 940–967.

    Article  Google Scholar 

  • Mattos e Silva, R. V. (2013). O Português do Brasil [Brazilian Portuguese]. In: E. B. P. Raposo, E. B. P., Nascimento, M. A. C., Mota, L., & Mendes, A. (eds.), Gramática do português (pp.145-154). Lisboa: Fundação Calouste Gulbenkian.

  • Meara, P. (1992). EFL vocabulary tests. Swansea: Centre for Applied Language Studies, University of Wales Swansea.

  • Meara, P. (1996). The dimensions of lexical competence. In: Brown, G., K. Malmkjaer, & Williams, J. (eds.), Performance and Competence in Second Language Acquisition (pp. 35–53). Cambridge University Press.

    Google Scholar 

  • Meara, P. & Buxton, B. (1987). An alternative to multiple choice vocabulary tests, Language Testing, 4, 142–154. https://doi.org/10.1177/026553228700400202

    Article  Google Scholar 

  • Milton J., Wade, J., & Hopkins, H. (2010). Aural word recognition and oral competence in a foreign language. In Chacón-Beltrán, R., Abello-Contesse, C., & Torreblanca-López, M. (eds.): Further insights into non-native vocabulary teaching and learning (pp. 83-98). : Multilingual Matters.

    Chapter  Google Scholar 

  • Milton, J. (2013). Measuring the contribution of vocabulary knowledge to proficiency in the four skills. In Bardel, C., Lindquist, C., & Laufer, B. (eds.): L2 Vocabulary Acquisition, Knowledge, and Use: New Perspectives on Assessment and corpus Analysis (pp. 57-78). Amsterdam, the Netherlands: European Second Language Association.

  • Molsing, K. V., Lopes Perna, C. B., & Tramunt Ibaños, A. M. (2020). Linguistic Approaches to Portuguese as an Additional Language. Amsterdam. John Benjamins Publishing Company.

  • Nation, I. S. P. (2001). Learning Vocabulary in Another Language. Cambridge University Press.

    Book  Google Scholar 

  • Nation, I. S. P. (2013). Vocabulary Size in a Second Language. In Chapelle, C. A. (eds), The Encyclopedia of Applied Linguistics. https://doi.org/10.1002/9781405198431.wbeal1270

  • Nizonkiza, D. (2011). The relationship between lexical competence, collocational competence, and second language proficiency. English Text Construction 4(1), 113-145. https://doi.org/10.1075/etc.4.1.06niz9

    Article  Google Scholar 

  • Oscarson, M. (1989). Self-assessment of language proficiency: Rationale and applications. Language Testing, 6(1), 1–13. https://doi.org/10.1177/026553228900600103

    Article  Google Scholar 

  • Qian, D. D. (1999). Assessing the roles of depth and breadth of vocabulary knowledge. The Canadian Modern Language Review, 56, 282-307. https://doi.org/10.3138/cmlr.56.2.282

    Article  Google Scholar 

  • R Core Team. (2020). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/.

  • Revelle, W. (2021). Psych: Procedures for psychological, psychometric, and personality research. Evanston, Illinois: Northwestern University. R package version 2.1.9. https://CRAN.R-project.org/package=psych.

  • Rizopoulos, D. (2006). ltm: An R package for latent variable modeling and item response theory analyses. Journal of Statistical Software, 17(5), 1–25. https://doi.org/10.18637/jss.v017.i05

    Article  Google Scholar 

  • Segura, L. (2013). Geografia da língua portuguesa [Geographical distribution of Portuguese speakers]. In: E. B. P. Raposo, E. B. P., Nascimento, M. A. C., Mota, L., & Mendes, A. (eds.), Gramática do português (pp.145-154). Lisboa: Fundação Calouste Gulbenkian.

  • Stæhr, L. S. (2008). Vocabulary size and the skills of listening, reading and writing. Language Learning Journal, 36(2), 139-152. https://doi.org/10.1080/09571730802389975

    Article  Google Scholar 

  • Strong-Klause, D. (2000). Exploring the effectiveness of self-assessment strategies in ESL placement. In Ekbatani, G., & Pierson, H. (eds.), Learner-directed assessment in ESL (pp. 49–73). : Lawrence Erlbaum Associates, Inc.

  • Soares, A. P., Machado, J., Costa, A., Iriarte, Á., Simões, A., de Almeida, J. J., Comesaña, M., & Perea, M. (2015). On the advantages of word frequency and contextual diversity measures extracted from subtitles: The case of Portuguese. Quarterly journal of experimental psychology, 68(4), 680–696. https://doi.org/10.1080/17470218.2014.964271

    Article  Google Scholar 

  • Sollai, S., Alvim, R., Bianconi, C. & Parma, A. (2018). Portuguese is in! From less commonly taught to critical to world language. Todas as Letras Revista de Língua e Literatura. 20. https://doi.org/10.5935/1980-6914/letras.v20n1p105-121.

  • Tang, K. (2012). A 61 Million Word Corpus of Brazilian Portuguese Film Subtitles as a Resource for Linguistic Research. In University College London, Working Papers in Linguistics, 25, 208-214.

    Google Scholar 

  • Thomas, M. (1994). Assessment of L2 Proficiency in Second Language Acquisition Research. Language Learning, 44(2), 307–336. https://doi.org/10.1111/j.1467-1770.1994.tb01104.x

    Article  Google Scholar 

  • Tremblay, A. (2011). Proficiency assessment standard in second language acquisition research “Clozing” the gap. Studies in Second Language Acquisition, 33(3), 339–372. https://doi.org/10.1017/S0272263111000015

    Article  Google Scholar 

  • Venâncio, R. (2018). Geração de Pseudopalavras para Avaliação Linguística [Pseudo-word Generator for Linguistic Evaluation]. (Unpublished master’s thesis), University of Coimbra, Coimbra, Portugal.

  • Wetzels, W. L. , Costa, J., & Menuzzi, S. (2016). The handbook of portuguese linguistics. Malden: WILEY Blackwell.

  • Wittmann, L., Pêgo, T., & Santos, D. (1995). Português do Brasil e de Portugal: alguns contrastes. In Actas do XI Encontro Nacional da Associação Portuguesa de Linguística, 465–487. Lisboa: APL/Colibri.

  • Zareva, A., Schwanenflugel, P., & Nikolova, Y. (2005). Relationship Between Lexical Competence and Language Proficiency: Variable Sensitivity. Studies in Second Language Acquisition, 27(4), 567–595. https://doi.org/10.1017/S0272263105050254

    Article  Google Scholar 

  • Zimmerman, K. J. (2004). The role of Vocabulary Size in Assessing Second Language Proficiency. (Unpublished master’s thesis), Brigham Young University, Utah, USA.

Download references

Acknowledgements

We would like to thank João Veríssimo for his comments and suggestions on earlier versions of this article.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xinyi Li.

Additional information

Open Practices Statement

None of the data or materials for the experiments reported here are available, and none of the experiments were preregistered.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

ESM 1

(PDF 75 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhou, C., Li, X. LextPT: A reliable and efficient vocabulary size test for L2 Portuguese proficiency. Behav Res (2021). https://doi.org/10.3758/s13428-021-01731-1

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.3758/s13428-021-01731-1

Keywords

  • L2 proficiency
  • Vocabulary size test
  • Bilingualism
  • Brazilian Portuguese
  • European Portuguese