Advertisement

Assessing Pronunciation

  • Martha C. Pennington
  • Pamela Rogerson-Revell
Chapter
Part of the Research and Practice in Applied Linguistics book series (RPAL)

Abstract

The complexity of pronunciation presents challenges in both what and how to measure this aspect of language proficiency. Standardized tests of spoken language proficiency typically incorporate pronunciation, though with varying and often inconsistent measures and criteria. Traditionally, pronunciation has been assessed through human rating, which has received considerable research attention, while recently computerized assessment of speaking proficiency that includes pronunciation has been developed and researched. The assessment of pronunciation can also be taken to include research on pronunciation aptitude and its relation to pronunciation achievement. Current discussions on the nature of pronunciation and the increased demand for testing of language competence for academic study and for specific jobs suggest the need for further attention to micro and macro aspects of pronunciation in speech production and perception.

References

  1. Abrahamsson, N., & Hyltenstam, K. (2009). Age of onset and nativelikeness in a second language: Listener perception versus linguistic scrutiny. Language Learning, 59(2), 249–306.  https://doi.org/10.1111/j.1467-9922.2009.00507.x
  2. American Council on the Teaching of Foreign Languages. (2012). ACTFL proficiency guidelines 2012. Alexandria, VA: American Council on the Teaching of Foreign Languages.Google Scholar
  3. Anderson-Hsieh, J., & Koehler, K. (1988). The effect of foreign accent and speaking rate on native speaker comprehension. Language Learning, 38(4), 561–613.  https://doi.org/10.1111/j.1467-1770.1988.tb00167.x
  4. Ballard, L., & Winke, P. (2017). Students’ attitudes towards English teachers’ accents: The interplay of accent familiarity, comprehensibility, intelligibility, perceived native speaker status, and acceptability as a teacher. In T. Isaacs & P. Trofimovich (Eds.), Second language pronunciation assessment: Interdisciplinary perspectives (pp. 121–140). Bristol, UK and Blue Ridge Summit, PA: Multilingual Matters.Google Scholar
  5. Bernstein, J. (2004). Spoken language testing: Ordinate’s SET-10. Presentation given at the TESOL Convention, April 2004, Long Beach, CA. Retrieved September 2, 2017, from https://www.versanttest.com/technology/featured/3.TESOL-04.SET.JB.pdf
  6. Bernstein, J., Van Moere, A., & Cheng, J. (2010). Validating automated speaking tests. Language Testing, 27(3), 355–377.  https://doi.org/10.1177/0265532210364404CrossRefGoogle Scholar
  7. Bosker, H. R., Pinget, A.-F., Quené, H., Sanders, T., & de Jong, N. H. (2012). What makes speech sound fluent? The contributions of pauses, speed and repairs. Language Testing, 30(2), 159–175.  https://doi.org/10.1177/0265532212455394CrossRefGoogle Scholar
  8. Brown, J. D. (2005). Testing in language programs: A comprehensive guide to English language assessment. Upper Saddle River, NJ: Prentice Hall Regents.Google Scholar
  9. Browne, K., & Fulcher, G. (2017). Pronunciation and intelligibility in assessing spoken fluency. In T. Isaacs & P. Trofimovich (Eds.), Second language pronunciation assessment: Interdisciplinary perspectives (pp. 37–53). Bristol, UK and Blue Ridge Summit, PA: Multilingual Matters.Google Scholar
  10. Bygate, M. (1987). Speaking. Oxford: Oxford University Press.Google Scholar
  11. Carey, M. D., Mannell, R. H., & Dunn, P. K. (2011). Does a rater’s familiarity with a candidate’s pronunciation affect the rating in oral proficiency interviews? Language Testing, 28(2), 201–219.  https://doi.org/10.1177/0265532210393704CrossRefGoogle Scholar
  12. Carroll, J. B., & Sapon, S. (2002/1959). Modern language aptitude test. San Antonio, TX and Bethesda, MD: Psychological Corporation and Second Language Teaching, Inc.Google Scholar
  13. Chalhoub-Deville, M. (2003). Second language interaction: Current perspectives and future trends. Language Testing, 20(4), 369–383.  https://doi.org/10.1191/0265532203lt264oaCrossRefGoogle Scholar
  14. Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge: Cambridge University Press.Google Scholar
  15. Council of Europe. (2017). Common European framework of reference for languages: Learning, teaching, assessment. Companion volume with new descriptors. Retrieved February 26, 2018, from https://rm.coe.int/cefr-companion-volume-with-new-descriptors-2018/1680787989
  16. Couper, G. (2011). What makes pronunciation teaching work? Testing for the effect of two variables: Socially constructed metalanguage and critical listening. Language Awareness, 20(3), 159–182.  https://doi.org/10.1080/09658416.2011.570347CrossRefGoogle Scholar
  17. Couper, G. (2015). Applying theories of language and learning to teaching pronunciation. In M. Reed & J. M. Levis (Eds.), Handbook of English pronunciation (pp. 413–432). New York: John Wiley & Sons.Google Scholar
  18. Cruttenden, A. (2014). Gimson’s pronunciation of English (8th ed.). London: Routledge.CrossRefGoogle Scholar
  19. Cucchiarini, C., Strik, H., Binnenpoorte, D., & Boves, L. (2002). Pronunciation evaluation in read and spontaneous speech: A comparison between human ratings and automatic scores. In A. James & J. Leather (Eds.), New sounds 2000, Proceedings of the fourth international symposium on the Acquisition of Second-Language Speech, University of Amsterdam, September 2000 (pp. 72–79). Klagenfurt, Austria: University of Austria.Google Scholar
  20. Cucchiarini, C., Strik, H., & Boves, L. (2000). Quantitative assessment of second language learners’ fluency by means of automatic speech recognition technology. Journal of the Acoustical Society of America, 107(2), 989–999.  https://doi.org/10.1121/1.428279CrossRefGoogle Scholar
  21. Davies, A. (2010). Language assessment in call centres: The case of the customer service representative. In G. Forey & J. Lockwood (Eds.), Globalization, communication and the workplace (pp. 242–248). London: Continuum.Google Scholar
  22. Davies, A. (2017). Commentary on the native speaker status in pronunciation research. In T. Isaacs & P. Trofimovich (Eds.), Second language pronunciation assessment: Interdisciplinary perspectives (pp. 185–192). Bristol, UK and Blue Ridge Summit, PA: Multilingual Matters.Google Scholar
  23. de Bot, K. (1992). A bilingual production model: Levelt’s ‘speaking’ model adapted. Applied Linguistics, 13(1), 1–24. Retreived January 10, 2018, from https://www.researchgate.net/profile/Kees_De_Bot2/publication/249237191_A_Bilingual_Production_Model_Levelt%27s_%27Speaking%27_Model_Adapted/links/0deec537b3b8b45d98000000/A-Bilingual-Production-Model-Levelts-Speaking-Model-Adapted.pdf
  24. Derwing, T. M., Rossiter, M. J., Munro, M. J., & Thomson, R. I. (2004). Second language fluency: Judgments on different tasks. Language Learning, 54(4), 655–679.  https://doi.org/10.1111/j.1467-9922.2004.00282.xCrossRefGoogle Scholar
  25. Fulcher, G. (1996). Testing tasks: Issues in task design and the group oral. Language Testing, 13(1), 23–51.  https://doi.org/10.1177/026553229601300103CrossRefGoogle Scholar
  26. Fulcher, G. (2015). Assessing second language speaking. Language Teaching, 48(2), 198–216.  https://doi.org/10.1017/S0261444814000391CrossRefGoogle Scholar
  27. Granena, G., & Long, M. H. (2013). Age of onset, length of residence, language aptitude, and ultimate L2 attainment in three linguistic domains. Second Language Research, 29(3), 311–343.  https://doi.org/10.1177/0267658312461497
  28. Harding, L. (2017). What do raters need in a pronunciation scale? The user’s view. In T. Isaacs & P. Trofimovich (Eds.), Second language pronunciation assessment: Interdisciplinary perspectives (pp. 12–34). Bristol, UK and Blue Ridge Summit, PA: Multilingual Matters.Google Scholar
  29. Hincks, R. (2001). Using speech recognition to evaluate skills in spoken English. Lund University, Dept. of Linguistics Working Papers, 49, 58–61. Retrieved August 15, 2017, from journals.lub.lu.se/index.php/LWPL/article/viewFile/2367/1942
  30. Hu, X., Ackermann, J., Martin, J. A., Erb, M., Winkler, S., & Reiterer, S. M. (2013). Language aptitude for pronunciation in advanced second language (L2) learners: Behavioural predictors and neural substrates. Brain and Language, 127(3), 366–376.  https://doi.org/10.1016/j.bandl.2012.11.006CrossRefGoogle Scholar
  31. Isaacs, T. (2014). Assessing pronunciation. In A. J. Kunnan (Ed.), The companion to language assessment (pp. 140–155). Hoboken, NJ: Wiley-Blackwell.Google Scholar
  32. Isaacs, T., & Trofimovich, P. (2012). Deconstructing comprehensibility. Studies in Second Language Acquisition, 34(4), 475–505.  https://doi.org/10.1017/S0272263112000150CrossRefGoogle Scholar
  33. Isaacs, T., & Trofimovich, P. (2017). Key themes, constructs and interdisciplinary perspectives in second language pronunciation assessment. In T. Isaacs & P. Trofimovich (Eds.), Second language pronunciation assessment: Interdisciplinary perspectives (pp. 3–11). Bristol, UK and Blue Ridge Summit, PA: Multilingual Matters.Google Scholar
  34. Isaacs, T., Trofimovich, P., Yu, G., & Chereau, B. M. (2011). Examining the linguistic aspects of speech that most efficiently discriminate between upper levels of the revised IELTS pronunciation scale. IELTS Research Reports Online, 4, 1–48. Retrieved September 2, 2017, from http://www.ielts.org/-/media/research-reports/ielts_online_rr_2015-4.ashx
  35. Jenkins, J. (2000). The phonology of English as an international language. Oxford: Oxford University Press.Google Scholar
  36. Johnson, M. (2000). Interaction in the oral proficiency interview: Problems of validity. Pragmatics, 10(2), 215–231.  https://doi.org/10.1075/prag.10.2.03johCrossRefGoogle Scholar
  37. Kane, M. T. (1990). An argument-based approach to validation. Research report no. ACT-RR-90-13. Iowa City, Iowa: American College Testing Program. ERIC Document No. ED 336428.Google Scholar
  38. Kang, O., Moran, M., & Vo, S. (2016). Perceptual judgments of accented speech by listeners from different first language backgrounds. TESL-EJ, 20(1). Retrieved February 22, 2018, from http://www.tesl-ej.org/wordpress/issues/volume20/ej77/ej77a1/
  39. Kormos, J. (2006). Speech production and second language acquisition. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  40. Labov, W. (1966). The social stratification of English in New York City. Washington, DC: Center for Applied Linguistics.Google Scholar
  41. Lai, E. R. (2011, May). Performance assessment: Some new thoughts on an old idea. Pearson Bulletin, 20. Retrieved December 12, 2016, from https://www.pearsonassessments.com/research
  42. Lennon, P. (1990). Investigating fluency in EFL: A qualitative approach. Language Learning, 40(3), 387–417.  https://doi.org/10.1111/j.1467-1770.1990.tb00669.xCrossRefGoogle Scholar
  43. Levis, J. M. (2006). Pronunciation and the assessment of spoken language. In R. Hughes (Ed.), Spoken English, TESOL and applied linguistics: Challenges for theory and practice (pp. 245–270). New York: Palgrave Macmillan.CrossRefGoogle Scholar
  44. Levis, J. (2007). Computer technology in teaching and researching pronunciation. Annual Review of Applied Linguistics, 27, 184–202.  https://doi.org/10.1017/S0267190508070098CrossRefGoogle Scholar
  45. Loukina, A., Zechner, K., Chen, L., & Heilman, M. (2015). Feature selection for automated speech scoring. In Proceedings of the tenth workshop on innovative use of NLP for building educational applications (pp. 12–19). Denver, CO: Association for Computational Linguistics.CrossRefGoogle Scholar
  46. Meara, P. (2005). LLAMA language aptitude tests. Swansea, UK: Lognostics.Google Scholar
  47. Messick, S. (1996). Validity of performance assessments. In G. W. Phillips (Ed.), Technical Issues in Large-Scale Performance Assessments (Chap. 1, pp. 1–18). Washington, DC: National Center for Education Statistics, U. S. Department of Education.Google Scholar
  48. Mompean, J. A. (2006). Introduction: Cognitive phonology in cognitive linguistics. International Journal of English Studies, 6, vii–xii. Retrieved August 17, 2017, from https://www.scribd.com/document/298249578/Cognitive-Phonology-Ijes-6-2
  49. Mompean, J. A. (2014). Phonology. In J. R. Taylor & J. Littlemore (Eds.), Bloomsbury companion to cognitive linguistics (pp. 357–392). London: Bloomsbury.Google Scholar
  50. Munro, M. J., & Derwing, T. M. (1995). Foreign accent, comprehensibility, and intelligibility in the speech of second language learners. Language Learning, 45(1), 285–310.  https://doi.org/10.1111/j.1467-1770.1995.tb00963.x
  51. Munro, M. J., & Derwing, T. M. (1998). The effects of speaking rate on the comprehensibility of native and foreign accented speech. Language Learning, 48(2), 159–182. Retrieved January 10, 2018, from http://psycnet.apa.org/doi/10.1111/1467-9922.00038
  52. Munro, M. J., & Derwing, T. M. (2001). Modeling perceptions of the accentedness and comprehensibility of L2 speech: The role of speaking rate. Studies in Second Language Acquisition, 23(4), 451–468. Retrieved January 10, 2018, from https://www.cambridge.org/core/journals/studies-in-second-language-acquisition/article/modeling-perceptions-of-the-accentedness-and-comprehensibility-of-l2-speech-the-role-of-speaking-rate/E22C75E244694BF6D6E083149FBEFA51
  53. Munro, M. J., Derwing, T. M., & Morton, S. L. (2006). The mutual intelligibility of L2 speech. Studies in Second Language Acquisition, 28(1), 111–131.  https://doi.org/10.1017/S0272263106060049CrossRefGoogle Scholar
  54. Ockey, G. J., Koyama, D., Setoguchi, E., & Sun, A. (2015). The extent to which TOEFL iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students. Language Testing, 32(1), 39–62.  https://doi.org/10.1177/0265532214538014CrossRefGoogle Scholar
  55. Pennington, M. C. (1989). Teaching pronunciation from the top down. RELC Journal, 20(1), 20–38.  https://doi.org/10.1177/003368828902000103CrossRefGoogle Scholar
  56. Pennington, M. C. (1990). The context of L2 phonology. In H. Burmeister & P. Rounds (Eds.), Variability in second language acquisition: Proceedings of the tenth meeting of the second language research forum (pp. 541–564). Eugene, OR: University of Oregon. Available at https://books.google.com/books/about/Variability_in_second_language_acquisiti.html?id=wkYqAQAAIAAJ
  57. Pennington, M. C. (1992). Discourse factors in second language phonology: An exploratory study. In J. Leather & A. James (Eds.), New sounds 92, Proceedings of the 1992 Amsterdam Symposium on the Acquisition of Second-Language Speech, University of Amsterdam, April 1992 (pp. 137–155). Amsterdam: University of Amsterdam.Google Scholar
  58. Pennington, M. C. (1996). Phonology in English language teaching: An international approach. London and New York: Longman.Google Scholar
  59. Pennington, M. C. (2008). A strategic view of pronunciation in English as a second language. In G. Cane (Ed.), Strategies in language learning and teaching (pp. 104–115). Anthology Series 49. Singapore: SEAMEO Regional Language Centre.Google Scholar
  60. Pennington, M. C. (2015). Research, theory, and practice in second language phonology: A review and directions for the future. In J. A. Mompean & J. Fouz-González (Eds.), Investigating English pronunciation: Trends and directions (pp. 149–173). Basingstoke, UK and New York, NY: Palgrave Macmillan.CrossRefGoogle Scholar
  61. Piccardo, E. (2016) Common European Framework of Reference for Languages: Learning, teaching, assessment phonological scale revision process report. Retrieved February 26, 2018, from https://rm.coe.int/phonological-scale-revision-process-report-cefr/168073fff9
  62. Pinget, A.-F., Bosker, H. R., Quené, H., & de Jong, N. H. (2014). Native speakers’ perceptions of fluency and accent in L2 speech. Language Testing, 31(3), 349–365.  https://doi.org/10.1177/0265532214526177CrossRefGoogle Scholar
  63. Poulisse, N. (1999). Slips of the tongue: Speech errors in first and second language production. Amsterdam: John Benjamins.CrossRefGoogle Scholar
  64. Riggenbach, H. (1998). Evaluating learner interaction skills: Conversation at the micro level. In R. F. Young & A. W. He (Eds.), Talking and testing: Discourse approaches to the assessment of oral proficiency (pp. 53–67). Amsterdam: John Benjamins.CrossRefGoogle Scholar
  65. Sáfár, A., & Kormos, J. (2008). Revisiting problems with foreign language aptitude. IRAL—International Review of Applied Linguistics in Language Teaching, 46(2), 113–136.  https://doi.org/10.1515/IRAL.2008.005CrossRefGoogle Scholar
  66. Saito, K. (2018). Effects of sound, vocabulary and grammar learning aptitude on adult second language speech attainment in foreign language classrooms. Language Learning, 67.  https://doi.org/10.1111/lang.12244. Retrieved September 24, 2017, from Prepublication copy http://www.researchgate.net/publication/313337402_Effects_of_Sound_Vocabulary_and_Grammar_Learning_Aptitude_on_Adult_Second_Language_Speech_Attainment_in_Foreign_Language_Classrooms
  67. Saito, K., Trofimovich, P., & Isaacs, T. (2016). Second language speech production: Investigating linguistic correlates of comprehensibility and accentedness for learners at different ability levels. Applied Psycholinguistics, 37(2), 217–240.  https://doi.org/10.1017/S0142716414000502
  68. Saito, K., Trofimovich, P., & Isaacs, T. (2017). Using listener judgements to investigate linguistic influences on L2 comprehensibility and accentedness: A validation and generalization study. Applied Linguistics, 38(4), 439–462.  https://doi.org/10.1093/applin/amv047CrossRefGoogle Scholar
  69. Sawaki, Y., & Sinharay, S. (2013). Investigating the value of TOEFL iBT section scores. ETS Research Report RR-13-35, TOEFLiBT-21.Google Scholar
  70. Scharenborg, O. (2007). Reaching over the gap: A review of efforts to link human and automatic speech recognition research. Speech Communication, 49(5), 336–347.  https://doi.org/10.1016/j.specom.2007.01.009CrossRefGoogle Scholar
  71. Segalowitz, N. (2010). Cognitive bases of second language fluency. New York: Routledge.CrossRefGoogle Scholar
  72. Shavelson, R. J., Baxter, G. P., & Gao, X. (1993). Sampling variability of performance assessments. Journal of Educational Measurement, 30(3), 215–232.  https://doi.org/10.1111/j.1745-3984.1993.tb00424.x
  73. Skehan, P. (2001). Tasks and language performance assessment. In M. Bygate, P. Skehan, & M. Swain (Eds.), Researching pedagogic tasks: Second language learning, teaching, and testing (pp. 167–185). London: Longman.Google Scholar
  74. Skehan, P. (2007). Tradeoff and cognition: Two hypotheses regarding attention during task-based performance. Plenary delivered at the 2nd Annual Conference on Task-based Language Teaching, University of Hawaii, September 2007. Retrieved from http://www.hawaii.edu/tblt2007/PP/Plenaries/Skehan.ppt
  75. Skehan, P. (2014). Task-based performance and task-based instruction: Research contributions. Plenary delivered at the JALT Task-based Learning Special Interest Group meeting, Osaka, Japan. May 2014. Retrieved from http://www.tblsig.org/wp-content/uploads/2014/05/Skehan-Osaka-May-2014.pdf
  76. Smith, L., & Nelson, C. (1985). International intelligibility of English: Directions and resources. World Englishes, 4(3), 333–342.  https://doi.org/10.1111/j.1467-971X.1985.tb00423.x
  77. Suzuki, M., Balogh, J., & Bernstein, J. (2006). Psycholinguistics in computerized language testing. Paper presented at the AAAL/CAAL conference, June 20–26, Montreal. Retrieved December 1, 2017, from https://www.versanttest.com/technology/featured/6.AAAL2006.pdf
  78. Tarone, E. (1998). Research on interlanguage variation: Implications for language testing. In L. F. Bachman & A. D. Cohen (Eds.), Interfaces between second language acquisition and language testing research (pp. 71–111). New York: Cambridge University Press.Google Scholar
  79. Tarone, E., Bigelow, M., & Hansen, K. (2009). Literacy and second language oracy. Oxford and New York: Oxford University Press.Google Scholar
  80. Trofimovich, P., & Isaacs, T. (2017). Second language pronunciation assessment: A look at the present and the future. In T. Isaacs & P. Trofimovich (Eds.), Second language pronunciation assessment: Interdisciplinary perspectives (pp. 259–266). Bristol, UK and Blue Ridge Summit, PA: Multilingual Matters.Google Scholar
  81. van Doremalen, J., Cucchiarini, C., & Strik, H. (2010). Phoneme errors in read and spontaneous non-native speech: Relevance for CAPT system development. Interspeech Satellite Workshop. Second Language Studies: Acquisition, Learning, Education and Technology, Tokyo, Japan: Waseda University. Retrieved July 12, 2017, from http://www.gavo.t.u-tokyo.ac.jp/L2WS2010/papers/L2WS2010_O3-04.pdf
  82. van Lier, L. (1989). Reeling, writhing, drawling, stretching and fainting in coils: Oral proficiency interviews as conversation. TESOL Quarterly, 23(3), 489–508.  https://doi.org/10.2307/3586922CrossRefGoogle Scholar
  83. van Zanten, M. (2011). Evaluating the spoken English proficiency of international medical graduates for certification and licensure in the United States. In B. J. Hoekje & S. M. Tipton (Eds.), English language and the medical profession: Instructing and assessing the communication skills of international physicians (pp. 75–90). Bingley, UK; Leiden and Boston: Emerald and Brill.CrossRefGoogle Scholar
  84. Wennerstrom, A. (2000). The role of intonation in second language fluency. In H. Riggenbach (Ed.), Perspectives on fluency (pp. 102–127). Ann Arbor, MI: The University of Michigan Press.Google Scholar
  85. Winke, P. (2011). Evaluating the validity of a high-stakes ESL test: Why teachers’ perceptions matter. TESOL Quarterly, 45(4), 628–660.  https://doi.org/10.5054/tq.2011.268063CrossRefGoogle Scholar
  86. Winke, P., Gass, S., & Myford, C. (2012). Raters’ L2 background as a potential source of bias in rating oral performance. Language Testing, 30(2), 231–252.  https://doi.org/10.1177/0265532212456968
  87. Xi, X., Higgins, D., Zechner, K., & Williamson, D. (2012). A comparison of two scoring methods for an automated speech scoring system. Language Testing, 29(3), 371–394.  https://doi.org/10.1177/0265532211425673CrossRefGoogle Scholar
  88. Yates, L., Zielinski, E., and Pryor, E. (2011). The assessment of pronunciation and the new IELTS Pronunciation Scale. In IELTS Research Reports, 12 (pp. 23–68). Melbourne & Manchester: IDP IELTS Australia and British Council.Google Scholar
  89. Young, R. F. (2000). Interactional competence: Challenges for validity. Paper presented at a joint symposium on Interdisciplinary Interfaces with Language Testing, held at the annual meeting of the American Association for Applied Linguistics and the Language Testing Research Colloquium, March 2000, Vancouver. Retrieved December 1, 2017, from http://www.english.wisc.edu/rfyoung/IC_C4V.Paper.PDF
  90. Young, R. F., & Milanovic, M. (1992). Discourse variation in oral proficiency interviews. Studies in Second Language Acquisition, 14(4), 403–424.  https://doi.org/10.1017/S0272263100011207CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Authors and Affiliations

  • Martha C. Pennington
    • 1
  • Pamela Rogerson-Revell
    • 2
  1. 1.SOAS and Birkbeck CollegeUniversity of LondonLondonUK
  2. 2.EnglishUniversity of LeicesterLeicesterUK

Personalised recommendations