Design and Analysis of a Database to Evaluate Children’s Reading Aloud Performance

  • Jorge Proença
  • Dirce Celorico
  • Carla Lopes
  • Miguel Sales Dias
  • Michael Tjalve
  • Andreas Stolcke
  • Sara Candeias
  • Fernando Perdigão
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9727)

Abstract

To evaluate the reading performance of children, human assessment is usually involved, where a teacher or tutor has to take time to individually estimate the performance in terms of fluency (speed, accuracy and expression). Automatic estimation of reading ability can be an important alternative or complement to the usual methods, and can improve other applications such as e-learning. Techniques must be developed to analyse audio recordings of read utterances by children and detect the deviations from the intended correct reading i.e. disfluencies. For that goal, a database of 284 European Portuguese children from 6 to 10 years old (1st–4th grades) reading aloud amounting to 20 h was collected in private and public Portuguese schools. This paper describes the design of the reading tasks as well as the data collection procedure. The presence of different types of disfluencies is analysed as well as reading performance compared to known curricular goals.

Keywords

Reading aloud performance Child speech Speech corpus Reading disfluencies 

Notes

Acknowledgements

This work was supported in part by Fundação para a Ciência e Tecnologia under the projects UID/EEA/50008/2013 (pluriannual funding in the scope of the LETSREAD project), and Marie Curie Action IRIS (ref. 610986, FP7-PEOPLE-2013-IAPP). Jorge Proença is supported by the SFRH/BD/97204/2013 FCT Grant. We would like to thank João de Deus, Bissaya Barreto and EBI de Pereira school associations and CASPAE parent’s association for collaborating in the database collection.

References

  1. 1.
    Abdou, S.M., Hamid, S.E., Rashwan, M., Samir, A., Abdel-Hamid, O., Shahin, M., Nazih, W.: Computer aided pronunciation learning system using speech recognition techniques. In: INTERSPEECH (2006)Google Scholar
  2. 2.
    Probst, K., Ke, Y., Eskenazi, M.: Enhancing foreign language tutors – in search of the golden speaker. Speech Commun. 37(3–4), 161–173 (2002)CrossRefMATHGoogle Scholar
  3. 3.
    Cincarek, T., Gruhn, R., Hacker, C., Nöth, E., Nakamura, S.: Automatic pronunciation scoring of words and sentences independent from the non-native’s first language. Comput. Speech Lang. 23(1), 65–88 (2009)CrossRefGoogle Scholar
  4. 4.
    Mostow, J., Roth, S.F., Hauptmann, A.G., Kane, M.: A prototype reading coach that listens. In: Proceedings of the Twelfth National Conference on Artificial Intelligence, Menlo Park, CA, USA, vol. 1, pp. 785–792 (1994)Google Scholar
  5. 5.
    Black, M., Tepperman, J., Lee, S., Price, P., Narayanan, S.: Automatic detection and classification of disfluent reading miscues in young children’s speech for the purpose of assessment. In: presented at the Proceedings of Interspeech, pp. 206–209 (2007)Google Scholar
  6. 6.
    Duchateau, J., Kong, Y.O., Cleuren, L., Latacz, L., Roelens, J., Samir, A., Demuynck, K., Ghesquière, P., Verhelst, W., hamme, H.V.: Developing a reading tutor: design and evaluation of dedicated speech recognition and synthesis modules. Speech Commun. 51(10), 985–994 (2009)CrossRefGoogle Scholar
  7. 7.
    Bolaños, D., Cole, R.A., Ward, W., Borts, E., Svirsky, E.: FLORA: fluent oral reading assessment of children’s speech. ACM Trans. Speech Lang. Process. 7(4), 16:1–16:19 (2011)CrossRefGoogle Scholar
  8. 8.
    Black, M.P., Tepperman, J., Narayanan, S.S.: Automatic prediction of children’s reading ability for high-level literacy assessment. Trans. Audio, Speech and Lang. Proc. 19(4), 1015–1028 (2011)CrossRefGoogle Scholar
  9. 9.
    Duchateau, J., Cleuren, L., hamme, H.V., Ghesquière, P.: Automatic assessment of children’s reading level. In: Proceedings of the Interspeech, Antwerp, Belgium, pp. 1210–1213 (2007)Google Scholar
  10. 10.
    ELRA: ELRA - ELRA-S0180: Portuguese Speecon database. http://catalog.elra.info/product_info.php?products_id=798. Accessed 06 May 2015
  11. 11.
    Lopes, C., Veiga, A., Perdigão, F.: A european portuguese children speech database for computer aided speech therapy. In: Caseli, H., Villavicencio, A., Teixeira, A., Perdigão, F. (eds.) PROPOR 2012. LNCS, vol. 7243, pp. 368–374. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  12. 12.
    Hämäläinen, A., Rodrigues, S., Júdice, A., Silva, S.M., Calado, A., Pinto, F.M., Dias, M.S.: The CNG corpus of european portuguese children’s speech. In: Habernal, I., Matoušek, V. (eds.) Text, Speech, and Dialogue, pp. 544–551. Springer, Heidelberg (2013)Google Scholar
  13. 13.
    Santos, A.L., Généreux, M., Cardoso, A., Agostinho, C., Abalada, S.: A corpus of European Portuguese child and child-directed speech. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2014), Reykjavik, Iceland (2014)Google Scholar
  14. 14.
    Hämäläinen, A., Cho, H., Candeias, S., Pellegrini, T., Abad, A., Tjalve, M., Trancoso, I., Dias, M.S.: Automatically recognising European Portuguese children’s speech. In: Baptista, J., Mamede, N., Candeias, S., Paraboni, I., Pardo, T.A., Volpe Nunes, MdG (eds.) PROPOR 2014. LNCS, vol. 8775, pp. 1–11. Springer, Heidelberg (2014)Google Scholar
  15. 15.
    Lee, S., Potamianos, A., Narayanan, S.: Acoustics of children’s speech: developmental changes of temporal and spectral parameters. J. Acoust. Soc. Am. 105(3), 1455–1468 (1999)CrossRefGoogle Scholar
  16. 16.
    Hämäläinen, A., Candeias, S., Cho, H., Meinedo, H., Abad, A., Pellegrini, T., Tjalve, M., Trancoso, I., Dias, M.S.: Correlating ASR errors with developmental changes in speech production: a study of 3–10-year-old European Portuguese children’s speech. In: Proceedings WOCCI 2014 – Workshop on Child Computer Interaction, Singapore, pp. 7–11 (2014)Google Scholar
  17. 17.
    Potamianos, A., Narayanan, S.: Robust recognition of children’s speech. IEEE Trans. Speech Audio Process. 11(6), 603–616 (2003)CrossRefGoogle Scholar
  18. 18.
    Buescu, H.C., Morais, J., Rocha, M.R., Magalhães, V.F.: Programa e Metas Curriculares de Portugês do Ensino Básico. Ministério da Educação e Ciência, May 2015Google Scholar
  19. 19.
    Soares, A.P., Medeiros, J.C., Simões, A., Machado, J., Costa, A., Iriarte, Á., de Almeida, J.J., Pinheiro, A.P., Comesaña, M.: ESCOLEX: a grade-level lexical database from European Portuguese elementary to middle school textbooks. Behav Res Methods 46(1), 240–253 (2014)CrossRefGoogle Scholar
  20. 20.
    Mendonça, G., Candeias, S., Perdigao, F., Shulby, C., Toniazzo, R., Klautau, A., Aluisio, S.: A method for the extraction of phonetically-rich triphone sentences. In: Proceedings of the International Telecommunications Symposium (ITS), São Paulo, Brazil, pp. 1–5 (2014)Google Scholar
  21. 21.
    Keuleers, E., Brysbaert, M.: Wuggy: a multilingual pseudoword generator. Behav Res Methods 42(3), 627–633 (2010)CrossRefGoogle Scholar
  22. 22.
    Rocha, P., Santos, D.: CETEMPúblico: Um corpus de grandes dimensões de linguagem jornalística portuguesa. In: Presented at the PROPOR, pp. 131–140 (2000)Google Scholar
  23. 23.
    Candeias, S., Celorico, D., Proença, J., Veiga, A., Perdigão, F.: HESITA(tions) in Portuguese: a database. In: ISCA, Interspeech Satellite Workshop on Disfluency in Spontaneous Speech - DiSS, KTH Royal Institute of Technology, Stockholm, Sweden, pp. 13–16 (2013)Google Scholar
  24. 24.
    Veiga, A., Celorico, D., Proença, J., Candeias, S., Perdigão, F.: Prosodic and phonetic features for speaking styles classification and detection. In: Torre Toledano, D., Ortega Giménez, A., Teixeira, A., González Rodríguez, J., Hernández Gómez, L., San Segundo Hernández, R., Ramos Castro, D. (eds.) IberSPEECH 2012. CCIS, vol. 328, pp. 89–98. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  25. 25.
    Hasbrouck, J., Tindal, G.A.: Oral reading fluency norms: a valuable assessment tool for reading teachers. Reading Teacher 59(7), 636–644 (2006)CrossRefGoogle Scholar
  26. 26.
    Pellegrini, T., Hämäläinen, A., de Mareüil, P.B., Tjalve, M., Trancoso, I., Candeias, S., Dias, M.S., Braga, D.: A corpus-based study of elderly and young speakers of European Portuguese: acoustic correlates and their impact on speech recognition performance. In: INTERSPEECH, pp. 852–856 (2013)Google Scholar
  27. 27.
    Hämäläinen, A., Avelar, J., Rodrigues, S., Dias, M.S., Kolesiński, A., Fegyó, T., Németh, G., Csobánka, P., Lan, K., Hewson, D.: The EASR Corpora of European Portuguese, French, Hungarian and Polish Elderly speech. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2014), Reykjavik, Iceland (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Jorge Proença
    • 1
    • 2
  • Dirce Celorico
    • 1
  • Carla Lopes
    • 1
    • 3
  • Miguel Sales Dias
    • 4
    • 5
  • Michael Tjalve
    • 6
  • Andreas Stolcke
    • 7
  • Sara Candeias
    • 4
  • Fernando Perdigão
    • 1
    • 2
  1. 1.Instituto de TelecomunicaçõesCoimbraPortugal
  2. 2.Department of Electrical and Computer EngineeringUniversity of CoimbraCoimbraPortugal
  3. 3.Polytechninc Institute of LeiriaLeiriaPortugal
  4. 4.Microsoft Language Development CentreLisbonPortugal
  5. 5.ISCTE – University Institute of LisbonLisbonPortugal
  6. 6.Microsoft and University of WashingtonSeattleUSA
  7. 7.Microsoft ResearchMountain ViewUSA

Personalised recommendations