Abstract
The evaluation of ASR technology and applications is approached from different perspectives. The recognition performance gives only a partial indication of user satisfaction: the entire manmachine interface should be considered. Many factors contribute to the variability of speech and affect the performance of recognizers. Some recommendations are expressed concerning the size and the content of databases distributed for training and test purposes. Techniques to model the variability of speech are proposed. They are implemented on a test workstation. Predicting the performance of a given recognizer in a particular situation is possible. It is argued that most of these techniques could also be adapted to improve the robustness of recognizers and speaker verifiers.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
G. Chollet, A. Astier, and M. Rossi. Evaluating the performance of speech recognizers at the acoustic-phonetic level. In IEEE-IASSP, Denver, 1981.
G. Chollet, F. Çapman, Ph. Bardaud, and Ch. Tadj. Measurement of the limits of the reference recogniser SAMREC1: noise addition and simulation of the lombard effect. In Final report SAM, June 1992.
G. Chollet, F. Çapman, Ch. Mokbel, and J. Farah Abou Daoud. On the evaluation of recognizers: statistical validity of the tests. In Final report SAM, June 1992.
G. Chollet and C. Gagnoulet. Evaluating speech recognizers and data bases using a reference system. In IEEE-ICASSP, Paris, 1982.
L. Gillick and S J. Cox. Some statistical issues in the comparison of speech recognition algorithms. In IEEE-ICASSP, pages 532–535, Glasgow, 1989.
J.Ph. Goldman and G. Chollet. Voice tranformations for the evaluation of speaker verification systems. In NATO-ASI, Bubion, 1993.
W.A. Lea. Procedures for evaluating the performances of commercial speech recognizers. In Speech Science Publication, Santa Barbara, 1981.
W.A. Lea. What causes speech recognizers to make mistakes? IEEE-ICASSP, 3:2030–2033, 1982.
J.N. Marcus. Significance tests for comparing speech recognizer performance using small test sets. EUROSPEECH, 2:465–468, 1989.
C. Montacié and G. Chollet. Système de reference pour revaluation d’ applications et la caractérisation de bases de données en reconnaissance automatique de la parole. In l6ème JEP, Hammamet, 1987.
R. Moore. Evaluating speech recognizers. IEEE-ICASSP, pages 178–185, 1977.
D.S. Pallett. Performance assessment of automatic speech recognizers. Journal of Research of the NBS, 90:5:371–387, 1985.
SAM. Multi-lingual Speech input/output Assessment Methodology and Standardisation. Final report, June 1992.
H. Steeneken and J. Van Velden. RAMOS: Recogniser assessment by means of manipulation of speech. Proc. EUROSPEECH, pages 316–319, 1989.
M.M. Taylor. Issues in the evaluation of speech recognition systems. In NATO RSG-10, DCIEM, Downsview, 1980.
H. Valbret, E. Moulines, and J. P. Tubach. Voice transformation using psola technique. Speech Communication, 11:175–187, 1992.
J. Woodard and J. Nelson. An information theoretic measure of speech recognition performance. Proc. Workshop on Standardizationfor Speech I/O, Mar. 1982.
S. J. Young. HTK: Hidden Markov Toolkit (Version 1.4). Cambridge University Engineering Department, Cambridge, England, 1991.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1995 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chollet, G. (1995). Evaluation of ASR Systems, Algorithms and Databases. In: Ayuso, A.J.R., Soler, J.M.L. (eds) Speech Recognition and Coding. NATO ASI Series, vol 147. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-57745-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-57745-1_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-63344-7
Online ISBN: 978-3-642-57745-1
eBook Packages: Springer Book Archive