Language Resources and Evaluation

, Volume 41, Issue 2, pp 129–146

Validation of phonetic transcriptions in the context of automatic speech recognition

  • Christophe Van Bael
  • Henk van den Heuvel
  • Helmer Strik
Article

DOI: 10.1007/s10579-007-9033-9

Cite this article as:
Van Bael, C., van den Heuvel, H. & Strik, H. Lang Resources & Evaluation (2007) 41: 129. doi:10.1007/s10579-007-9033-9

Abstract

Some of the speech databases and large spoken language corpora that have been collected during the last fifteen years have been (at least partly) annotated with a broad phonetic transcription. Such phonetic transcriptions are often validated in terms of their resemblance to a handcrafted reference transcription. However, there are at least two methodological issues questioning this validation method. First, no reference transcription can fully represent the phonetic truth. This calls into question the status of such a transcription as a single reference for the quality of other phonetic transcriptions. Second, phonetic transcriptions are often generated to serve various purposes, none of which are considered when the transcriptions are compared to a reference transcription that was not made with the same purpose in mind. Since phonetic transcriptions are often used for the development of automatic speech recognition (ASR) systems, and since the relationship between ASR performance and a transcription’s resemblance to a reference transcription does not seem to be straightforward, we verified whether phonetic transcriptions that are to be used for ASR development can be justifiably validated in terms of their similarity to a purpose-independent reference transcription. To this end, we validated canonical representations and manually verified broad phonetic transcriptions of read speech and spontaneous telephone dialogues in terms of their resemblance to a handcrafted reference transcription on the one hand, and in terms of their suitability for ASR development on the other hand. Whereas the manually verified phonetic transcriptions resembled the reference transcription much closer than the canonical representations, the use of both transcription types yielded similar recognition results. The difference between the outcomes of the two validation methods has two implications. First, ASR developers can save themselves the effort of collecting expensive reference transcriptions in order to validate phonetic transcriptions of speech databases or spoken language corpora. Second, phonetic transcriptions should preferably be validated in terms of the application they will serve because a higher resemblance to a purpose-independent reference transcription is no guarantee for a transcription to be better suited for ASR development.

Keywords

Broad phonetic transcriptionsValidationAutomatic speech recognition

Abbreviations

ASR

Automatic speech recognition

CGN

Corpus Gesproken Nederlands—Spoken Dutch Corpus

MPT

Manual phonetic transcription

RT

Reference transcription

WER

Word error rate

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  • Christophe Van Bael
    • 1
  • Henk van den Heuvel
    • 1
  • Helmer Strik
    • 1
  1. 1.Centre for Language and Speech Technology Radboud University NijmegenNijmegenThe Netherlands