Language Resources and Evaluation

, Volume 44, Issue 4, pp 371–386

Remote-based text-to-speech modules’ evaluation framework: the RES framework


DOI: 10.1007/s10579-009-9110-3

Cite this article as:
Rojc, M., Höge, H. & Kačič, Z. Lang Resources & Evaluation (2010) 44: 371. doi:10.1007/s10579-009-9110-3


The ECESS consortium (European Center of Excellence in Speech Synthesis) aims to speed up progress in speech synthesis technology, by providing an appropriate evaluation framework. The key element of the evaluation framework is based on the partition of a text-to-speech synthesis system into distributed TTS modules. A text processing, prosody generation, and an acoustic synthesis module have been specified currently. A split into various modules has the advantage that the developers of an institution active in ECESS, can concentrate its efforts on a single module, and test its performance in a complete system using missing modules from the developers of other institutions. In this way, complete TTS systems can be built using high performance modules from different institutions. In order to evaluate the modules and to connect modules efficiently, a remote evaluation platform—the Remote Evaluation System (RES) based on the existing internet infrastructure—has been developed within ECESS. The RES is based on client–server architecture. It consists of RES module servers, which encapsulate the modules of the developers, a RES client, which sends data to and receives data from the RES module servers, and a RES server, which connects the RES module servers, and organizes the flow of information. RES can be used by developers for selecting RES module from the internet, which contains a missing TTS module needed to test and improve the performances of their own modules. Finally, the RES allows for the evaluation of TTS modules running at different institutions worldwide. When using the RES client, the institution performing the evaluation is able to set-up and performs various evaluation tasks by sending test data via the RES client and receiving results from the RES module servers. Currently ELDA is setting-up an evaluation using the RES client, which will then be extended to an evaluation client specializing in the envisaged evaluation tasks.


Remote text-to-speech synthesis evaluation Text-to-speech synthesis modules ECESS consortium 

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  1. 1.Faculty of Electrical Engineering and Computer ScienceUniversity of MariborMariborSlovenia
  2. 2.IC 5Siemens AG, Corporate TechnologyMünchenGermany

Personalised recommendations