Multimedia Tools and Applications

, Volume 72, Issue 1, pp 21–40

An automatic caption alignment mechanism for off-the-shelf speech recognition technologies


DOI: 10.1007/s11042-012-1318-3

Cite this article as:
Federico, M. & Furini, M. Multimed Tools Appl (2014) 72: 21. doi:10.1007/s11042-012-1318-3


With a growing number of online videos, many producers feel the need to use video captions in order to expand content accessibility and face two main issues: production and alignment of the textual transcript. Both activities are expensive either for the high labor of human resources or for the employment of dedicated software. In this paper, we focus on caption alignment and we propose a novel, automatic, simple and low-cost mechanism that does not require human transcriptions or special dedicated software to align captions. Our mechanism uses a unique audio markup and intelligently introduces copies of it into the audio stream before giving it to an off-the-shelf automatic speech recognition (ASR) application; then it transforms the plain transcript produced by the ASR application into a timecoded transcript, which allows video players to know when to display every single caption while playing out the video. The experimental study evaluation shows that our proposal is effective in producing timecoded transcripts and therefore it can be helpful to expand video content accessibility.


Automatic caption alignment Speech recognition Off-the-shelf ASR 

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  1. 1.Servizio Accoglienza Studenti DisabiliUniversità di Modena e Reggio EmiliaModenaItaly
  2. 2.Dipartimento di Comunicazione ed EconomiaUniversità di Modena e Reggio EmiliaReggio EmiliaItaly

Personalised recommendations