© 2012

Supervised Sequence Labelling with Recurrent Neural Networks


Part of the Studies in Computational Intelligence book series (SCI, volume 385)

Table of contents

  1. Front Matter
    Pages 1-11
  2. Alex Graves
    Pages 1-3
  3. Alex Graves
    Pages 5-13
  4. Alex Graves
    Pages 15-35
  5. Alex Graves
    Pages 37-45
  6. Alex Graves
    Pages 47-56
  7. Alex Graves
    Pages 57-60
  8. Alex Graves
    Pages 61-93
  9. Alex Graves
    Pages 95-108
  10. Alex Graves
    Pages 109-131
  11. Back Matter
    Pages 0--1

About this book


Supervised sequence labelling is a vital area of machine learning, encompassing tasks such as speech, handwriting and gesture recognition, protein secondary structure prediction and part-of-speech tagging. Recurrent neural networks are powerful sequence learning tools—robust to input noise and distortion, able to exploit long-range contextual information—that would seem ideally suited to such problems. However their role in large-scale sequence labelling systems has so far been auxiliary. 


The goal of this book is a complete framework for classifying and transcribing sequential data with recurrent neural networks only. Three main innovations are introduced in order to realise this goal. Firstly, the connectionist temporal classification output layer allows the framework to be trained with unsegmented target sequences, such as phoneme-level speech transcriptions; this is in contrast to previous connectionist approaches, which were dependent on error-prone prior segmentation. Secondly, multidimensional recurrent neural networks extend the framework in a natural way to data with more than one spatio-temporal dimension, such as images and videos. Thirdly, the use of hierarchical subsampling makes it feasible to apply the framework to very large or high resolution sequences, such as raw audio or video.


Experimental validation is provided by state-of-the-art results in speech and handwriting recognition.


Computational Intelligence Neural Networks Recurrent Neural Networks Sequence Labelling

Authors and affiliations

  1. 1., Department of Computer ScienceUniversity of TorontoTorontoCanada

Bibliographic information

  • Book Title Supervised Sequence Labelling with Recurrent Neural Networks
  • Authors Alex Graves
  • Series Title Studies in Computational Intelligence
  • DOI
  • Copyright Information Springer-Verlag GmbH Berlin Heidelberg 2012
  • Publisher Name Springer, Berlin, Heidelberg
  • eBook Packages Engineering Engineering (R0)
  • Hardcover ISBN 978-3-642-24796-5
  • Softcover ISBN 978-3-642-43218-7
  • eBook ISBN 978-3-642-24797-2
  • Series ISSN 1860-949X
  • Series E-ISSN 1860-9503
  • Edition Number 1
  • Number of Pages XIV, 146
  • Number of Illustrations 0 b/w illustrations, 0 illustrations in colour
  • Topics Computational Intelligence
    Artificial Intelligence
  • Buy this book on publisher's site