In this paper we describe ESMERALDA—an integrated Environment for Statistical Model Estimation and Recognition on Arbitrary Linear Data Arrays—which is a framework for building statistical recognizers operating on sequential data as, e.g., speech, handwriting, or biological sequences. ESMERALDA primarily supports continuous density Hidden Markov Models (HMMs) of different topologies and with user-definable internal structure. Furthermore, the framework supports the incorporation of Markov chain models (realized as statistical n-gram models) for long-term sequential restrictions and Gaussian mixture models (GMMs) for general classification tasks. ESMERALDA is used by several academic and industrial institutions. It was successfully applied to a number of challenging recognition problems in the fields of automatic speech recognition, offline handwriting recognition, and protein sequence analysis. The software is open source and can be retrieved under the terms of the LGPL.